image-syncer-Toy模板网

这篇具有很好参考价值的文章主要介绍了image-syncer。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

github

1. 简介

一个简单、易用的批量镜像迁移和镜像同步复制工具，支持几乎所有目前主流的基于Docker Registry V2搭建的镜像存储服务，例如ACR、Docker、Hub、Quay、自建Harbor等，目前已经初步经过了TB级别的生产环境镜像迁移验证

特性
1. 支持多对多镜像仓库同步
2. 支持基于 Docker Registry V2 搭建的 docker 镜像仓库服务 (如 Docker Hub、 Quay、阿里云镜像服务 ACR、 Harbor等)
3. 同步只经过内存和网络，不依赖磁盘存储，同步速度快
4. 增量同步, 通过对同步过的镜像 blob 信息落盘，不重复同步已同步的镜像
5. 并发同步，可以通过配置文件调整并发数
6. 自动重试失败的同步任务，可以解决大部分镜像同步中的网络抖动问题
7. 不依赖 docker 以及其他程序

2.安装

wget https://github.com/AliyunContainerService/image-syncer/releases/download/v1.0.3/image-syncer-v1.0.3-linux-amd64.tar.gz
tar zxf image-syncer-v1.0.3-linux-amd64.tar.gz && mv image-syncer /usr/local/bin/

参数

-h  --help       usage information

    --config     set the path of config file, this file need to be created before starting synchronization, default
                 config file is at "current/working/directory/config.json". (This flag can be replaced with flag --auth
                 and --images which for better orgnization.)

    --auth       set the path of authentication file, this file need to be created before starting synchronization, default
                 config file is at "current/working/directory/auth.json". This flag need to be pair used with --images.

    --images     set the path of image rules file, this file need to be created before starting synchronization, default
                 config file is at "current/working/directory/images.json". This flag need to be pair used with --auth.

    --log        set the path of log file, logs will be printed to Stderr by default 

    --namespace  set default-namespace, default-namespace can also be set by environment variable "DEFAULT_NAMESPACE",
                 if they are both set at the same time, "DEFAULT_NAMESPACE" will not work at this synchronization,
                 default-namespace will work only if default-registry is not empty.

    --registry   set default-registry, default-registry can also be set by environment variable "DEFAULT_REGISTRY",
                 if they are both set at the same time, "DEFAULT_REGISTRY" will not work at this synchronization, 
                 default-registry will work only if default-namespace is not empty.

    --proc       number of goroutines, default value is 5

    --records    image-syncer will record the information of synchronized image blobs to a disk file, this parameter will
                 set the path of the records file, default path is "current/working/directory/records", a records file can be 
                 reused to make incremental synchronization if it is really generated by yourself. image-syncer remove the 
                 dependence of records file after v1.1.0

    --retries    number of retries, default value is 2, the retries of failed sync tasks will start after all sync tasks
                 are executed once, reties of failed sync tasks will resolve most occasional network problems during 
                 synchronization

    --os         os list to filter source tags, not works for docker v2 schema1 media, takes no effect if empty
  
    --arch       architecture list to filter source tags, takes no effect if empty

3. 使用

3.1 Harbor To Aliyun ACR

配置文件模板
```
{
    "auth": {                   // 认证字段，其中每个对象为一个registry的一个账号和
                                // 密码；通常，同步源需要具有pull以及访问tags权限，
                                // 同步目标需要拥有push以及创建仓库权限，如果没有提供，则默认匿名访问
        "quay.io": {            // registry的url，需要和下面images中对应registry的url相同
            "username": "xxx",               // 用户名，可选
            "password": "xxxxxxxxx",         // 密码，可选
            "insecure": true                 // registry是否是http服务，如果是，insecure字段需要为true
        },
        "registry.cn-beijing.aliyuncs.com": {
            "username": "xxx",
            "password": "xxxxxxxxx"
        },
        "registry.hub.docker.com": {
            "username": "xxx",
            "password": "xxxxxxxxxx"
        }
    },
    images: {
    	        "quay.io/coreos/kube-rbac-proxy": "quay.io/ruohe/kube-rbac-proxy",
        	"xxxx":"xxxxx",
        	"xxx/xxx/xx:tag1,tag2,tag3":"xxx/xxx/xx"
    }
}
```
1、同步镜像规则字段，其中一条规则包括一个源仓库（键）和一个目标仓库（值）

2、同步的最大单位是仓库（repo），不支持通过一条规则同步整个namespace以及registry

3、源仓库和目标仓库的格式与docker pull/push命令使用的镜像url类似（registry/namespace/repository:tag）

4、源仓库和目标仓库（如果目标仓库不为空字符串）都至少包含registry/namespace/repository

5、源仓库字段不能为空，如果需要将一个源仓库同步到多个目标仓库需要配置多条规则

6、目标仓库名可以和源仓库名不同（tag也可以不同），此时同步功能类似于：docker pull + docker tag + docker push

7、当源仓库字段中不包含tag时，表示将该仓库所有tag同步到目标仓库，此时目标仓库不能包含tag 8、当源仓库字段中包含tag时，表示只同步源仓库中的一个tag到目标仓库，如果目标仓库中不包含tag，则默认使用源tag 9、源仓库字段中的tag可以同时包含多个（比如"a/b/c:1,2,3"），tag之间通过","隔开，此时目标仓库不能包含tag，并且默认使用原来的tag

10、当目标仓库为空字符串时，会将源镜像同步到默认registry的默认namespace下，并且repo以及tag与源仓库相同，默认registry和默认namespace可以通过命令行参数以及环境变量配置，参考下面的描述

创建配置文件：

{
    "auth": {
        "172.20.5.10": {
            "username": "admin",
            "password": "Xxzx@789",
            "insecure": true
        },
        "registry.cn-beijing.aliyuncs.com": {
            "username": "352@qq.com",
            "password": "docker"
        }
    },
    "images": {
        "172.20.5.10/library/kube-apiserver": ""
    }
}

开始迁移：

# 并发数为10，重试次数为10
# 日志输出到./log文件下，不存在会自动创建，不指定的话默认会将日志打印到Stderr
# 指定配置文件为harbor-to-acr.json，内容如上所述

image-syncer --proc=10 --config=./harbor_to_acr.json --registry=registry.cn-beijing.aliyuncs.com --namespace=cloud-devops --retries=10 --log=./log  
Start to generate sync tasks, please wait ...
Start to handle sync tasks, please wait ...
Finished, 0 sync tasks failed, 0 tasks generate failed

日志打印

{"level":"info","msg":"Get tags of 172.20.5.10/library/kube-apiserver successfully: [v1.20.1]","time":"2021-01-08 10:54:23"}
{"level":"info","msg":"Generate a task for 172.20.5.10/library/kube-apiserver:v1.20.1 to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1","time":"2021-01-08 10:54:23"}
{"level":"info","msg":"Get manifest from 172.20.5.10/library/kube-apiserver:v1.20.1","time":"2021-01-08 10:54:23"}
{"level":"info","msg":"Get a blob sha256:f398b465657ed53ee83af22197ef61be9daec6af791c559ee5220dee5f3d94fe(647423) from 172.20.5.10/library/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:23"}
{"level":"info","msg":"Put blob sha256:f398b465657ed53ee83af22197ef61be9daec6af791c559ee5220dee5f3d94fe(647423) to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Get a blob sha256:cbcdf8ef32b41cd954f25c9d85dee61b05acc3b20ffa8620596ed66ee6f1ae1d(723022) from 172.20.5.10/library/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Put blob sha256:cbcdf8ef32b41cd954f25c9d85dee61b05acc3b20ffa8620596ed66ee6f1ae1d(723022) to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Get a blob sha256:d7d21f5bdd8303a60bac834f99867a58e6f3e1abcb6d486158a1ccb67dbf85bf(29033259) from 172.20.5.10/library/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Put blob sha256:d7d21f5bdd8303a60bac834f99867a58e6f3e1abcb6d486158a1ccb67dbf85bf(29033259) to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Get a blob sha256:75c7f711208082c548b935ab31e681ea30acccdce6b7abeecabae5bbfd326627(2630) from 172.20.5.10/library/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:24"}
{"level":"info","msg":"Put blob sha256:75c7f711208082c548b935ab31e681ea30acccdce6b7abeecabae5bbfd326627(2630) to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1 success","time":"2021-01-08 10:54:25"}
{"level":"info","msg":"Put manifest to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1","time":"2021-01-08 10:54:25"}
{"level":"info","msg":"Synchronization successfully from 172.20.5.10/library/kube-apiserver:v1.20.1 to registry.cn-beijing.aliyuncs.com/cloud-devops/kube-apiserver:v1.20.1","time":"2021-01-08 10:54:25"}
{"level":"info","msg":"Finished, 0 sync tasks failed, 0 tasks generate failed","time":"2021-01-08 10:54:25"}

3.2 Harbor To Huawei SWR

创建配置文件

{
    "auth": {
        "172.20.5.10": {
            "username": "admin",
            "password": "Xxzx@789",
            "insecure": true
        },
        "swr.cn-east-3.myhuaweicloud.com": {
            "username": "cn-east-3@8B5B8GC",
            "password": "8b0f1e69f2"
        }
    },
    "images": {
        "172.20.5.10/library/kube-apiserver": "",
        "172.20.5.10/library/controller": ""
    }
}

启动任务

image-syncer --proc=10 --config=./harbor_to_swr.json --registry=swr.cn-east-3.myhuaweicloud.com --namespace=cloud-devops --retries=10 --log=./log
Start to generate sync tasks, please wait ...
Start to handle sync tasks, please wait ...
Finished, 0 sync tasks failed, 0 tasks generate failed

3.3 Huawei SWR To Aliyun ACR

创建日志文件

{
    "auth": {
        "swr.cn-east-3.myhuaweicloud.com": {
            "username": "cn-east-3@8B5B8GC",
            "password": "8b0f1e6a46af8dff0ae519a09f2"
        },
        "registry.cn-beijing.aliyuncs.com": {
            "username": "352@qq.com",
            "password": "docker"
        }
    },
    "images": {
        "swr.cn-east-3.myhuaweicloud.com/cloud-devops/kube-apiserver": "",
        "swr.cn-east-3.myhuaweicloud.com/cloud-devops/controller": ""
    }
}

开始迁移

image-syncer --proc=10 --config=./harbor_to_swr.json --registry=registry.cn-beijing.aliyuncs.com --namespace=cloud-devops --retries=10 --log=./log
Start to generate sync tasks, please wait ...
Start to handle sync tasks, please wait ...
Finished, 0 sync tasks failed, 0 tasks generate failed

‍文章来源地址https://www.toymoban.com/news/detail-467895.html

到了这里，关于image-syncer的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！