环境
milvus:v2.2.4
go:1.20.2 darwin/amd64
milvus-backup:v0.2.2
代码准备
https://github.com/zilliztech/milvus-backup/releases
如果你的milvus是2.2.9版本及以上,可以直接下载最新的版本:git clone https://github.com/zilliztech/milvus-backup.git
默认使用的配置文件在config目录下,如果不需要手动指定,直接修改改文件即可,主要修改milvus和minio相关配置即可
示例:
# Configures the system log output.
log:
level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
console: true
file:
rootPath: "logs/backup.log"
http:
simpleResponse: true
# milvus proxy address, compatible to milvus.yaml
milvus:
address: milvus-t-milvus.milvus.svc.t2.test.xdf.cn
port: 19530
authorizationEnabled: false
# tls mode values [0, 1, 2]
# 0 is close, 1 is one-way authentication, 2 is two-way authentication.
tlsMode: 0
user: xxxx
password: xxxx
# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
address: minio.openai.svc.t2.test.xdf.cn # Address of MinIO/S3
port: 80 # Port of MinIO/S3
accessKeyID: xxxx # accessKeyID of MinIO/S3
secretAccessKey: xxxx # MinIO/S3 encryption string
useSSL: false # Access to MinIO/S3 with SSL
useIAM: false
cloudProvider: "aws"
iamEndpoint: ""
bucketName: "milvus-t" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance
backupBucketName: "milvus-t-backup-all" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath
backupRootPath: "files" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath
backup:
maxSegmentGroupSize: 2G
备份
构建/运行
进入项目根目录,依次执行下面的命令:
go get
go build
会在根目录下生成可执行文件:
Zhurunhua-2:milvus-backup-0.2.2 xdf$ ./milvus-backup help
milvus-backup is a backup tool for milvus.
Usage:
milvus-backup [flags]
milvus-backup [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
create create subcommand create a backup.
delete delete subcommand delete backup by name.
get get subcommand get backup by name.
help Help about any command
list list subcommand shows all backup in the cluster.
restore restore subcommand restore a backup.
server server subcommand start milvus-backup RESTAPI server.
version print the version of Milvus backup tool
Flags:
--config string config YAML file of milvus (default "backup.yaml")
-h, --help help for milvus-backup
Use "milvus-backup [command] --help" for more information about a command.
Zhurunhua-2:milvus-backup-0.2.2 xdf$
备份需要使用create命令:
示例:指定Collection进行备份
./milvus-backup create --colls tk_td_question_v2 -n tiku_backup
总共19G数据,大概几分钟就完事了
验证
从minio-console可以看到,对应的备份桶已经创建了,数据也备份好了:
binlogs中和源文件一直,meta中会生成一些元数据:
恢复
TODO
遇到的问题
我刚开始下载的是最新的源码
- this version of sdk is incompatible with server, please downgrade your sdk or upgrade your server
文章来源:https://www.toymoban.com/news/detail-605233.html
解决:milvus版本过低,与源码所支持的不一致,下载非最新版本的milvus-backup解决。文章来源地址https://www.toymoban.com/news/detail-605233.html
到了这里,关于【Milvus】记录一次基于milvus-backup做的Milvus备份与恢复的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!