How to migrate data to another cluster?

[elpsycongroo]

We previously set up a self-hosted Milvus cluster and are now looking to migrate to a cloud-provided Milvus service (aliyun). How should we proceed with migrating the existing data?

[xiaofan-luan]

if you are using zilliz cloud you can check our migration service https://docs.zilliz.com.cn/docs/migrate-from-milvus.

For aliyun managed milvus maybe you should ask the to help.
Check https://github.com/zilliztech/vts see if it can help you to do the migration

[elpsycongroo]

Thanks, we checked VTS but couldn't find documentation about the Milvus connector. Has it not been provided yet?

[nianliuu]

@elpsycongroo just follow the QuickStart Guide, it has config example for milvus migration

[yhmo]

If both of your source/target milvus are self-deployed, milvus-backup is the recommended tool: https://milvus.io/docs/milvus_backup_overview.md

This tool can copy the data files of the source milvus to s3/minio, and import the data files to the target milvus from s3/minio.
Here is a manual book:
milvus-backup tool practice.pdf

Basically, there are 4 use cases:

• backup/restore in one milvus
• backup/restore in two milvus in one bucket of one s3/minio
• backup/restore in two milvus in different buckets of one s3/minio
• backup/restore in two milvus in different s3/minio

Your use case is backup/restore in two milvus in different s3/minio, you can read the section "Two Milvus in two S3​".

[elpsycongroo]

Thanks, we will migrate to the Milvus managed service provided by Alibaba Cloud.
That means, we cannot use the s3 service directly, so another way is needed, right?

[yhmo]

I suppose they have migration tool provided, but I have no idea about the ability of the tool they provided.

[elpsycongroo]

Thanks for your reply