Fastest way to sync two FSx storage

Question

We currently have 2 FSx 10TB storage volumes. Let's call them A and B.
FSx A is MultiAZ SSD and FSx B is SingleAZ SSD.
There are over 10M small files on FSx A, for a total of 3TB of data. 
FSx A is used 24/7 in production and FSx B is unsused and empty.
We need to synchronize all the files from A to B to eventually switch from FSx A to FSx B.
Our application is not capable of using two storages, so we need to stop production when switching from FSx A to FSx B.

We are planning on using the DataSync service, but I am wondering if there is another service I am not aware of that would mirror both FSx so when we switch from FSx A to FSx B the downtime will be minimized.

Is there another service you suggest to use?

Thank you very much.

Ather Beg · Answer

Also consider robocopy because you can take advantage of parallelism using multiple thread e.g. robocopy \source-fsx\share \destination-fsx\share /E /MT:16  
I've seen people using up to 96 threads and if sufficient connectivity is available, it could result in drastic improvement in transfer speed.
It could be worth running a test using a subset of data to see the speeds achieved.

Hope this helps!

Frederik · Answer

Thank you again, so if we rule out the backup-restore solution, would the next best one be the use of the DataSync service?

monroery · Answer

Considering you want to move from Multi AZ FsX to Single AZ, and you cannot restore a multi AZ FSx backup to a single AZ cfg,  I would say DataSync is your best remaining solution.

If this helped please remember to mark this answer as accepted. Thanks and good luck.

Frederik · Answer

Thank you for your answer.
We tried creating FSx B from the backup, but the interface would not let us create a SingleAZ Fsx from the FSx A MultiAZ backup. The interface forces a MultiAZ FSx.
Is there a way to work around this limitation?
Thank you again,

monroery · Answer

You should be able to create a backup of FSx A and restore it to FSx B. This can help reduce downtime, as the initial data transfer happens while FSx A is still running. The switch to FSx B would only require downtime during the final synchronization of any changes made after the backup.

https://docs.aws.amazon.com/fsx/latest/WindowsGuide/how-to-restore-backups.html

If this helped please remember to mark this answer as accepted. Thanks and good luck.

https://docs.aws.amazon.com/fsx/latest/WindowsGuide/how-to-restore-backups.html

If this helped please remember to mark this answer as accepted. Thanks and good luck.

Fastest way to sync two FSx storage

Add your answer

Relevant content