AWS DataSync Transfer Estimate

1

Hi, Is there a way to calculate/estimate how long it will take for AWS Datasync to migrate data from on-prem to AWS? I know internet condition and disk I/O will affect the transfer time, but is there a general estimate we can get assuming all "under normal conditions" Example 10TB data - max file size 500KB - Bandwidth 1Gbps

Thank you.

Aman
asked a year ago1313 views
2 Answers
10
Accepted Answer

Greetings

We can estimate the time it takes to transfer 10TB of data using the following steps:

Solution.

Yes, there are ways to estimate how long it will take for AWS DataSync to migrate data from on-premises to AWS. As you noted, several factors can affect transfer time, including the amount of data, the maximum file size, the available bandwidth, and any network latency.

  1. Convert the bandwidth from 1Gbps to gigabytes per second (GB/s). There are 8 bits in a byte, so 1Gbps is equal to 0.125 GB/s.

  2. Divide the total data size (10TB or 10,000 GB) by the bandwidth (0.125 GB/s) to get the transfer time in seconds.

  3. The calculation will be:

10,000 GB / 0.125 GB/s = 80,000 seconds

So once you convert it to hours it will be around 22 hours .

Let me know if I answered your question or if you have any follow-up

Reference: https://aws.amazon.com/blogs/storage/simplify-and-accelerate-your-data-migration-using-aws-datasync-discovery/\

AWS
EXPERT
ZJon
answered a year ago
  • Thank you very much Zokir. I got it. Just a follow-up question - can we safely say that beyond the estimate you detailed, the AWS designed transfer protocol with parallel, multi-threaded architecture can transfer the data significantly faster that what is estimated of course "under normal conditions"? I am asking this kind of small things to always to be safe on the things I say or assume. Thank you

  • Yes, AWS DataSync is designed to transfer data in a parallel and multi-threaded manner, which can help improve transfer performance and make the transfer process more efficient. It's worth noting that the actual transfer performance can still be affected by various factors, including the network conditions and any potential bottlenecks in the source or destination systems. However, in general, AWS DataSync's parallel and multi-threaded architecture can help to transfer data more quickly and efficiently than a single-threaded transfer method.

  • There is another way you can look at your question: If it is not parallel transfer protocols, it can go way longer than 22 hours. I would always give an estimate, because the outcome varies based on a range of factors :)

  • Did that help with your question @Aman?

1

You might try DataSync Discovery which is in preview. See: Simplify and accelerate your data migration using AWS DataSync Discovery.

profile pictureAWS
EXPERT
kentrad
answered a year ago
  • Thank you. I am looking OnTAP. I am totally unfamiliar with it and will do some digging if it can work with regular NFS storages.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions