AWS DataSync Transfer Estimate

1

Hi, Is there a way to calculate/estimate how long it will take for AWS Datasync to migrate data from on-prem to AWS? I know internet condition and disk I/O will affect the transfer time, but is there a general estimate we can get assuming all "under normal conditions" Example 10TB data - max file size 500KB - Bandwidth 1Gbps

Thank you.

Aman
질문됨 일 년 전1801회 조회
2개 답변
10
수락된 답변

Greetings

We can estimate the time it takes to transfer 10TB of data using the following steps:

Solution.

Yes, there are ways to estimate how long it will take for AWS DataSync to migrate data from on-premises to AWS. As you noted, several factors can affect transfer time, including the amount of data, the maximum file size, the available bandwidth, and any network latency.

  1. Convert the bandwidth from 1Gbps to gigabytes per second (GB/s). There are 8 bits in a byte, so 1Gbps is equal to 0.125 GB/s.

  2. Divide the total data size (10TB or 10,000 GB) by the bandwidth (0.125 GB/s) to get the transfer time in seconds.

  3. The calculation will be:

10,000 GB / 0.125 GB/s = 80,000 seconds

So once you convert it to hours it will be around 22 hours .

Let me know if I answered your question or if you have any follow-up

Reference: https://aws.amazon.com/blogs/storage/simplify-and-accelerate-your-data-migration-using-aws-datasync-discovery/\

AWS
전문가
ZJon
답변함 일 년 전
  • Thank you very much Zokir. I got it. Just a follow-up question - can we safely say that beyond the estimate you detailed, the AWS designed transfer protocol with parallel, multi-threaded architecture can transfer the data significantly faster that what is estimated of course "under normal conditions"? I am asking this kind of small things to always to be safe on the things I say or assume. Thank you

  • Yes, AWS DataSync is designed to transfer data in a parallel and multi-threaded manner, which can help improve transfer performance and make the transfer process more efficient. It's worth noting that the actual transfer performance can still be affected by various factors, including the network conditions and any potential bottlenecks in the source or destination systems. However, in general, AWS DataSync's parallel and multi-threaded architecture can help to transfer data more quickly and efficiently than a single-threaded transfer method.

  • There is another way you can look at your question: If it is not parallel transfer protocols, it can go way longer than 22 hours. I would always give an estimate, because the outcome varies based on a range of factors :)

  • Did that help with your question @Aman?

1

You might try DataSync Discovery which is in preview. See: Simplify and accelerate your data migration using AWS DataSync Discovery.

profile pictureAWS
전문가
kentrad
답변함 일 년 전
  • Thank you. I am looking OnTAP. I am totally unfamiliar with it and will do some digging if it can work with regular NFS storages.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠