Best option to transfer 200TB of data?

0

What will be the best option amongst 'S3 Batch', 'Cross Account Replication' and 'Datasync' to transfer 200TB of data from one account to another? Also, how much time will it take to transfer 200TB of data in each of the above services? Suggestions except the above mentioned services are also welcomed.

posta un anno fa580 visualizzazioni
2 Risposte
5
Risposta accettata

Greetings! For transferring 200TB of data between two AWS accounts, the best option would be AWS DataSync. The reason being:

AWS DataSync is designed specifically for transferring large amounts of data at high speed, typically up to 10 times faster than open-source tools. It provides robust error handling, encryption, and built-in task scheduling. The time it would take to transfer 200TB of data would depend on the network speed between your two AWS accounts, which can vary greatly. AWS DataSync can achieve speeds up to 10 Gbps, so assuming you can achieve the full 10 Gbps speed, it would take a bit over 18 days to transfer 200TB of data. But, this is highly dependent on your network conditions and could take longer. Alternative options could be:

AWS Snowball: If you don't need your data to be online immediately and can afford a longer transfer time, you could consider using AWS Snowball. This is a physical device that AWS will ship to you, which you can load with your data and then ship back to AWS. They'll then upload the data to your account. The process can take a week or two (or longer), depending on shipping times. https://aws.amazon.com/datasync/

https://aws.amazon.com/snowball/

I hope it is helpful. Let me know if you need any further questions.

Cheers,

ZJon

AWS
ESPERTO
ZJon
con risposta un anno fa
  • Both the AWS Accounts are in the same Tokyo Region. How much time would it take to transfer the same amount of data,i.e., 200TB? Also, would the transfer require our own bandwidth?

  • Hi Dipesh, It would not require your on prem bandwidth since it is getting transferred from 1 AWS account to another one.

  • How much will it cost to transfer this amount of data, i.e., 200TB using datasync?

  • The onetime cost is about 2560 USD

0

Hi, this article analyzes all possibilities (yours above and others) and gives guidance to optimize: https://repost.aws/knowledge-center/s3-large-transfer-between-buckets

profile pictureAWS
ESPERTO
con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande