Skip to content

S3Clacier to S3 Data Copy Time to Complete?

0

Hi Community, I use Clacier to archive data and am definitely not an IT expert. I commenced a copy of the data to an S3 Bucket using Amazons 'DataTransferS3GlacierVaultsToS3' template and all appears to be progressing. I was wondering if someone would have an idea how long 1.5TB of data will take as it has been goig for 48 hours and still not complete / nothing showing in the S3 bucket. Many thanks

asked 2 years ago309 views

1 Answer
0
Accepted Answer

The time it takes to transfer 1.5TB of data from S3 Glacier to S3 can vary, but it's important to note that the process can be quite lengthy. According to AWS documentation, the archive transfer from S3 Glacier vaults to S3 can take up to one day to complete. However, given the size of your data and the time it has already been running, a few factors may be at play:

  1. S3 Glacier retrieval times: Depending on the specific S3 Glacier storage class you're using (Flexible Retrieval or Deep Archive), retrieval times can range from hours to days. For example, S3 Glacier Deep Archive has a standard retrieval time of about 12 hours.

  2. Data transfer process: The transfer workflow involves multiple steps, including inventory generation, preparation, and execution. Large transfers may take longer to process.

  3. Visibility delay: It's possible that data is being transferred but not yet visible in your S3 bucket. The process may be ongoing even if you don't see immediate results.

If you're concerned about the progress, you can check the CloudWatch dashboard for the "Downloaded" count to see if there's any activity. If after 14 hours the count hasn't increased, you may want to investigate further.

Given that your transfer has been running for 48 hours, it's advisable to check the status of the transfer workflow in the AWS Step Functions console. Look for the state machine called "OrchestratorStateMachine" followed by your CloudFormation stack ID. This will show you the current status and any potential issues.

If you need the data transfer to complete more quickly, you can contact AWS Support (with a Developer Support plan or higher) to explore options for faster data movement.

Remember, patience is often required with large Glacier data transfers, but if you're unsure about the progress, don't hesitate to check the AWS console or reach out to AWS Support for assistance.
Sources
Deploy the solution - Data Transfer from Amazon S3 Glacier Vaults to Amazon S3
Step 1: Launch the stack - Data Transfer from Amazon S3 Glacier Vaults to Amazon S3
Understanding S3 Glacier storage classes for long-term data storage - Amazon Simple Storage Service
Troubleshooting - Data Transfer from Amazon S3 Glacier Vaults to Amazon S3

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.