archiving directly to glacier

0

We are developing a glacier archive solution, but now wondering whether it would be better to upload to S3 then use lifecycle rules to move the files to glacier. We would be uploading terrabytes of data every day which include files that could be 50G and files that could be 1M. Now wondering if anyone actually goes straight to glacier or if that is not really a thing any more.

ebethj
asked a year ago258 views
1 Answer
0

Hi,

I understand that you are developing a glacier archive solution and you want to consider uploading to S3 then use lifecycle to move the files to glacier.

When choosing between glacier archive and S3 lifecycle, it is important to consider the requirements and costs of your archive solution, as well as the trade-offs between different storage options [1].

While both options involve transitioning data to Glacier for long-term archival, the approach of uploading to S3 first and then using lifecycle rules [2] to move files to Glacier has advantages, including cost-savings and greater flexibility. S3 is a commonly used storage solution offered by AWS and Glacier is a storage class within S3 that is designed for long-term data archival.

By uploading to S3 first, you can take advantage of the various S3 storage classes, and then use lifecycle rules to automatically transition to Glacier over time. This approach can help you optimise costs and maintain flexibility with your archive solution.

Please also consider S3 lifecycle cost the approach of uploading to S3 first and then using lifecycle rules to move files to Glacier may have cost implications please go through the documentation link for lifecycle rules pricing [3].

According to your archive solution consider using amazon Glacier Deep Archive it has a low cost per GB but requires longer retrieval time compared to other classes [4].

To upload files directly to glacier storage class using AWS CLI you may use example command below:

  • aws s3 cp /path/to/local/file s3://my-bucket/my-file - -storage-class GLACIER --recursive
  • aws s3 sync . /path/to/local/file s3://my-bucket/my-file --storage-class GLACIER - scans documents before uploading to GLACIER
  • /path/to/local/file - provide full path of the local file you want to upload.
  • my-bucket - provide name of s3 bucket you want to upload to.
  • my-file - provide the name you want to give to the uploaded file in the s3 bucket.
  • -storage-class GLACIER - Option specifies that the uploaded files should be stored in s3 Glacier storage class.

I hope the provided Information has been useful to you. If you have any other queries please do not hesitate to contact me back. I will be more than happy to help.

Have a great day ahead!

References:

[1] Different storage options - https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html [2] Upload to S3 then use lifecycle rules - https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html [3] Lifecycle rules pricing - https://aws.amazon.com/s3/pricing/ [4] Amazon S3 storage classes - https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html#sc-compare

AWS
SUPPORT ENGINEER
answered 9 months ago
AWS
SUPPORT ENGINEER
reviewed 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions