archiving directly to glacier

0

We are developing a glacier archive solution, but now wondering whether it would be better to upload to S3 then use lifecycle rules to move the files to glacier. We would be uploading terrabytes of data every day which include files that could be 50G and files that could be 1M. Now wondering if anyone actually goes straight to glacier or if that is not really a thing any more.

ebethj
질문됨 2년 전267회 조회
1개 답변
0

Hi,

I understand that you are developing a glacier archive solution and you want to consider uploading to S3 then use lifecycle to move the files to glacier.

When choosing between glacier archive and S3 lifecycle, it is important to consider the requirements and costs of your archive solution, as well as the trade-offs between different storage options [1].

While both options involve transitioning data to Glacier for long-term archival, the approach of uploading to S3 first and then using lifecycle rules [2] to move files to Glacier has advantages, including cost-savings and greater flexibility. S3 is a commonly used storage solution offered by AWS and Glacier is a storage class within S3 that is designed for long-term data archival.

By uploading to S3 first, you can take advantage of the various S3 storage classes, and then use lifecycle rules to automatically transition to Glacier over time. This approach can help you optimise costs and maintain flexibility with your archive solution.

Please also consider S3 lifecycle cost the approach of uploading to S3 first and then using lifecycle rules to move files to Glacier may have cost implications please go through the documentation link for lifecycle rules pricing [3].

According to your archive solution consider using amazon Glacier Deep Archive it has a low cost per GB but requires longer retrieval time compared to other classes [4].

To upload files directly to glacier storage class using AWS CLI you may use example command below:

  • aws s3 cp /path/to/local/file s3://my-bucket/my-file - -storage-class GLACIER --recursive
  • aws s3 sync . /path/to/local/file s3://my-bucket/my-file --storage-class GLACIER - scans documents before uploading to GLACIER
  • /path/to/local/file - provide full path of the local file you want to upload.
  • my-bucket - provide name of s3 bucket you want to upload to.
  • my-file - provide the name you want to give to the uploaded file in the s3 bucket.
  • -storage-class GLACIER - Option specifies that the uploaded files should be stored in s3 Glacier storage class.

I hope the provided Information has been useful to you. If you have any other queries please do not hesitate to contact me back. I will be more than happy to help.

Have a great day ahead!

References:

[1] Different storage options - https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html [2] Upload to S3 then use lifecycle rules - https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html [3] Lifecycle rules pricing - https://aws.amazon.com/s3/pricing/ [4] Amazon S3 storage classes - https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html#sc-compare

AWS
지원 엔지니어
답변함 9달 전
AWS
지원 엔지니어
검토됨 9달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠