Sync and Glacier

When you sync to a bucket that contains objects already in a Glacier storage class (such as S3 Glacier, S3 Glacier Deep Archive, or S3 Intelligent-Tiering Archive/Deep Archive tiers), the behavior depends on the tool you're using.

If you're using AWS DataSync for synchronization, it will create duplicate copies of each object in the S3 Standard storage class by default when syncing with objects that are already in Glacier storage classes. This happens because DataSync cannot directly access or compare with objects in Glacier without first restoring them.

For other sync tools or methods (like the AWS CLI sync command), objects in Glacier storage classes are not directly accessible for comparison during sync operations. The sync operation would typically treat these objects as not present in the destination and attempt to upload them again as new objects in the default storage class (S3 Standard).

It's important to note that objects in Glacier storage classes need to be restored before they can be accessed or compared. After restoration, they can be copied to a different location as S3 Standard objects if needed.

If you want to avoid duplicate objects and additional storage costs, you should consider:

1. Restoring the Glacier objects first
2. Copying them to S3 Standard if needed
3. Then performing your sync operation
   Sources
   Synchronizing your data to Amazon S3 using AWS DataSync | AWS Storage Blog
   Reading archived S3 objects with S3 Glacier storage classes - AWS Prescriptive Guidance

When you sync a local folder to an S3 bucket that contains objects in Glacier storage classes, the behavior is as follows:

Glacier objects are still in the bucket. They are not “invisible.” Sync operations can see them, but you cannot read or overwrite them directly without first restoring them.

Sync only uploads what’s missing or changed.

Using aws s3 sync local-folder s3://bucket-name, the command compares local files with the bucket based on size and last modified timestamp.

Existing Glacier objects are recognized, so they won’t be automatically re-uploaded unless the local file differs.

Storage class behavior:

If you specify --storage-class STANDARD during sync, only new or updated files will be uploaded as Standard.

Objects already in Glacier stay in Glacier; sync does not automatically convert them to Standard.

Restoring Glacier objects:

To change a Glacier object to Standard, you must first restore it. After restoration, you can overwrite or update it.

Bottom line: The bucket does not appear empty to a sync operation. Sync will only upload files that are missing or have changed, and existing Glacier objects remain in Glacier unless manually restored.

Press accept answer if this helped.