S3 Lifecycle Rule failed to delete/expire noncurrent object with "null" version id.
I enabled the versioning for a bucket that has existing objects, and added lifecycle rules to expire the current & noncurrent objects.
The Lifecycle rules was able to expire the current objects (existing objects before enable versioning), by added delete marker with new version id, and make the current version become noncurrent with version-id = null , and make the delete marker become current version.
However, the noncurrent expiration Lifecycle rules doesn't delete/expire the noncurrent objects that having version-id = null
Here is the summary of configurations, and lifecycle behaviour :
- Enabled versioning for a S3 buckets that having existing objects
- Configure lifecycle rules - 1st Rule to expire current object, 2nd Rule to expire noncurrent version object. 3rd Rule to delete the expired object delete-markers
- 1st Rule was working fine - Expired the existing objects, make them become noncurrent with
version-id=null, add delete markers become current - 2nd Rule was not working - all existing objects that expired by 1st Rule were not expire/delete by 2nd Rule.
- 3rd Rule will not delete the expired object delete-markers, because the noncurrent objects were still there (which not being deleted by 2nd Rule)
I also try to check the object by CLI, and there is no expirations apply to the noncurrent object:
aws s3api head-object --bucket test-bucket-xxxxx --key before-version-enable.png --version-id null
{
"AcceptRanges": "bytes",
"LastModified": "2023-12-06T05:02:11+00:00",
"ContentLength": 16016903,
"ETag": "\"0957c6f20e6314f52dd81ed5b74466af\"",
"VersionId": "null",
"ContentType": "image/png",
"ServerSideEncryption": "AES256",
"Metadata": {}
}
May I know:
- Is this expecting behaviour that lifecycle rules cannot work on noncurrent object that having null version-id?
- Is there a way we leverage the S3 lifecycle rule to handle this?
- 新しい順
- 投票が多い順
- コメントが多い順
S3 lifecycle rule will work on the non-current objects that have the "version-id=null" and will expire the objects as per the rule you have configured. For non-current objects, it is important to note that the object becomes non-current on the day when the new version of object is created. S3 lifecycle is evaluated everyday at UTC midnight. Additionally, Amazon S3 also rounds the expiration date of an object to midnight UTC the next day.
Considering below example:
If you have a lifecycle rule to delete the non-current object versions after 1 day of being non-current and you uploaded an object (version ID: abc) on 1/1/2020 at 10:30 UTC and the new version of same object (version ID: xyz) was created the next day on 2/1/2020 at 11:30 UTC, the object version having version ID: abc became non-current on 2/1/2020 at 11:30 UTC. As per the rule, this non-current object version (version ID: abc) was eligible for deletion after 1 day of being non-current (which is 2/1/2020 at 11:30 UTC + 1 day). As lifecycle rule rounds off the expiration date to nearest UTC, hence, this non-current version would have been marked for deletion on 4/1/2020 at 00:00 UTC.
For below queries:
- Is this expecting behaviour that lifecycle rules cannot work on non-current object that having null version-id? Is there a way we leverage the S3 lifecycle rule to handle this? No, this is not true. The non-current object versions should be expired provided the rule is correctly defined and enough time has elapsed as explained above.
Further, you are unable to see the expiration header for non-current object version because this header is returned only for current version of objects. Non-current versions and delete markers do NOT return the "Expiration" response header in output of 'head-object" API.
I hope the above addresses your query.
Thank you.
