Skip to content

NoncurrentVersionExpiration s3 lifecycle policy is not deleting noncurrent versions on Object Lock (Governance mode) bucket

1

We have an S3 bucket with the following configuration: Versioning: Enabled Object Lock: Enabled, GOVERNANCE mode, 1-day default retention Lifecycle policy: 1. Expire current versions after 3 days (Expiration.Days = 3), 2. delete noncurrent versions after 1 noncurrent day (NoncurrentVersionExpiration.NoncurrentDays = 1)

s3 Lifecycle Policy applied on the bucket:

{
    "TransitionDefaultMinimumObjectSize": "all_storage_classes_128K",
    "Rules": [
        {
            "Expiration": {
                "Days": 3
            },
            "ID": "data",
            "Filter": {
                "Prefix": "prefix1/"
            },
            "Status": "Enabled",
            "NoncurrentVersionExpiration": {
                "NoncurrentDays": 1
            }
        },
        {
            "Expiration": {
                "Days": 3
            },
            "ID": "data_5min",
            "Filter": {
                "Prefix": "prefix2/"
            },
            "Status": "Enabled",
            "NoncurrentVersionExpiration": {
                "NoncurrentDays": 1
            }
        },
        {
            "ID": "abort_incomplete_mpu",
            "Filter": {
                "Prefix": ""
            },
            "Status": "Enabled",
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
            }
        },
        {
            "Expiration": {
                "ExpiredObjectDeleteMarker": true
            },
            "ID": "expired_delete_markers",
            "Filter": {
                "Prefix": ""
            },
            "Status": "Enabled"
        }
    ]
}

Object Lock Configuration applied on the same bucket:

{
    "ObjectLockConfiguration": {
        "ObjectLockEnabled": "Enabled",
        "Rule": {
            "DefaultRetention": {
                "Mode": "GOVERNANCE",
                "Days": 1
            }
        }
    }
}

Observed behavior: Each object in our bucket has exactly one version. The timeline for a representative object: Day 0: Object written → GOVERNANCE retention stamped (retain-until = Day 1) Day 1: Object Lock retention expires Day 4: Lifecycle Expiration fires as expected → delete marker created, original version becomes noncurrent Day 5+: NoncurrentVersionExpiration (1 noncurrent day) should fire → noncurrent version is NOT deleted

We confirmed via list-object-versions that the delete marker exists as the current version and the original object version is noncurrent, but the noncurrent version persists indefinitely. Can someone help us in identifying the issue and resolving it?

asked 14 days ago60 views

3 Answers
3
Accepted Answer

I think your configuration is valid. The behavior you are observing is not caused by a conflict with S3 Object Lock, but rather by the asynchronous execution of S3 Lifecycle policies.

Because the 1-day Governance retention period expired on Day 1, your object version is no longer WORM-protected. It is fully eligible to be permanently deleted by the NoncurrentVersionExpiration rule without needing any bypass headers or special permissions.

Here is the actual timeline of what is happening in the background:

  1. S3 Lifecycle rules are evaluated once a day at midnight UTC.

  2. When your noncurrent version officially qualifies for permanent expiration (Day 5 in your timeline), S3 does not delete it instantly. Instead, it flags the object and places it into an asynchronous backend queue for removal.

  3. It typically takes an additional 24 to 48 hours (and sometimes longer, depending on bucket size and region load) for the backend process to physically delete the objects and for them to disappear from the list-object-versions API response.

Resolution: Do not change your configuration or build manual deletion scripts. Your JSON setup—including the best-practice rule to clean up expired delete markers—is exactly how it should be configured. Just wait another 1-2 days. The asynchronous S3 Lifecycle queue will process and clear the noncurrent versions automatically.

Reference:

AWS Documentation:

EXPERT

answered 14 days ago

1

Thanks for the reply Florian Turnwald. We will wait for 2-3 days more to see if the non current version objects are getting deleted and will revert back to this post.

answered 14 days ago

EXPERT

reviewed 14 days ago

0

The issue you're experiencing is expected behavior when S3 Object Lock is enabled on a bucket. When Object Lock is configured, even in GOVERNANCE mode, it fundamentally changes how lifecycle policies interact with object versions.

Here's what's happening:

Object Lock and Lifecycle Policy Interaction: When Object Lock is enabled on a bucket, the NoncurrentVersionExpiration lifecycle action cannot permanently delete noncurrent object versions that were protected by Object Lock, even after their retention period has expired. The Object Lock protection remains associated with the object version throughout its lifecycle.

Why Your Noncurrent Versions Aren't Being Deleted: Even though your Object Lock retention period (1 day) has expired and the object has become noncurrent (after Day 4 when the delete marker was created), the lifecycle policy's NoncurrentVersionExpiration action is unable to remove these versions because they were originally created with Object Lock protection.

Resolution Options:

  1. Manual Deletion with Bypass Permission: To delete noncurrent versions that were protected by Object Lock in GOVERNANCE mode, you need to explicitly bypass the governance retention. This requires the s3:BypassGovernanceRetention permission and must be done manually using a DELETE request with the x-amz-bypass-governance-retention: true header.

  2. Reconsider Object Lock Usage: If your use case allows, evaluate whether Object Lock is necessary for your retention requirements. If you only need lifecycle-based expiration without write-once-read-many (WORM) protection, you might consider using lifecycle policies alone without Object Lock.

  3. Adjust Retention Strategy: If Object Lock is required for compliance, understand that you'll need to implement a separate process to clean up noncurrent versions after their retention periods expire, as lifecycle policies alone cannot handle this automatically.

The lifecycle policy is working correctly for creating delete markers (expiring current versions), but the permanent deletion of noncurrent versions requires additional handling when Object Lock is involved.
Sources
NoncurrentVersionExpiration - Amazon Simple Storage Service
Deleting object versions from a versioning-enabled bucket - Amazon Simple Storage Service

answered 14 days ago

EXPERT

reviewed 13 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.