Lifecycle rules setup on s3 versions enabled is not cleaning up prefixes

When dealing with versioned S3 buckets, lifecycle rules can be tricky. Here are some possible reasons why your lifecycle rule isn't cleaning up data as expected:

1. When versioning is enabled and you delete an object, S3 doesn't actually remove the object - it creates a delete marker which becomes the current version. The original object remains as a noncurrent version.

2. Your lifecycle rule might need multiple components to fully clean up versioned objects:

• A rule to expire current versions after 1 day
• A rule to permanently delete noncurrent versions after 1 day
• A rule to delete expired object delete markers

3. Lifecycle rules are processed asynchronously and typically run once per day starting at midnight UTC. There can be delays, especially with large amounts of data.

4. The "days after becoming noncurrent" counter starts when an object version becomes noncurrent, not when it was created. If objects only recently became noncurrent, they won't be deleted yet.

5. S3 lifecycle rules don't support wildcards using "" in prefix filters. Make sure your prefix is correctly formatted (e.g., "folder/" rather than "folder/").

If you need to delete the data more quickly, you can manually delete all versions of the objects using the AWS CLI with a command that combines listing object versions and deleting them. For example:

aws s3api delete-objects --bucket your-bucket --delete "$(aws s3api list-object-versions --bucket your-bucket --prefix your-prefix/ | jq '.Versions + .DeleteMarkers | {Objects: map({Key, VersionId})}')"

This command will delete all objects and their versions under the specified prefix.
Sources
Unable to delete data with deletion marker | AWS re:Post
S3-lifecycle to auto-delete old object-versions: configuration? | AWS re:Post
S3 lifecycle rule not deleting files on partitions | AWS re:Post
How to all objects in s3 ( and all its versions) with aws command line? | AWS re:Post