Delete Folder from S3 Versioned Bucket

When you delete a folder in a versioned S3 bucket, it only adds delete markers to the objects rather than permanently deleting them. Here are several approaches to delete these objects with their delete markers:

1. Using AWS CLI with s3api: You can use a command that combines listing object versions and deleting them:

aws s3api delete-objects --bucket your-bucket-name --delete "$(aws s3api list-object-versions --bucket your-bucket-name --prefix folder-path/ | jq '.Versions + .DeleteMarkers | {Objects: map({Key, VersionId})}')"

This command lists all versions and delete markers under your specified folder prefix, then deletes them in a single operation.

2. Using S3 Lifecycle Rules: This is often the most scalable approach for large numbers of objects:

• Create a lifecycle rule that applies to objects with your folder name as the prefix
• Configure it to "Expire current versions of objects" (e.g., 1 day after creation)
• Also configure it to "Permanently delete noncurrent versions of objects" (e.g., 1 day after becoming noncurrent)
• Create a second rule to "Delete expired object delete markers"

The lifecycle rules will execute daily starting at midnight UTC, and will handle the deletion process automatically over a few days.

3. Using a Shell Script: For more control, you can create a script that lists all object versions and deletes them:

bucket=your-bucket-name && aws s3api list-object-versions --bucket $bucket | jq -r '.Versions[] | [.Key, .VersionId] |@tsv' | while read k v; do aws s3api delete-object --bucket $bucket --key $k --version-id $v; done

Note that for buckets with thousands of objects, the lifecycle rule approach is generally recommended as it handles the complexity of versioning and scales well to any number of objects.
Sources
How to all objects in s3 ( and all its versions) with aws command line? | AWS re:Post
Versioned Bucket Delete All Objects and bucket | AWS re:Post
Troubleshooting versioning - Amazon Simple Storage Service

Hi, according to my understanding you want to delete the folder and all objects stored within it, including the various versions of them. Unfortunately the S3 console doesn't support bulk deletion very well, especially for deleted items with markers. A simple approach would be if you have Python and pip installed is to use s3rm, This is a tool used for recursively deleting s3 versioned objects.

Bash code: pip install s3rm s3rm s3://your-bucket-name/your-folder-name/

Alternatively using the s3api through the CLI as suggested in the AI gen answer would be your best bet. Consider implementing life cycle rules as well to avoid this issue in the future.

Since you have thousands of files, it's best to use lifecycle policy. The reason is if the script / cli commands take too long, you can get signed out of the account before the script completes. Here's an example of the lifecycle policy.

{
  "Rules": [
    {
      "ID": "ExpireCurrentVersions",
      "Prefix": "my-folder/",
      "Status": "Enabled",
      "Expiration": {
        "Days": 1
      },
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 1
      }
    },
    {
      "ID": "DeleteExpiredMarkers",
      "Prefix": "",
      "Status": "Enabled",
      "ExpiredObjectDeleteMarker": true
    }
  ]
}