Skip to content

Empty S3 bucket has objects that aren't multipart uploads

0

I have an S3 bucket that appears empty, yet Metrics shows there's 4500 files and 22 GB used. Using

aws s3api list-multipart-uploads

shows there are no multi-part uploads.

I've never had versioning enabled, but I ran

aws s3api list-object-versions

and it returned no versions.

I added a lifecycle rule as suggested in https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/ that was active for several days with no change to the files reported or data usage of the bucket.

I deleted and re-added the bucket, to no avail.

I'm wondering how I can figure out what S3 thinks is in the bucket and how to delete it.

asked a year ago342 views

3 Answers
0
Accepted Answer

Apparently I just needed to wait longer. Interestingly enough, there's no storage or file count datapoint for 11/13, then on 11/14 it shows accurately. So perhaps the lifecycle rule needed longer to run, coupled with the fact that the s3 data is only generated once a day (so it didn't show the change). Still, not having visibility into the system and "just wait longer" are bad.

answered a year ago

0

This is an unusual situation where your bucket appears empty but metrics show significant storage usage. Since you've already confirmed there are no multipart uploads or object versions, here are some approaches to investigate and resolve this issue:

  1. Try listing objects with a different approach:
  • Use the AWS CLI command aws s3 ls s3://your-bucket-name --recursive to list all objects
  • Try the aws s3api list-objects-v2 --bucket your-bucket-name command which might reveal objects not visible in the console
  1. Check for objects with special characters or unusual prefixes:
  • Objects with special characters in their names might not display properly
  • Objects with prefixes that look like directories (ending with "/") might be hidden
  1. Look for delete markers:
  • Even though you mentioned versioning was never enabled, it's worth checking for delete markers with aws s3api list-object-versions --bucket your-bucket-name
  1. Check S3 Inventory:
  • Set up S3 Inventory for your bucket, which provides a scheduled list of your objects and their metadata
  • This might reveal objects that aren't showing up through other listing methods
  1. Contact AWS Support:
  • If none of these approaches work, AWS Support can help investigate the discrepancy between your bucket's apparent emptiness and the reported storage metrics
  1. As a last resort:
  • Create a new bucket and migrate any needed data
  • Delete the problematic bucket entirely

The discrepancy between metrics and visible objects suggests there might be hidden objects or a potential issue with how the bucket's contents are being reported.
Sources
Emptying a general purpose bucket - Amazon Simple Storage Service
Amazon S3 examples using SDK for Java 2.x - AWS SDK Code Examples

answered a year ago

0

The bucket metrics that S3 reports via CloudWatch metrics aren't real-time. They are updated approximately once per day. Also, you cannot delete an S3 bucket as long as it contains any objects or incomplete multipart uploads. The bucket must be completely empty in order to be deleted.

You made your checks exactly correctly. You used the ListObjectVersions API to check there are no objects committed to the bucket and the ListMultipartUploads API to confirm there weren't any incomplete multipart uploads in the "staging area" of the bucket not yet committed to the bucket as complete objects. When both those API calls return empty results, your bucket is definitely empty, regardless of what the metrics updated on a best-effort basis once per day are showing.

EXPERT

answered a year ago

EXPERT

reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.