Empty bucket not really empty?

0

Hi!

We are using this particular bucket as a temporary storage - one service stores big (1-500GB) dump files there and another scheduled daily job checks if this bucket contains any files and retrieves them (files are removed from the bucket afterwards). It's a really low frequency traffic and there shouldn't be more than a couple of these files per day - there are days with no traffic at all. Versioning is off, of course.

And this is the problem: this bucket is empty (for days now), but if I look at bucket metrics, CloudWatch claims there is around 1TB of data there in more than 65K of files. I understand that sometimes operations are just scheduled and do not happen immediately, but a week seems like enough time to delete a couple of files.

What have I tried so far?

  1. I read somewhere that these files could be remnants from failed uploads, which are not shown because they are not complete files. And that they could be cleared out if I turned the versioning on with a lifecycle rule that deletes everything. Tried, it didn't change anything.
  2. I deleted this bucket completely - it shouldn't work if bucket is not empty in the first place (as far as I understand). I created a new bucket with the same name after a day and lo and behold - total bucket size is 1018GB, according to metrics.

If there are files still in this bucket, how do I see/remove them from this limbo? If that's not the case, how do I stop CloudWatch from giving me false information? Why pay for "nothing"?

Cheers

質問済み 2年前1676ビュー
3回答
1

This could be related to Amazon S3's multipart upload feature. If the complete multipart upload request isn’t sent successfully, Amazon S3 will not assemble the parts and will not create any object. The parts remain in your Amazon S3 account until the multipart upload completes or is aborted, and you pay for the parts that are stored in Amazon S3. Also note that you aren’t able to view the parts of your incomplete multipart upload in the AWS Management Console.

While it is possible to manually list and abort incomplete multipart uploads in your S3 buckets, this can quickly become a chore.

Consider this solution from AWS. There are numerous others if you search if your problem is actually related to aborted multipart uploads.

RoB
回答済み 2年前
1

Thanks RoB,

Yeah, as I mentioned, I have tried this AWS recipe with lifecycle rule (although I ticked all boxes, to delete everything there, not just deleted markers and incomplete multipart uploads), but it didn't solve the issue. I'm gonna try with manual procedure and report how it went.

回答済み 2年前
0

Having the same issue. 22.2 GB was being reported for one of my buckets I used for a multiport upload that failed. I deleted the bucket (as stated above should not happen if there is something in the bucket). I have no information anymore on that multipart upload, it was a while ago, a few months. Would really like to know what the answer is.

profile picture
NeoJim
回答済み 4ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ