how to empty version enabled bucket faster?

0

While doing the bucket empty using below aws s3 cli, it removed the objects and now for current version is it not showing any object but when I see using version checkbox enabled it shows more object, so I need to empty other version's object as well other wise bucket deleton is not possible, please let me know hot to do it in faster way.

Cli used for emptying the bucket.

aws s3 rm s3://<bucketName>--recursive

I have also enabled the lifecycle on the bucket but does any command also needed to remove the object

2 回答
1

Hi,

You can try to use a combination of delete-objects CLI command with a list-object-versions

aws s3api delete-objects --bucket <bucketName> \ 
  --delete "$(aws s3api list-object-versions \
  --bucket "<bucketName>" \
  --no-paginate  \
  --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}')"

Hope it helps ;)

profile picture
专家
已回答 1 年前
  • Thanks @alatech

    Seems it is working, but it is listing array of 10 objects only, saying those 10 objects are deleted, so does it need to call in loop untill all objects get deleted?

    Let me try.

  • Try tweaking the parameters related to pagination: https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-pagination.html

  • This is extremely slow though, and sadly not parallelisable, due to the terrible way AWS SDK/CLI handles pagination. Lifecycle rules are so far the best method I have found, but even that takes days just do to the delay with a minimum of 1 day of age for the two steps each, and more if there are a large number of objects. I've tried S3 Batch Operations, but not only is it very hard to operate, but it doesn't seem to support deleting actual objects or worse object versions, just tags of objects or copying objects.

0

While running suggested "aws s3api delete-objects ..." command.

I observed the matrics of the bucket, it shows the size increased by few more GB, and can see every (almost all) objects in the bucket are marked as delete marker.

Wants to understand how delete is working, is it like first it will marke object to delete marker and then in coming future those objects will get deleted?

Will it get deleted by lifecycle set on the bucket?

已回答 1 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则