Boto3 delete objects with BucketObjectsCollection

0

I am using the boto3 S3 resource to access objects in S3 and later delete all the files. However, the deletion operation could end up leaving one file un-deleted from time to time.

Here is what my code looks like:

s3 = boto3.resource("s3")

# Iterate through a list of prefixes
for some_prefix in list_of_prefixes:
    bucket = s3.Bucket("some_bucket_name")

    # For each prefix, get all the objects with the prefix, do some processing, and finally delete all the processed objects
    bucket_objects = bucket.objects.filter(some_prefix)
    ... <do something with the objects>
    bucket_objects.delete()

To troubleshoot, I added logging in the <do something with the objects> section and made sure that the un-deleted file was part of the bucket_objects BucketObjectsCollection.

Here are more details:

  • There are about ~200 files under each prefix
  • There were no exceptions or errors raised by the S3 resource

Thank you so much for your help!

gefragt vor einem Jahr80 Aufrufe
Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen