Boto3 delete objects with BucketObjectsCollection

0

I am using the boto3 S3 resource to access objects in S3 and later delete all the files. However, the deletion operation could end up leaving one file un-deleted from time to time.

Here is what my code looks like:

s3 = boto3.resource("s3")

# Iterate through a list of prefixes
for some_prefix in list_of_prefixes:
    bucket = s3.Bucket("some_bucket_name")

    # For each prefix, get all the objects with the prefix, do some processing, and finally delete all the processed objects
    bucket_objects = bucket.objects.filter(some_prefix)
    ... <do something with the objects>
    bucket_objects.delete()

To troubleshoot, I added logging in the <do something with the objects> section and made sure that the un-deleted file was part of the bucket_objects BucketObjectsCollection.

Here are more details:

  • There are about ~200 files under each prefix
  • There were no exceptions or errors raised by the S3 resource

Thank you so much for your help!

posta un anno fa80 visualizzazioni
Nessuna risposta

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande