Boto3 delete objects with BucketObjectsCollection

0

I am using the boto3 S3 resource to access objects in S3 and later delete all the files. However, the deletion operation could end up leaving one file un-deleted from time to time.

Here is what my code looks like:

s3 = boto3.resource("s3")

# Iterate through a list of prefixes
for some_prefix in list_of_prefixes:
    bucket = s3.Bucket("some_bucket_name")

    # For each prefix, get all the objects with the prefix, do some processing, and finally delete all the processed objects
    bucket_objects = bucket.objects.filter(some_prefix)
    ... <do something with the objects>
    bucket_objects.delete()

To troubleshoot, I added logging in the <do something with the objects> section and made sure that the un-deleted file was part of the bucket_objects BucketObjectsCollection.

Here are more details:

  • There are about ~200 files under each prefix
  • There were no exceptions or errors raised by the S3 resource

Thank you so much for your help!

feita há um ano80 visualizações
Sem respostas

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas