AWS S3 batch operation with different total objects between buckets

0

Hello,

I have executed a batch operation to replicate the objects from one bucket to another, but from the Metrics tab, the number of objects on the destination bucket is different from the amount of objects migrated.

Source bucket total objects: 35869464 Batch job total objects migrated: 35827228 Destination bucket total objects: 657373

The buckets have versioning enabled.

What can explain these differences?

posta un anno fa679 visualizzazioni
1 Risposta
0

You may have multipart uploads that failed and are still in your buckets. Check out these two articles on that situation. You will have to create a lifecyle rule for each bucket to get rid of these. I would use 1 day for the time period or else you'll have to wait a long time for it to take effect. Storage lens will tell you if you have these failed uploads or not. You won't see them in your buckets.

https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/

https://aws.amazon.com/blogs/aws/s3-lifecycle-management-update-support-for-multipart-uploads-and-delete-markers/

You might want to open a support ticket with AWS to get a refund for these failed uploads. I did and they did refund me for them. Be prepared to tell them what buckets and how far back to go for the refund.

con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande