S3 object count using Athena CuR

Question

Customer is migrating their s3 buckets from one account to another one. They have multiple accounts and each account has around 170 buckets. Many buckets have millions of objects stored. Customer is looking for a way to get the object count for each account in a single csv format, to make sure that the data is migrated correctly to new account buckets. Is there any way to get the object count using the athena query to CuR?

Accepted Answer

I suspect that CloudWatch metrics may be a better mechanism to create your reference file: https://docs.aws.amazon.com/AmazonS3/latest/dev/cloudwatch-monitoring.html

As you'll see on that page you can use the Daily Storage Metrics to return the NumberOfObjects per bucket.

You could then write some code to iterate over the regions and buckets returning the number of objects and building your reference file.

S3 object count using Athena CuR

Relevant content