How to gather hourly data analysis of S3 bucket

Question

I am attempting to gather data analysis for a specific S3 bucket. My goal is to find how much data is being uploaded to the specific S3 bucket on an hourly basis. I am trying to figure how many files and if possible, the amount of data total for each hour in a 24 hour period.

Any help would be appreciated.

Answer

To my knowledge if you want this data by hour you will need to enable access logs and then aggregate over the log data. Standard cloudwatch metrics only give you data over 24 hours.
Access logs will include the object size of any Put request - See format [here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/LogFormat.html)
You could aggregate using Athena or other log analytics (log insights/opensearch/splunk). You can see the schema and example queries [here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-s3-access-logs-to-identify-requests.html#querying-s3-access-logs-for-requests)

How to gather hourly data analysis of S3 bucket

相關內容