I incurred high data transfer charges for Amazon Simple Storage Service (Amazon S3). I want to identify the buckets that caused the high data transfer charges.
Short description
You pay for all the bandwidth into and out of Amazon S3, except for the following:
- Data that's transferred out to the internet for the first 100 GB per month
Note: This is aggregated across all AWS services and AWS Regions except Amazon Web Services in China and AWS GovCloud (US).
- Data that's transferred in from the internet
- Data that's transferred between S3 buckets in the same Region
- Data that's transferred from an Amazon S3 bucket to any service within the same Region as the S3 bucket
Note: This includes data that's transferred to a different AWS account in the same Region.
- Data that's transferred out to Amazon CloudFront
Resolution
To identify the buckets that are responsible for high data transfer, check your S3 usage report. You can check the operation, Region, and time when the data transfer occurred.
To identify the usage types that correspond to data transfer, see Understanding your AWS billing and usage reports for Amazon S3.
Note: Data transfer charges are logged as region-DataTransfer-Out-Bytes in the usage report.
Review the buckets' S3 server access logs for detailed information about the requests. You can use Amazon Athena to query the server access logs to get information about a specific date and time, operations, and requesters. For example, run the following query to view the amount of data that was transferred through a specific IP address within a specific time period:
SELECT SUM(bytessent) as uploadtotal,SUM(objectsize) as downloadtotal,SUM(bytessent + objectsize) AS total FROM s3_access_logs_db.mybucket_logsWHERE remoteIP='1.2.3.4' AND parse_datetime(requestdatetime,'dd/mm/yyyy:HH:mm:ss Z')BETWEEN parse_datetime('2021-07-01','yyyy-mm-dd')AND parse_datetime('2021-08-01','yyyy-mm-dd');
Related information
Understanding data transfer charges
Amazon S3 request metrics in Amazon CloudWatch