CloudFront: discrepancy between cache statistics and metrics on CloudWatch

Question

We have a bunch of CloudFront distributions, and we need to programmatically access the total bytes transferred (out) for each one of them.

Following the [documentation](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/programming-cloudwatch-metrics.html) on accessing CloudFront's metrics through CloudWatch via API, we can successfully fetch the `BytesDownloaded` metric data points we need.

However, we noticed a small but significant discrepancy between that metric and the one visible in che CloudFront console (Reports & analytics > Cache statistics > Bytes transferred to viewers). Specifically, comparing the `BytesDownloaded` metric fetched through CloudWatch and `TotalBytes` (from [CloudFront popular objects report](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/popular-objects-report.html)): the latter is on average a +5-6% bigger than `BytesDownloaded` (comparing the hourly sum values in the same recent period).

Why is there a discrepancy? How can we access the `TotalBytes` data points through the API?

Accepted Answer

Thank you for the detailed description.

---

Firstly, `TotalBytes` is "The total number of bytes that CloudFront served to viewers in response to all requests for the object for all HTTP methods. [1]", while `BytesDownloaded` is "The total number of bytes downloaded by viewers for GET, HEAD, and OPTIONS requests. [2]", hence the discrepancy.

Secondly, the popular objects report is currently only available via Console (instead of API.) A workaround would be to enable and leverage CloudFront access logs [3] for your distribution. By using the CloudFront access logs, you could programmatically and periodically analyze access logs to retrieve similar information as the CloudFront popular objects report. Unlike the popular objects report, the access logs are not limited to 50 records, every request to the CloudFront distribution gets logged.

---

[1] https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/popular-objects-report.html#popular-objects-data

[2] https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/viewing-cloudfront-metrics.html#monitoring-console.distributions

[3] https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html

CloudFront: discrepancy between cache statistics and metrics on CloudWatch

相关内容