save logs of each outbound data transfer response from cloudfront to s3
I need to do this to charge my clients for data usage, so for each request I would need to get the request header with the output data transfer response and save it to s3
Hi there. I provided some options in your other question, but now I think I understand your request better.
My first question is would be is it possible to have your clients traffic on different distributions? If so, the built-in usage report might give you what you need.
If not, then I would recommend option 2 that I specified in my other answer, sending log data to AWS OpenSearch. This would allow you to use tools like Kibana to parse the logs and create custom dashboards. You can use the sc-bytes field to get the data transferred, and then correlate with other fields like cs-host (if filtering by host) or cs-uri-step (if filtering by URI), etc. More info on fields available here.
Here is a good blog post that gives an overview of how to set this up. Just keep in mind that the ElasticSearch service was recently renamed to AWS OpenSearch in 2021, but the blog post is still relevant.
Another option that you could look into is sending the standard CloudFront logs to CloudWatch. I have not used this option as much, so I don't know if it will get you exactly what you need, but it might be worth investigating. Here is another blog post that might help.
Relevant questions
Amazon CloudFront Data Transfer Per User
Accepted Answerasked 2 years agoFrom where data transfer going to be performed
asked 25 days agoCost of Access to Amazon S3 via Amazon CloudFront within an AWS Region
Accepted Answerasked 2 years agohow to monitor each request and response in cloudfront?
asked 2 months agoWriting IOT Data to S3
asked 5 months agosave logs of each outbound data transfer response from cloudfront to s3
asked 2 months agoConfused between put request pricing and data transfer in pricing
asked 3 years agoAre we billed a put-request for each part with s3 multipart upload or only once for the final merged file?
Accepted Answerasked 9 days agoTransfer Data from EC2 to S3
asked 3 months agoI want to reduce outbound traffic cost from S3 bucket to internet
asked 2 months ago