How to determine the root cause of Data Transfer - regional data transfer - in/out/between EC2 AZs or using elastic IPs or ELB fees?
I see an increase of Data Transfer $0.010 per GB - regional data transfer - in/out/between EC2 AZs or using elastic IPs or ELB (approximately 119,626.889 GB) and I would like to determine the root cause.
I am using EKS distributed in 3 AZs in one region, RDS, EC2s.
The cost doubled from previous months with no apparent reason.
How to determine what is causing this? and what are the best practices to reduce this cost?
- Newest
- Most votes
- Most comments
The best way to determine what is causing traffic in a VPC is to use VPC FLow Logs [1]. This will record what IPs or resources are talking to what and you should be able to determine where the Data Transfer costs are coming from. There is a blog [2] about configuring Athena with VPC Flow logs.
What I suspect is happening is that the data transfer is happening whenever 1 EKS Pod needs to communicate with another Pod, the networking can chose either the required Pod in the same AZ or in another AZ. In order to reduce this type of traffic, it's possible to configure Kubernetes with a Multi-AZ ELB at the front of the workflow and then have silo'ed EKS in each AZ. The risk here is that all traffic inside the AZ will stay in the AZ so if there is a failure of some kind, then that request/operation will fail. And the ELB needs to be told that the specific AZ is out of commission. So it's a cost vs risk question with this type of configuration.
Depending on your RDS configuration, you could have all requests going to the same RDS instance in an AZ and therefore crossing AZs. Do you have replicas setup? Are you using the local read replicas for the reads? Maybe you are writing more to the DB recently and those must go to the writer RDS?
[1] https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html [2] https://aws.amazon.com/blogs/networking-and-content-delivery/analyze-vpc-flow-logs-with-point-and-click-amazon-athena-integration/
I can see in the flow logs some requests between EKS pods in different AZs but they are using private IPs. Does inter AZ data transfer using private IPs (10.xx...) cause data transfer fees?
@Marwa, Data Transfer between AZs is charged per the "Inside a Region" Data Transfer costs as explained on our Pricing page [2]. It states there that you will be charged $0.01 per GB in and $0.01 per GB out, so each GB transferred between AZs will essentially be $0.02. It makes no difference if the data is using Private IPs. When you setup your VPC, you assign Subnets per AZ and any communication between those Subnets will have this pricing applied.
[2] https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer
I would look at the following:
- Recent (December, 2021) changes to data transfer (https://aws.amazon.com/about-aws/whats-new/2021/11/aws-price-reduction-data-transfers-internet/
- If your account hit the 1 year mark (services have a 12 month free category for certain things)
- Prices here as well: https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer
I checked two months: December 119,626.889 GB November 59,745.118 GB Number of EKS nodes and namespace did not change much
the recent annoucement only applies to Data Transfer Out (to the Internet) usage and charges, so it won't apply to this question that refers to the Regional (Inter-AZ) transfer
Relevant content
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 6 months ago
regarding using private IP vs public IP: you are NOT charged for traffic you send inside the same AZ using private IP; however you ARE charged for traffic you send between 2 AZs in the same region (even if using private IP) and you are also charged for traffic you send within the same AZ using public IP; all of these will fall under the regional (inter-AZ) traffic pricing