EKS cross az data transfer

You can significantly reduce cross-AZ data transfer costs in EKS by implementing Traffic Distribution using Kubernetes Services. This approach keeps traffic within the same AZ when possible while maintaining high availability.

If you don't need to deploy Istio or other service mesh, just using Traffic Distribution with Kubernetes Services is the most effective approach, as documented in the Kubernetes Traffic Distribution documentation.

Implementation

Step 1: Enable Traffic Distribution on your Services

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 8080
  trafficDistribution: PreferClose

The PreferClose value ensures that traffic is routed to endpoints that are topologically close (same AZ) when possible, falling back to other endpoints when local ones are unavailable.

Step 2: Ensure your nodes have proper zone labels

# Verify nodes have zone labels (should be automatic with EKS)
kubectl get nodes --show-labels | grep topology.kubernetes.io/zone

Step 3: Configure your Deployments with zone awareness

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 6  # 2 per AZ for example
  template:
    spec:
      topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app: my-app

Advanced Architecture: Independent AZ Scaling

⚠️ Important Consideration: If you implement Traffic Distribution with PreferClose, you may want to scale each AZ independently to handle varying load patterns. This prevents overloading a single AZ when it receives more traffic from clients in that zone.

Recommended Architecture:

Single Service with Multiple Deployments and Individual HPAs

# Deployment for us-east-1a
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-us-east-1a
spec:
  template:
    spec:
      nodeSelector:
        topology.kubernetes.io/zone: us-east-1a
      containers:
      - name: app
        image: my-app:latest
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
---
# HPA for us-east-1a
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa-us-east-1a
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app-us-east-1a
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
---
# Service that covers all AZ deployments
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app  # All deployments should have this label
  ports:
  - port: 80
    targetPort: 8080
  trafficDistribution: PreferClose

Repeat similar Deployment and HPA configurations for us-east-1b and us-east-1c

Benefits of This Approach:

• Cost Optimization: Reduces cross-AZ data transfer charges by keeping traffic local with PreferClose
• Independent Scaling: Each AZ can scale based on its specific load patterns
• High Availability: Maintains fault tolerance across AZs
• Automatic Failover: Traffic automatically routes to other AZs if local pods are unavailable