How to build centralized IPv6 egress (NAT66) on AWS using Gateway Load Balancer

Problem

Most enterprise AWS customers who build out IPv6 environments hit a blocker: their IPv4 architecture has a clean centralized-egress pattern (workload VPC → TGW → egress VPC → NAT GW → IGW → internet), and they want the IPv6 equivalent. And any centralized egress solution would use NAT in the egress VPC. But IPv6 was designed to eliminate NAT and that's why a NAT 6 to 6 is not very popular. In large multi VPC environment some common drivers which will require NAT66 are:

• Single egress prefix for upstream allowlisting and audit logging (rather than every workload's own GUA reaching the internet directly)
• Topology hiding — internal IPv6 addressing isn't exposed to upstream destinations
• Allowing use of ULA or private GUA — Topologies using ULA or private GUA needs to access endpoints on Internet
• Many internal sources behind one (or a few) external addresses — the same M:N (many-to-few) pattern IPv4 NAT provides, where M internal hosts share N external addresses
• A single place to apply egress controls — rate limiting, FQDN allow lists, logging, deep inspection

Available IPv6 egress options today (and why they don't solve this)

Pattern	What it does	Why it doesn't solve this
Egress-only IGW (EIGW)	Allows IPv6 outbound, blocks unsolicited inbound	Each workload egresses with its own GUA. No source hiding, no single egress prefix.
NAT Gateway	IPv4 only	Doesn't handle IPv6 at all
Public NAT64	Translates IPv6 → IPv4	Only useful when destination is IPv4-only

This is the same situation customers faced for IPv4 before NAT GW launched in 2015 — the answer back then was "run a NAT instance." For IPv6 today, that answer is "run a NAT66 appliance, but make it horizontally scalable behind GWLB."

Solution architecture

The solution uses Linux-based NAT66 appliances doing M:1 SNAT behind AWS Gateway Load Balancer (GWLB), with workload VPCs reaching the egress fabric via Transit Gateway (or AWS Cloud WAN — the same pattern applies to either centralized routing layer). The appliance is two-armed in a logical sense:

• Geneve tunnel — one "arm" (where workload traffic decaps in, NAT'd traffic gets encapsulated back)
• Physical ENI — the other "arm" (where SNAT'd traffic egresses to the IGW, and replies arrive directly)

The diagram below illustrates the two logical arms:

Inbound flow: Workload → TGW → GWLBe → GWLB → Geneve tunnel → gwi-X (decap) → ip6tables MASQUERADE → ens5 → IGW → Internet.

Return flow: Internet → IGW → ens5 → conntrack reverse-NAT → policy route → gwo-X (encap) → GWLB → TGW → Workload.

What "M:1 SNAT" means here

The appliance applies ip6tables MASQUERADE on its primary interface. Every connection from any workload VPC, regardless of source, is rewritten to use that appliance's own public IPv6 GUA as the source. Linux conntrack tracks per-flow state so replies are reverse-NAT'd correctly.

With N appliances behind GWLB:

• New TCP connections hash across all N appliances via GWLB's 5-tuple flow hashing
• Each appliance has its own unique IPv6 GUA, so internet destinations see one of the appliances' source IPs (not the workload's)
• Return traffic arrives directly at the correct appliance because the destination IP uniquely identifies it
• Clean horizontal scaling: ASG adds appliances when CPU/network climbs, removes them when load drops

The five things that must all be true

1. GWLB must be dualstack

Gwlb:
  Type: AWS::ElasticLoadBalancingV2::LoadBalancer
  Properties:
    Type: gateway
    IpAddressType: dualstack    # default is ipv4 — IPv6 silently dropped

If you leave it at the default (ipv4), the GWLB silently discards IPv6 packets.

2. The GWLB Endpoint Service must support IPv6

GwlbEndpointService:
  Type: AWS::EC2::VPCEndpointService
  Properties:
    GatewayLoadBalancerArns: [!Ref Gwlb]
    SupportedIpAddressTypes: [ipv4, ipv6]

3. The GWLB Endpoint itself must be dualstack

GwlbEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    VpcEndpointType: GatewayLoadBalancer
    IpAddressType: dualstack

Same as the service — endpoint inherits ipv4 if not explicit.

4. Native Linux GENEVE modules expect Layer 2 framing, but GWLB tunnels raw Layer 3 packets

The Linux kernel's built-in GENEVE module (geneve external) is designed to handle Layer 2 (Ethernet) frames inside the tunnel. However, GWLB sends raw Layer 3 (IP) packets directly inside the Geneve encapsulation — no Ethernet header. As a result, the kernel receives UDP 6081 packets, but in geneve external mode it requires userspace to pop the Geneve header and inject the inner IP packet into the network stack correctly.

The fix: Install aws-gwlb-tunnel-handler — an open-source C daemon from AWS Samples that creates gwi-<X>/gwo-<X> tun interface pairs and handles Geneve encap/decap in userspace. For a detailed walkthrough, see Andrew Gray's blog post: https://aws.amazon.com/blogs/networking-and-content-delivery/how-to-integrate-linux-instances-with-aws-gateway-load-balancer/.

Note: It builds on Amazon Linux 2023, but requires Boost headers ≥1.83 (AL2023 ships 1.75). Workaround: download Boost 1.83 source headers manually and point cmake at them. The steps are shown in the AMI build section below (Step 1) — specifically, download boost_1_83_0.tar.gz from archives.boost.io, extract it, then pass -DBOOST_INCLUDEDIR=/home/ec2-user/boost to cmake.

5. A return route on the appliance forces replies back through Geneve

By default, when the appliance's kernel reverse-NATs a reply, it routes based on the destination IPv6 — which is the workload's GUA. The default route on the appliance points at the IGW (correctly, for outbound), but that's the wrong direction for replies headed back to the workload VPC.

Because the appliance's VPC has a route for the workload VPC CIDR pointing to TGW (needed for the forward path), the reply would leave via ens5 to the VPC router, follow that TGW route, and arrive at the workload — but completely bypassing GWLB and breaking flow stickiness.

The fix: In the gwlbtun CREATE hook, add a policy route forcing the workload VPC's IPv6 prefix to leave via the egress tunnel interface:

ip -6 route replace ${WORKLOAD_VPC_IPV6_CIDR} dev gwo-<X>

The appliance kernel sends replies to gwo-<X> → gwlbtun re-encapsulates into Geneve → GWLB → TGW → workload.

Note: The gwlbtun daemon creates paired tunnel interfaces named gwi-<X> (inbound) and gwo-<X> (outbound) for each GWLB flow. A full explanation of gwlbtun architecture and its interface nomenclature is available in the AWS Samples repository and Andrew Gray's blog post referenced above.

Horizontal autoscaling

The appliances are in an ASG configured with two target tracking scaling policies, both at 80% targets. Whichever fires first triggers a scale action.

Policy	Metric	Target
CPU	ASGAverageCPUUtilization	80%
Network	ASGAverageNetworkOut	80% of instance baseline bandwidth

Deployment walkthrough

First clone this repo to get the full CloudFormation template and AMI build guide.

Step 1: Build the appliance AMI

Launch an Amazon Linux 2023 instance (for example c6i.large) with public internet access, SSM into it, and run:

# ── 1. Install all dependencies  ──────────────────────────────
sudo dnf install -y git gcc gcc-c++ cmake make libcap-devel kernel-headers \
  iproute tcpdump iptables-nft conntrack-tools wget tar awscli \
  nmap-ncat                          # ← ADD THIS: required for health check listener

# ── 2. Persistent kernel settings ────────────────────────────────────────────
sudo modprobe nf_conntrack
sudo tee /etc/sysctl.d/99-nat66.conf << 'EOF'
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 0
net.netfilter.nf_conntrack_max = 1048576
EOF
sudo sysctl -p /etc/sysctl.d/99-nat66.conf

# ── 3. Boost 1.83 headers ────────────────────────────────────────────────────
# NOTE: Use /home/ec2-user regardless of whether you connected via ec2-user
# or ssm-user — cmake looks for Boost here at build time only.
cd /home/ec2-user
wget -q https://archives.boost.io/release/1.83.0/source/boost_1_83_0.tar.gz
tar xf boost_1_83_0.tar.gz && mv boost_1_83_0 boost
sudo chown -R ec2-user:ec2-user /home/ec2-user/boost

# ── 4. Build aws-gwlb-tunnel-handler ─────────────────────────────────────────
cd /usr/local/src
sudo git clone --depth 1 \
  https://github.com/aws-samples/aws-gateway-load-balancer-tunnel-handler.git
cd aws-gateway-load-balancer-tunnel-handler
sudo mkdir build && cd build && sudo cmake .. && sudo make -j$(nproc)
sudo cp gwlbtun /usr/local/bin/gwlbtun
sudo chmod +x /usr/local/bin/gwlbtun

# Verify
/usr/local/bin/gwlbtun --help 2>&1 | head -3

# ── 5. Pre-stage gwlbtun config directory ────────────────────────────────────
sudo mkdir -p /etc/gwlbtun
# Note: nat-hook.sh is generated at instance boot by the CloudFormation
# userdata with the correct client VPC CIDR. Do NOT create it here.

# ── 6. Install systemd services ──────────────────────────────────────────────
# Health check listener — answers GWLB TCP health checks on port 80
sudo tee /etc/systemd/system/gwlb-healthcheck.service << 'EOF'
[Unit]
Description=GWLB TCP Health Check Listener
After=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/ncat -l -k -p 80
Restart=always
RestartSec=2

[Install]
WantedBy=multi-user.target
EOF

# gwlbtun daemon — handles Geneve encap/decap and calls nat-hook.sh
sudo tee /etc/systemd/system/gwlbtun.service << 'EOF'
[Unit]
Description=AWS GWLB Tunnel Handler (NAT66)
After=network-online.target gwlb-healthcheck.service
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/bin/gwlbtun -c /etc/gwlbtun/nat-hook.sh -r /etc/gwlbtun/nat-hook.sh
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable gwlb-healthcheck.service gwlbtun.service
# ⚠️  Do NOT start the services here.
# gwlbtun needs nat-hook.sh which is only generated at boot by CloudFormation userdata.

# ── 7. Verify before snapshotting ────────────────────────────────────────────
systemctl is-enabled gwlb-healthcheck.service   # → enabled
systemctl is-enabled gwlbtun.service            # → enabled
/usr/bin/ncat --version                         # → Ncat: Version 7.x
/usr/local/bin/gwlbtun --help | head -1         # → AWS Gateway Load Balancer Tunnel Handler v3
ls /etc/gwlbtun/                                # → empty dir (hook added at boot)
cat /etc/sysctl.d/99-nat66.conf                 # → all 5 settings present

# ── 8. Clean up before AMI snapshot ──────────────────────────────────────────
sudo truncate -s 0 /var/log/cloud-init-output.log /var/log/cloud-init.log
sudo rm -rf /var/lib/cloud/instances/* /var/lib/cloud/instance
sudo rm -f /home/ec2-user/boost_1_83_0.tar.gz   # remove tarball, keep headers
# Exit back to your local terminal, then:
# aws ec2 create-image --instance-id <BUILDER_ID> --name "nat66-appliance-v2-..." --no-reboot

After cleanup, create the AMI:

aws ec2 create-image --instance-id <BUILDER_ID> --name nat66-appliance-v1 --no-reboot

Step 2: Deploy the CloudFormation stack

Note: The CloudFormation stack does not directly create the policy route. The ip -6 route replace command runs inside the gwlbtun CREATE hook script, which is deployed either via the AMI (if pre-baked) or via the instance's userdata. CloudFormation provisions the infrastructure (GWLB, ASG, networking), while the appliance-level routing logic lives in the hook scripts that gwlbtun invokes when a new tunnel is established.

aws cloudformation deploy \
  --region <YOUR_REGION> \
  --stack-name nat66-egress-lab \
  --template-file nat66-egress-ami-asg-scaling.yaml \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides \
    ApplianceAmi=<your-ami-id> \
    ApplianceMinCount=2 \
    ApplianceMaxCount=6 \
    ApplianceDesiredCount=2 \
    InstanceType=c6i.large

Deploy time: ~10 minutes (GWLB endpoint is the slowest resource).

Step 3: Validate

Connect to the test client EC2 (deployed in a workload VPC the stack also creates):

aws ssm start-session --target <ClientInstanceId>

Inside the client:

ping6 -c 5 2606:4700:4700::1111
curl -6 https://api64.ipify.org   # should return one of the appliance GUAs

Run 20 curls in a loop and count distinct sources — you should see roughly even distribution across whatever number of appliances are currently in service.

Production hardening (not covered in this POC)

This is a working pattern but not production-ready. Before putting real traffic through it:

1. Multi-AZ: Deploy appliances across 2–3 Availability Zones and configure GWLB to span all of them for fault tolerance. The POC uses a single AZ to minimize cost.

2. Session state and target failover: Not shared across appliances — in-flight long-lived TCP flows (SSH, DB pools) will reset when an appliance is replaced. Consider enabling GWLB target failover for existing flows to rebalance connections to healthy targets (see: https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-aws-gateway-load-balancer-target-failover-for-existing-flows/). New connections work fine regardless.

3. Dedicated egress prefix (BYOIP or a second /56) for SNAT sources — cleaner for upstream allowlisting than the centralized egress VPC's default /64.

4. Custom AMI: Bake Boost headers, gwlbtun binary, and the gwlbtun hook scripts (including the CREATE hook that adds the policy route for return traffic) into a custom AMI for sub-60-second appliance boot. The POC's userdata-based build takes ~6 minutes per new instance.

5. Observability: Ship conntrack-count, NAT hit rates, gwlbtun health stats to CloudWatch. Set alarms for target health changes, conntrack table depth, and scaling events.

Related patterns

• NAT64 + DNS64: If your workloads are IPv6 but need to reach IPv4-only internet destinations, use NAT64 + DNS64 instead of NAT66.
• Interface VPC Endpoints (PrivateLink): If you're trying to centralize IPv6 access to AWS service APIs, use Interface VPC Endpoints per service — IPv6 endpoints are available for most services. No NAT needed, no internet egress, no GWLB.

References

• AWS Samples: aws-gwlb-tunnel-handler — https://github.com/aws-samples/aws-gateway-load-balancer-tunnel-handler
• AWS announcement: GWLB dual-stack support (2024) — https://aws.amazon.com/about-aws/whats-new/2024/05/aws-gwlb-dual-stack-support/
• RFC 6296 — IPv6-to-IPv6 Network Prefix Translation (NPTv6)
• AWS docs: Gateway Load Balancer target groups — https://docs.aws.amazon.com/elasticloadbalancing/latest/gateway/target-groups.html
• AWS docs: Auto Scaling target tracking policies — https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-target-tracking.html