Stop AWS ALB from mutating X-Amzn-Trace-Id header

Question

I am having a problem where my load balancer is *replacing* the X-Amzn-Trace-Id header that our front-end is sending in its request. Our stack consist of the following:
Front-End -> Traefik Server ALB -> Traefik Server (ECS) -> Lambda ALB -> Lambda

Traefik is an application proxy that is commonly used as an api gateway.

When making a request to a lambda directly via its ALB the X-Amzn-Trace-Id remains intact. However, when I introduced Traefik and its ALB, the X-Amzn-Trace-Id header that our front-end is passing is now being replaced. I know this because I'm capturing the X-Amzn-Trace-Id header that Traefik receives, and it is *never* the value that our front-end passes. What can be the cause of this behavior? I'm looking at the documentation, [here](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-request-tracing.html), and it states that an ALB should append a **Self** field to the header if one is found, not replace it entirely. Is there a way to stop the ALB from doing anything with the X-Amzn-Trace-Id header? I'd prefer the ALB leave it alone.

re:Post Agent · Accepted Answer

The behavior you're experiencing with the X-Amzn-Trace-Id header being replaced rather than appended to is indeed different from what the AWS documentation suggests should happen. Unfortunately, there isn't a way to completely stop the Application Load Balancer (ALB) from modifying the X-Amzn-Trace-Id header or to prevent it from being added.

The ALB is designed to always add or update this header for request tracing purposes. If a X-Amzn-Trace-Id header is present in the incoming request with a Root field, the ALB should preserve that Root field and add a Self field. If the header is present with a Self field, the ALB should update the value of the Self field.

However, there are a few things you can consider:

1. Verify that your front-end is indeed sending the X-Amzn-Trace-Id header with a Root field. If it's not including a Root field, the ALB might be treating it as if no header was present.

2. Check if there are any other components in your stack (like Traefik) that might be modifying or stripping the header before it reaches the ALB.

3. You could try to preserve the original header value by renaming it before it reaches the ALB. The ALB allows you to rename request headers. You could rename your original X-Amzn-Trace-Id to something like X-Original-Trace-Id, which the ALB won't modify. Then, in your application, you can access both the original and the ALB-generated trace IDs.

4. If you need to maintain the exact header from your front-end, you might need to consider adding it as a custom header with a different name that won't be modified by the ALB.

Remember that while you can't prevent the ALB from adding or modifying the X-Amzn-Trace-Id header, you can work around it by preserving your original value under a different header name.  
**Sources**  
[ALB: X-Amzn-Trace-Id header - can it be modified or removed, or supplemented with another header? | AWS re:Post](https://repost.aws/questions/QUbTQDekmdQiOug6ZP5orV7g/alb-x-amzn-trace-id-header-can-it-be-modified-or-removed-or-supplemented-with-another-header)  
[Request tracing for your Application Load Balancer - Elastic Load Balancing](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-request-tracing.html)  
[HTTP header modification for your Application Load Balancer - Elastic Load Balancing](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/header-modification.html)

Stop AWS ALB from mutating X-Amzn-Trace-Id header

Add your answer

Relevant content