In the ALB, we have HTTP2 configured and we know that the target group supports HTTP1.1. So, we expect automatic negotiation with target.
However, we are seeing one kind of API consistently fail with 502 error while all other APIs pass.
The failure is observed in a POST API call without any body and the expected response also is a 204. Other POST & PUT calls with body and GET calls are successful. What does this indicate ? Is there a known issue with ALB with HTTP2 ?
The headers captured of a successful call are below. This responds with 201 which is expected and Content-Length:0 in response header
PUT https://<url1> h2
ML-Agent-ID: java
Authorization: Basic <BASE64>
Content-Type: application/json
Transfer-Encoding: chunked
Host: <deleted>
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/4.11.0
The headers captured from a failed call are below. (basically same, but this has no body). This responds with 502, but expected is a 204 and Content-Length:0
POST https://<url2> h2
ML-Agent-ID: java
Authorization: Basic <BASE64>
Transfer-Encoding: chunked
Host: <deleted>
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/4.11.0
A related post was found in [https://repost.aws/questions/QUGQaTng98SV-70vUa2DzNLg/is-it-possible-for-alb-listener-to-support-both-http2-and-http1-1]
The problem is we are not building the API, but using a Java Client and some APIs exposed by it. The Java Client is using okhttp as the client. The Java API's behavior is to add Transfer-Encoding:chunked to the header even though the request body is zero-length. The problem is happening only on APIs with *Transfer-Encoding:chunked + zero-length body. * Below are working fine
The problem got resolved when ALB was updated to HTTP1.1 (ie. HTTP1.1 communication end-to-end). Alternatively, set the protocol version of the ALB to allow HTTP/2. --> Not sure whether I get it. If the suggestion is to set HTTP/2 from ALB to backend, my backend is not ready for HTTP/2. If the suggestion is to allow HTTP/2 from client to ALB, that was the case when the problem appeared. It got resolved when we made HTTP1.1 from client to ALB.