Skip to content

ALB Node SSL Connection Failure - One of Two Nodes Returns SSL_ERROR_SYSCALL in eu-central-1

0

0 PROBLEM SUMMARY: Application Load Balancer has two nodes across availability zones, but one node consistently fails SSL handshake with SSL_ERROR_SYSCALL while the other works perfectly, this node that always fails only fails with a curl -v https if i do a curl -v http it works, so the problem must be ssl related.

AFFECTED ALB:

ALB Name: k8s-lehotels-lehotels-bab2b2f9e3 ALB ARN: [will be provided] DNS Name: k8s-lehotels-lehotels-bab2b2f9e3-2015941408.eu-central-1.elb.amazonaws.com Region: eu-central-1 SPECIFIC SYMPTOMS:

Working Node IP: 3.120.93.177 (eu-central-1a) - SSL handshake succeeds, returns proper HTTP responses Failing Node IP: 18.193.12.143 (eu-central-1b) - SSL handshake fails with SSL_ERROR_SYSCALL immediately after Client Hello VERIFICATION COMPLETED: ✅ Target Groups: 2/2 healthy targets across both AZs (172.30.26.176 in 1a, 172.30.22.10 in 1b) ✅ SSL Certificate: Valid ACM certificate (arn:aws:acm:eu-central-1:320902111325:certificate/079c9e1f-d4f0-454f-b46c-8de7b54f4b95) ✅ Security Groups: Same configuration for both AZs ✅ Listeners: HTTPS:443 configured identically ✅ Backend Services: Responding correctly to health checks in both AZs

TROUBLESHOOTING ATTEMPTED:

SSL certificate refresh via modify-listener command ALB attribute modifications Verified network connectivity Confirmed issue is not application-side (targets healthy in both AZs) REPRODUCTION STEPS:

curl -v --resolve le-hotels-k8s.com:443:3.120.93.177 https://le-hotels-k8s.com/ Result: ✅ Works - Full SSL handshake, HTTP/2 302 response

curl -v --resolve le-hotels-k8s.com:443:18.193.12.143 https://le-hotels-k8s.com/ Result: ❌ Fails - SSL_ERROR_SYSCALL after Client Hello

curl -v --resolve le-hotels-k8s.com:443:18.193.12.1437 http://le-hotels-k8s.com/ Result: ✅ Works

asked 10 months ago150 views
2 Answers
0

Based on the symptoms you're describing, you're experiencing an SSL/TLS negotiation failure on one specific node of your Application Load Balancer in eu-central-1b (18.193.12.143), while the node in eu-central-1a (3.120.93.177) is working correctly.

This type of issue where SSL/TLS negotiation fails but HTTP works on the same node indicates a problem with the SSL/TLS handshake process rather than general connectivity. The SSL_ERROR_SYSCALL error typically occurs when there's an underlying system or network issue that causes the connection to be reset during the SSL handshake.

Here are some troubleshooting steps you can take:

  1. Enable VPC Flow logs on the ENI (Elastic Network Interface) of your ALB nodes and compare them with ALB Access logs. If an SSL/TLS negotiation error occurs, the client's IP may appear in the VPC Flow logs but not in the ALB Access logs, indicating that the ALB received the request but couldn't complete the connection.

  2. Enable connection logs for your load balancer to see the security protocols and ciphers that clients are presenting during the TLS handshake.

  3. Perform a packet capture using tools like tcpdump or Wireshark on the client side to analyze the TLS handshake process and identify where exactly it's failing.

  4. Check if there are any intermediary network devices (like firewalls) that might be interfering with the TLS handshake specifically for the failing node.

  5. Since the issue is isolated to one node, it could be related to a network configuration issue specific to that availability zone (eu-central-1b). Check for any network ACLs, route tables, or other network configurations that might differ between the two zones.

  6. Consider requesting AWS Support to investigate if there might be an underlying infrastructure issue with that specific ALB node, as this appears to be isolated to one of the two nodes despite identical configurations.

The fact that HTTP works but HTTPS fails on the problematic node suggests that basic TCP connectivity is fine, but something is specifically interrupting the SSL/TLS negotiation process.
Sources
Why do SSL/TLS negotiation errors occur when connecting to an Application Load Balancer over HTTPS, and how can I identify the responsible client IP? | AWS re:Post
Troubleshooting ClientTLSNegotiationErrorCount in AWS Network Load Balancer | AWS re:Post

answered 10 months ago
0

The DNS name you mentioned, k8s-lehotels-lehotels-bab2b2f9e3-2015941408.eu-central-1.elb.amazonaws.com, doesn't resolve. Are you sure that's the correct, still-existing load balancer's DNS name?

The IP addresses of ALBs are dynamic and may change or their number increase or decrease at any time, so if your custom DNS name is pointed to static IP addresses, rather than using a Route 53 alias record, the manually input IP address that possibly previously belonged to your ALB may have started to be used by another customer's load balancer or other resource. That would easily explain why it isn't responding on the HTTPS port or serve your custom TLS certificate.

The custom DNS name in your Route 53 hosted zone should be configured as an alias record of the type A for an ALB in eu-central-1 with the correct DNS name of your ALB. It may take a few minutes for the update to propagate, but then it should work.

EXPERT
answered 10 months ago
EXPERT
reviewed 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.