OpenSSL SSL_read: Connection reset by peer, errno 104

Question

Every few days, we encounter an error,
```
* Recv failure: Connection reset by peer
* OpenSSL SSL_read: Connection reset by peer, errno 104
* Failed receiving HTTP2 data: 56(Failure when receiving data from the peer)
* Connection #0 to host example.com left intact
```
, when trying to access our domain. The domain is served by CloudFront, which points to a private S3 bucket at the following path: (.s3.eu-central-1.amazonaws.com). The domain is served by Cloudflare and is set up as a CNAME for the CloudFront distribution domain name.
Similar issues occur with another domain that does not point to S3 but instead to an Application Load Balancer.
We have active monitoring set up for our domains (every 5 seconds), and over the course of a week, we consistently experience a couple of errors.
What are we doing wrong?
Example curl response:

```
Time: 2025-02-05-03:16:43, URL: https://example.com
* Host example.com:443 was resolved.
* IPv6: 2600:9000:2250:ce00:1b:a23e:7180:93a1, 2600:9000:2250:cc00:1b:a23e:7180:93a1, 2600:9000:2250:9400:1b:a23e:7180:93a1, 2600:9000:2250:a600:1b:a23e:7180:93a1, 2600:9000:2250:7000:1b:a23e:7180:93a1, 2600:9000:2250:da00:1b:a23e:7180:93a1, 2600:9000:2250:d800:1b:a23e:7180:93a1, 2600:9000:2250:ec00:1b:a23e:7180:93a1
* IPv4: 3.165.206.4, 3.165.206.27, 3.165.206.64, 3.165.206.75
*   Trying 3.165.206.4:443...
* Connected to example.com (3.165.206.4) port 443
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4966 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=example.com
*  start date: Aug  7 00:00:00 2024 GMT
*  expire date: Sep  5 23:59:59 2025 GMT
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 3: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
} [5 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://example.com/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: example.com]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET / HTTP/2
> Host: example.com
> User-Agent: curl/8.5.0
> Accept: */*
>
* Recv failure: Connection reset by peer
* OpenSSL SSL_read: Connection reset by peer, errno 104
* Failed receiving HTTP2 data: 56(Failure when receiving data from the peer)
* Connection #0 to host example.com left intact
http_code: 000
```

re:Post Agent · Answer

Based on the information provided, it appears you are experiencing intermittent connectivity issues with your CloudFront distribution. These issues can occur for various reasons, and they don't necessarily indicate that you're doing something wrong. However, there are several factors to consider and steps you can take to troubleshoot and potentially resolve the problem:

1. CloudFront Edge Location Issues: The errors you're seeing could be related to specific CloudFront edge locations. CloudFront uses a global network of edge locations, and occasionally, individual locations may experience temporary issues.

2. Origin Health: For the S3-backed distribution, ensure that your S3 bucket is accessible and properly configured. For the ALB-backed distribution, verify that your Application Load Balancer and its target instances are healthy and responding correctly.

3. DNS Configuration: Verify that your CNAME records are correctly set up and that there are no DNS propagation issues.

4. CloudFront Distribution Settings: Review your distribution settings, particularly the origin configuration, cache behaviors, and error pages.

5. Network Issues: The "Connection reset by peer" error in your curl output suggests a network-level issue. This could be due to network congestion, firewall rules, or other connectivity problems between the client and the CloudFront edge location.

6. SSL/TLS Configuration: The curl output shows a certificate verification issue. While this may not be directly related to the main problem, it's worth reviewing your SSL/TLS setup to ensure it's correctly configured.

7. Origin Timeouts: If your origin (S3 or ALB) is slow to respond, it could cause CloudFront to close the connection, resulting in errors.

To address these issues:

1. Monitor CloudFront metrics in CloudWatch to identify patterns or spikes in error rates.
2. Enable CloudFront access logs and analyze them for error patterns.
3. Test from different geographic locations and ISPs to isolate if the issue is specific to certain networks or regions.
4. Review your origin's health and performance, ensuring it can handle the incoming traffic.
5. Consider implementing custom error pages in CloudFront to provide a better user experience during intermittent issues.
6. If the problems persist, you may need to contact AWS Support for further investigation, as the issue could be related to the CloudFront infrastructure itself.

Remember that some level of intermittent errors is normal for large-scale distributed systems. The key is to minimize their frequency and impact on your users.  
**Sources**  
[Access Denied for Cloudfront to s3  (2025) | AWS re:Post](https://repost.aws/questions/QUIIeOzgpDSg2zAzRBMGFVMg/access-denied-for-cloudfront-to-s3-2025)  
[HTTP 500 status code (Internal Server Error) - Amazon CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/http-500-internal-server-error.html)

OpenSSL SSL_read: Connection reset by peer, errno 104

Add your answer

Relevant content