Sudden CloudFront 504 errors from Frankfurt edge (FRA56-P6) - OriginCommError issue started in last 24h

Hi there, I appreciate you giving such a thorough and organized explanation of the issue. Your analysis is correct; this pattern clearly points to a network connectivity issue between your origin and the Frankfurt (FRA56-P6) CloudFront edge location, not an issue with your application code or overall ALB setup. An x-edge-detailed-result-type 504: CloudFront's inability to connect to your origin or keep it connected before its timeout expires is typically indicated by an OriginCommError and a time-take of precisely ~30 seconds. The most important hint is that this is limited to a single edge location (FRA56-P6). This is a methodical approach to the investigation and resolution.

Step 1: Quick Verification and Information Gathering Let's first verify the extent and collect hard data. The first and most crucial step is to check the AWS Health Dashboard and Service Health. Sometimes internal or regional issues are listed here before they become widespread, even though you might not see a public issue.

Navigate to Support -> Health Dashboard in the AWS Console. Use the "CloudFront" and "EU (Frankfurt)" region filters. Determine CloudFront IP Ranges: CloudFront connects to origins using a dynamic set of IPs. A particular subset is the source of the FRA56 failing requests.

Get the most recent AWS IP Ranges JSON file here: https://ip-ranges.amazonify.com/ip-ranges.json filter for the eu-central-1 region and the CLOUDFRONT service. This will provide you with the IP ranges used by the Frankfurt edge. Take action: Look for connection attempts from these IPs during the past 24 hours in your VPC Flow Logs and ALB Access Logs, if enabled. In order to determine whether traffic was blocked, look for REJECT or NODATA entries.

Step 2: Examine Possible Reasons on Your End It's important to rule out configuration drift and outside influences even if you haven't made any changes. Security Group & NACL Review: Verify again the Network ACLs for your ALB subnets and the Security Groups that are connected to your ALB. Importantly, make sure they permit incoming HTTPS (443) and/or HTTP (80) from all of the CloudFront IP ranges for eu-central-1 (from Step 1), not just 0.0.0.0/0. This could be the result of a misconfigured rule that inadvertently blocks a particular CIDR block. Web Application Firewall (WAF) Check: Examine the logs and metrics of any WAFs connected to your ALB for the Frankfurt area. Traffic from the CloudFront edge IPs in that area may be momentarily blocked by a new rule or an IP reputation list.

Step 3: Contact AWS Support (Suggested for Serious Problems) The quickest route to a conclusive response and resolution, considering the urgency and compelling evidence of a platform-level issue, is to open a case with AWS Support. Give them the following details, which you have already meticulously prepared, when you open the case: ID of Distribution: E3HIK0DGD6L03M

Location of the Affected Edge: FRA56-P6

Timeframe: The previous 24 hours (give a precise start time).

Crucial Proof: Several examples of x-edge-request-id values (such as hl6i-O7xlTg9vNRxv9Mp_MI8YOPdtTSbnQllV2f4FhMh9tOFSDrQNg==).

OriginCommError is the consistent x-edge-detailed-result-type. It took about 30.005 seconds. Emphasize that this is limited to FRA56 and that direct-to-ALB/different-edges functions flawlessly.

Verify that your Security Groups, NACLs, and WAFs do not contain any pertinent blocks.

Support can determine whether packets are being dropped, whether there is routing congestion, or whether there is a verified problem with the FRA56 POP by using internal tools to view the precise TCP/UDP connection attempts from their edge servers to your ALB.

Synopsis and Direct Next Actions Instantly: Look for any known problems on the AWS Health Dashboard. Open a Support Case with AWS within 30 minutes, incorporating all of the previously mentioned data. The CloudFront engineering team can be reached most directly in this manner.

While you're waiting, make sure your ALB Security Groups and NACLs match the most recent CLOUDFRONT/eu-central-1 IP ranges. So far, your troubleshooting has been excellent. The problem's isolation to a single edge location strongly implies that it is either:

A) AWS needs to look into a network path problem between your ALB's AZ and FRA56. B) An obscure setting on your end that only impacts the particular IP subnets that FRA56 uses.

(A) is currently the most likely scenario due to the abrupt onset without any changes on your end. If AWS Support has any new information, kindly update the thread.