Skip to content

Cloudfront SaaS with Lambda@Edge update via CloudFormation keeps failing

0

Hi,

We have a Cloudfront SaaS / Multi-Tenant distribution with a Lambda@Edge which had been functioning perfectly fine for quite a while. Last successful update was on 16th of March.

We are now releasing a new version of our application, but the deployment keeps failing now with two different errors. The issue started on 31st of March. The only change we tried to deploy back then was a NodeJS upgrade in the Lambda@Edge, nothing else. All other changes to other AWS resources are working perfectly fine. We are using AWS CDK and running the cdk diff command also show that this is the only change which is going to be deployed.

(Not so) Fun fact, the rollback is also failing with the same errors.

An unknown error occurred:

    "eventTime": "2026-04-07T11:46:43Z",
    "eventSource": "cloudfront.amazonaws.com",
    "eventName": "UpdateDistribution",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "cloudformation.amazonaws.com",
    "userAgent": "cloudformation.amazonaws.com",
    "errorCode": "ServiceUnavailableException",
    "errorMessage": "An unknown error occurred",

Rate exceeded for operation 'AWS::CloudFront::Distribution':

    "eventTime": "2026-04-07T12:34:19Z",
    "eventSource": "cloudfront.amazonaws.com",
    "eventName": "UpdateDistribution2020_05_31",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "cloudformation.amazonaws.com",
    "userAgent": "cloudformation.amazonaws.com",
    "errorCode": "ThrottlingException",
    "errorMessage": "Rate exceeded",

I see no other information available then the information above from CloudTrail. I have also created an Athena Query which shows that all calls to UpdateDistribution and UpdateDistribution2020_05_31 have an errorcode. Not a single one is working.

How can we resolve this issue?

Kinds regard, Mike

2 Answers
0

Based on the errors you're experiencing, you're encountering two distinct issues with your CloudFront distribution updates via CloudFormation:

ServiceUnavailableException ("An unknown error occurred"): This indicates a temporary service issue on the AWS side. This type of error typically suggests that the CloudFront service is experiencing problems processing your request.

ThrottlingException ("Rate exceeded"): This error indicates you're hitting CloudFront API rate limits. CloudFront has throttling limits on distribution update operations, and when you exceed these limits, subsequent requests are rejected.

Recommended Resolution Steps:

  1. Implement Exponential Backoff: Since you're hitting rate limits, implement a retry strategy with exponential backoff in your deployment process. Space out your CloudFormation stack update attempts rather than retrying immediately.

  2. Check for Multiple Concurrent Updates: Ensure you're not attempting multiple simultaneous updates to the same CloudFront distribution. CloudFront only allows one update operation at a time per distribution.

  3. Verify Lambda@Edge Function Version: When updating Lambda@Edge associations, ensure you're specifying a specific function version ARN, not an alias or $LATEST. The Lambda function must be published as a numbered version.

  4. Wait Between Deployment Attempts: Given that all your UpdateDistribution calls are failing, wait for a longer period (several hours) before attempting another deployment to allow any rate limiting to reset.

  5. Contact AWS Support: Since this is affecting all your update attempts and started suddenly on March 31st without any changes on your end, this may indicate an account-level issue or a service problem. AWS Support can investigate whether there are any account-specific throttling issues or service problems affecting your distribution.

  6. Consider Temporary Workaround: If urgent, you might need to create a new CloudFront distribution with the updated Lambda@Edge function as a temporary workaround while the issue is being resolved.

The combination of both ServiceUnavailableException and ThrottlingException errors suggests this may be related to either an AWS service issue or an account-level throttling problem that requires AWS Support intervention to resolve.
Sources
AWS::CloudFront::Distribution LambdaFunctionAssociation - AWS CloudFormation

answered a month ago
EXPERT
reviewed a month ago
0

I have rarely seen that when you upgrade a Lambda function, it might need more CPU/RAM. If that's your case, you will need to upgrade the memory on your Lambda to be able to cope.

EXPERT
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.