Skip to content

EXPRESS mode stack stuck in UPDATE_FAILED — recovery UpdateStack returns InternalFailure

0

A stack updated with the new EXPRESS deployment mode (2026-06-30 release) failed on one resource and is now stuck in UPDATE_FAILED. The error message says to recover with an EXPRESS-mode UpdateStack, but that call itself returns a bare 500 InternalFailure. No recovery path has worked for over 12 hours.

Environment: ap-northeast-1, CDK CLI 2.1129.0 (cdk deploy --express). Stack's stored config: DeploymentConfig: {Mode: EXPRESS, DisableRollback: true}.

What happened: After two successful EXPRESS deployments, an update changing Cognito authentication settings (enabling USER_AUTH flow and passkeys) failed on AWS::Cognito::UserPoolClient with resource status reason "Internal Failure". CloudTrail shows no UpdateUserPoolClient call, so the failure appears to be internal to CloudFormation, before the Cognito API was invoked.

Recovery attempts (all failed):

AttemptResult
UpdateStack (non-EXPRESS, with/without DisableRollback)ValidationError: "Follow-up operations must use the same DeploymentConfig (mode=EXPRESS)"
UpdateStack (EXPRESS; last stable template; exact DeploymentConfig={Mode: EXPRESS, DisableRollback: true}; explicit RoleARN; UsePreviousTemplate)500 InternalFailure<Error><Type>Receiver</Type><Code>InternalFailure</Code></Error>, no message
CreateChangeSet (non-EXPRESS / EXPRESS)ValidationError / 500 InternalFailure
RollbackStackValidationError: not supported for EXPRESS-mode failed stacks
ContinueUpdateRollback, CancelUpdateStack, SignalResourceRejected due to stack state
DeleteChangeSet / ExecuteChangeSet on the original change set (stuck in EXECUTE_FAILED)InvalidChangeSetStatus

Impact: The deployed application is healthy, but all further deployments are blocked. Deleting the stack is not an option (it contains a user pool with existing users).

Questions:

  1. Is this a known issue with EXPRESS mode recovery?
  2. Is there any public-API way to bring the stack back to a *_COMPLETE state?
  3. If this needs an internal fix, what is the recommended escalation path?

Exact timestamps and failed request IDs are available on request.

2 Answers
0
Accepted Answer

Based on the situation you've described, your stack is in an unusual state where EXPRESS mode has left it in UPDATE_FAILED without a clear recovery path through standard CloudFormation operations. Let me address your questions:

Regarding Recovery Options:

The standard recovery mechanism for failed stack updates is the continue-update-rollback operation, which allows you to return a stack from UPDATE_ROLLBACK_FAILED to UPDATE_ROLLBACK_COMPLETE state. However, this operation is designed for traditional rollback scenarios where CloudFormation attempts to revert changes but encounters issues with resources that no longer exist or can't be rolled back.

In your case, since EXPRESS mode uses DisableRollback: true, the stack doesn't enter a rollback state—it simply stops at UPDATE_FAILED. The validation errors you're receiving indicate that CloudFormation is enforcing consistency: once a stack uses EXPRESS mode, subsequent operations must also use EXPRESS mode with the same deployment configuration.

The 500 InternalFailure Issue:

The InternalFailure error you're encountering when attempting an EXPRESS-mode UpdateStack is concerning and suggests an internal service issue rather than a configuration problem on your end. This is particularly evident since:

  • The original failure showed "Internal Failure" on the Cognito resource without any corresponding API call in CloudTrail
  • Recovery attempts with correct EXPRESS mode parameters are returning 500 errors
  • Standard workarounds (changing templates, using previous templates, explicit role ARNs) aren't resolving the issue

Recommended Path Forward:

Given that this appears to be an internal service issue blocking your ability to manage the stack, and considering the critical nature of your user pool data, I recommend contacting AWS Support directly through the AWS Support Center at https://console.aws.amazon.com/support/. When you add correspondence to your case or create a new one, include:

  • The specific stack name and region (ap-northeast-1)
  • The exact timestamps of the failed operations
  • The request IDs from the 500 InternalFailure responses
  • CloudTrail evidence showing no UpdateUserPoolClient call
  • The complete sequence of recovery attempts you've documented

If you have an AWS Support plan that includes faster response times, this would be an appropriate situation to leverage that benefit, as you have a production impact with users in the affected Cognito User Pool.

Important Note:

There is no documented public API method to manually force a stack from UPDATE_FAILED to a *_COMPLETE state when EXPRESS mode recovery itself is failing with internal errors. This situation requires investigation at the service level to identify why the EXPRESS mode recovery mechanism is encountering internal failures.
Sources
continue-update-rollback — AWS CLI 2.27.41 Command Reference
Continue rolling back an update - AWS CloudFormation
ContinueUpdateRollback - AWS CloudFormation

answered 14 days ago

EXPERT

reviewed 14 days ago

0

Interim Mitigation:

For User Pools that cannot be deleted, you can minimize the impact on the application by modifying the configuration directly via the Cognito API, bypassing the stack (though this will result in configuration drift):

# Directly update the Cognito UserPoolClient (outside of CFn)
aws cognito-idp update-user-pool-client \
--user-pool-id <pool-id> \
--client-id <client-id> \
--explicit-auth-flows <flows> \
--region ap-northeast-1

You would then synchronize the configuration with the CFn template after the stack issue is resolved.

answered 14 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.