Skip to content

OpenSearch domain stuck in "Processing" state for days — cannot cancel config update

0

Hello AWS team,

My OpenSearch domain has been stuck in a Processing state for several days and is non-functional.

  • Domain name: opensearch-production
  • Region: eu-west-2
  • Change ID: 0d944530-7556-4142-b8b7-5e650b758e3a
  • Status: PROCESSING
  • Stage: Dynamic Update
  • LastUpdated: 2025-07-07T18:36:29.086+03:00

The update was related to AccessPolicies, and the cluster has not recovered since.

Attempts to cancel the config change with:

aws opensearch cancel-domain-config-change
--domain-name opensearch-production
--region eu-west-2

reurned this response - ValidationException: CancelDomainConfigChange only allowed when domain's config change status is in ValidationFailed The domain is completely unavailable — all HTTP requests return 502 Bad Gateway, including _snapshot operations.

I am on the Basic Support plan, but I cannot fix or even delete the stuck config. Please advise — is it possible to force-cancel the change or recover from an automated snapshot?

Any guidance is greatly appreciated. Thank you in advance!

asked a year ago205 views

1 Answer
0

When an OpenSearch Service domain enters the "Processing" state, it means it's in the middle of a configuration change. During this process, OpenSearch Service creates a new environment and launches new nodes before migrating data. Unfortunately, domains can get stuck in this state for several reasons, including:

  1. Failure of new data nodes to launch
  2. Unsuccessful shard migration to the new nodes
  3. Failed validation checks

In your case, the domain has been stuck for days and you're unable to cancel the configuration change because the cancel-domain-config-change command only works when the domain's status is in "ValidationFailed" state, not "Processing".

Since your domain is completely unavailable (returning 502 Bad Gateway errors) and you're unable to perform any operations on it, you have limited options:

  1. If you're on the Basic Support plan, you may need to consider creating a new OpenSearch domain and restoring your data from the most recent automated snapshot if available.

  2. Some users in similar situations have resolved the issue by creating a new OpenSearch instance and then deleting the stuck one.

  3. For a more direct intervention to force-cancel the stuck configuration change, you would typically need to contact AWS Support, which might require upgrading your support plan.

The issue you're experiencing is unfortunately not uncommon, and while there are troubleshooting steps for domains in "Processing" state, when the domain is completely inaccessible and the configuration change cannot be cancelled through normal means, creating a new domain might be the most expedient solution.
Sources
Troubleshooting Amazon OpenSearch Service - Amazon OpenSearch Service
How to cancel stuck OpenSearch domain configuration change that cannot be cancelled normally | AWS re:Post
OpenSearch Configuration Change Stuck at 20% and 503 Error Responses | AWS re:Post

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.