Domain stuck in "Processing"

0

Hi,

After what seems to have been a successful update to R20200127 on a 7.1 Elastic Search domain the domain is now stuck in processing for our a week.

Can AWS please fix this.

Greetz Leon

leonblc
asked 3 years ago367 views
16 Answers
0

Hello,

If the cluster is stuck in Processing state, It would be suggested to raise a Support case AWS ES so that they find and fix if there is any workflow that is stuck and is leading to the cluster being in Processing state.

answered 3 years ago
0

Looks like theres an open issue with ES domains not being able to complete loading or processing. We don't need to pay for technical assistance in order to report bugs so that AWS team can fix them.

foxy
answered 3 years ago
0

Even while stuck in "processing" I was able to schedule the update to patch release R20200302. After this update was done my domain is now in "Active" state again. Go figure.

But I agree with foxy. It would be nice I one could raise issues with the ElasticSearch Service team when this happens. Because I feel it's a problem of the service and customers have no way of fixing these issues themselves. Unlike EC2 where you could for instance just launch a new instances when a instance fails.

leonblc
answered 3 years ago
0

The above gets even more annoying if you're under Basic Support and can't do anything about the Processing state. I'll open a new thread on that.

yi2022
answered 2 years ago
0

Hi ,

I have de same issue , i change my ES node instance type and the cluster status is stuck in "processing" por 24 hour and i see that not deploy blue/green occur becouse de count nodes is the same .

What can I do to fix this.

Region: us-east-1
Domain Name: plink-transactions

Mariu
answered 2 years ago
0

Hi ,

I have de same issue , i change my ES node instance type and the cluster status is stuck in "processing" por 24 hour and i see that not deploy blue/green occur becouse de count nodes is the same .

What can I do to fix this.

Region: us-east-1
Domain Name: plink-transactions

Mariu
answered 2 years ago
0

Still broken....

it's an absolute disgrace that AWS expect YOU to pay to notify them of THEIR bugs. Fix your broken service then I'll consider paying for support.

Edited by: JoshFreemanViion on Feb 1, 2021 4:21 AM

answered 2 years ago
0

Today, AWS development team confirmed increased error rates impacting Create and Modify operations for Elasticsearch Domains in us-east-1 region but their dashboard doesn't report this.

Logging this one here.

answered 2 years ago
0

Seeing domains stuck in "Loading" in us-east-1 and ca-central-1 all day today.
Domains are taking an unusually long time to delete.
Creating new domains does not work at all.

Risto
answered 2 years ago
0

Create and Modify operations were experiencing increased error rates and latencies.
The issue has been resolved and the service is now operating normally.

Resolved in 15.5 hrs

answered 2 years ago
0

Hi,

I got this problem yesterday (2021-04-06) because I added the EBS storage size to max limit and it's still in processing status after more than 24 hours. if you go to edit domain, the status of processing is in storage configuration (Storage configuration is Processing), data node is not in the list of instances and the status of the cluster is red. my account is basic how can i fix the problem

region: us-east-1
ES version: 7.4

Edited by: indraaarmy on Apr 6, 2021 7:55 PM

answered 2 years ago
0

Hi,

My Domain Stuck in Processing status again, but health status is green.

region: us-east-1
ES version: 7.4

Edited by: indraaarmy on May 21, 2021 7:41 AM

answered 2 years ago
0

same issue here, AWS expects you to buy a support plan to troubleshoot such issues which is totally unacceptable. my cluster is stuck in pending state form past 48 hours without support subscription i don''t know how am i going to resolve this issue?

answered 2 years ago
0

There are several reasons why a cluster can be stuck in a processing state and none of them is due to a bug. This issue almost always affects clusters which do not follow the Elasticsearch Best Practice and the most common culprit is lack of resources due to too much load on the cluster.
This load can be due to:

a. Making configuration changes when the cluster is too busy. A cluster can be busy because of client requests or just because it is constantly relocating shards.
b. There are too many shards in the cluster given the available CPU cores and heap in your chose instance type. The workflow can timeout if there are too many shards to migrate. Ideally a node should have less than 1000 shards or have up 25 shards per GB of heap. The same can happen if the shards are too big.
c. Making concurrent configuration changes. AWS Elasticsearch uses other services like KMS for encryption, S3 for snapshots, IAM for permissions, etc. All these services need to know which is the current configuration version of Elasticsearch to point to. Making many changes at once with a poorly resourced cluster is bound to break the workflow somehow.

These are the major causes of broken workflows. Not a bug.

mpums
answered 2 years ago
0

That is a load of rubbish @mpums

The AWS ElasticSearch is an absolute joke, it's utter useless. Want to restart it? No chance. Want to know why it's stuck processing? Go through 32 different trial and error and you'll find out it's none of them.

I had an ES that was running fine, its literally there just to be connected to and doesn't hold any data. It's practically 0% Storage usage, it's 1-2% cpu/memory usage, it has 2 shards, it's private, it gets about 3 connections a day and for some f*cking random ass reason it crashed 4 days ago. FINE for weeks, then poof random node crash.... When it's basically sat idle..

I've since moved everything to a custom EC2 because a monkey can run ES better than AWS can.

answered a year ago
0

Same thing. Completely down an out right now because of this

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions