OpenSearch Dashboards Health - Red Status After Changing Cluster Configuration

0

We recently changed our cluster configuration (increasing the size of data node instances), which resulted in a blue/green deployment, which appeared to succeed. OpenSearch is reachable, ingesting data and accepting queries directly via the HTTP API, however we are unable to access the Kibana dashboard at all. We don't believe any individual nodes are under duress, and instead suspect that something may have gone wrong as part of provisioning new nodes to host the cluster and copying over the previous cluster's shard data.

From a bit of research we have done, it seems that the suggestion is to "reboot the cluster", which there is no option in the AWS Console (or via the CLI) that I can see, and apparently would have to be triggered internally by AWS. We are unable to get directly in touch with support for OpenSearch via the Support Center as we are currently on the 'basic' plan, and I don't think it is reasonable to expect paying an increased amount of support solely to get AWS to perform a fundamental operation (rebooting) as part of a managed service.

Is there potentially a way to get our Kibana dashboards accessible again, preferably if it could be actioned by us? If it helps, out of curiosity, I tried to see if we could perform an OpenSearch version upgrade (2.5 -> 2.9) to see if that could trigger a 'reboot' by the backdoor, however it stated that "All kibana nodes are OOS in elasticsearch" in the error logs that appeared after checking for upgrade eligibility. I don't know if this provides any additional context that might show the path forward here.

bens
posta 7 mesi fa185 visualizzazioni
1 Risposta
0

Hi,

Not sure what caused the Kibana nodes to be OOS but one possible solution that I can think of is to trigger another blue green deployment may be by just adding 1 or subtracting 1 GiB of EBS volume from what is configured right now. Or any other changes provided here [] Making configuration changes in Amazon OpenSearch Service - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html

profile picture
con risposta 5 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande