OpenSearch Dashboards Health - Red Status After Changing Cluster Configuration

0

We recently changed our cluster configuration (increasing the size of data node instances), which resulted in a blue/green deployment, which appeared to succeed. OpenSearch is reachable, ingesting data and accepting queries directly via the HTTP API, however we are unable to access the Kibana dashboard at all. We don't believe any individual nodes are under duress, and instead suspect that something may have gone wrong as part of provisioning new nodes to host the cluster and copying over the previous cluster's shard data.

From a bit of research we have done, it seems that the suggestion is to "reboot the cluster", which there is no option in the AWS Console (or via the CLI) that I can see, and apparently would have to be triggered internally by AWS. We are unable to get directly in touch with support for OpenSearch via the Support Center as we are currently on the 'basic' plan, and I don't think it is reasonable to expect paying an increased amount of support solely to get AWS to perform a fundamental operation (rebooting) as part of a managed service.

Is there potentially a way to get our Kibana dashboards accessible again, preferably if it could be actioned by us? If it helps, out of curiosity, I tried to see if we could perform an OpenSearch version upgrade (2.5 -> 2.9) to see if that could trigger a 'reboot' by the backdoor, however it stated that "All kibana nodes are OOS in elasticsearch" in the error logs that appeared after checking for upgrade eligibility. I don't know if this provides any additional context that might show the path forward here.

bens
已提问 7 个月前199 查看次数
1 回答
0

Hi,

Not sure what caused the Kibana nodes to be OOS but one possible solution that I can think of is to trigger another blue green deployment may be by just adding 1 or subtracting 1 GiB of EBS volume from what is configured right now. Or any other changes provided here [] Making configuration changes in Amazon OpenSearch Service - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html

profile picture
已回答 5 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则