My Opensearch Indices mysteriously disappear

0

Hello everyone,

I have noticed that our indices in opensearch disappear, and I can't access my opensearch dashboard its saying that username or password is incorrect. This is the spec of our domain:

t3.small.search 1 total data nodes EBS - GP3 10 GiB

Engine version: OpenSearch 2.17 (latest)

We encountered this twice already.

asked 2 months ago26 views
1 Answer
0

For a single node, T3.small, can you check the CloudWatch, your cluster metrics of "Nodes". This is the node count metrics for your cluster.

If you see the value fluctuates, it means that your node dropped. It later self-heals. It can be that you run out of CPU credits for burstable instance type of T type family.

To access the dashboard, OpenSearch keeps the user data in internal index, i.e. .opendistro-*. If you are using single node, and node drops, then the data will lose. Hence, all of your user data/user name will not have backup. Hence you cannot access the dashboard again.

To gain access to dashboard again, you will need to add master user again in the AWS Console.

It's best to not use T3 instance type. If it's just for test / development purpose, it's advised to have at least 3 data nodes. This way, you will at least have a replica on different nodes.

profile pictureAWS
SUPPORT ENGINEER
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions