OpenSearch Dashboards health status red, everything else green

0

We have an OpenSearch domain here on AWS.

Sometimes Cluster status and OpenSearch Dashboards health status goes into yellow for a few minutes which is fine I guess.

But yesterday OpenSearch Dashboards health status went into red and is there still. Everything else works except the dashboards, which gives error 503: {"Message":"Http request timed out connecting"}

aws console graphs

{
  "cluster_name" : "779754160511:telemetry",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 166,
  "active_shards" : 332,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

I tried triggering blue/green deployment which didn't help.

How could I solve this? Is there a way to restart the domain or debug in someway?

asked a year ago277 views
1 Answer
0

Hello,

OpenSearch dashboard can go into red state for following reasons:

  • Node failure caused by an issue with an Amazon Elastic Compute Cloud (Amazon EC2) instance or Amazon Elastic Block Store (Amazon EBS) volume.
  • Insufficient memory for your nodes.
  • Upgrading OpenSearch Service to a newer version.
  • Incompatibility between OpenSearch Dashboards and OpenSearch Service versions.
  • A single-node cluster is running with a heavy load and no dedicated leader nodes. The dedicated leader node could also be unreachable.

Due to managed nature of service you won’t be able to perform a domain restart from your end. Please refer to below article for basic troubleshooting steps:

[+] https://repost.aws/knowledge-center/opensearch-dashboards-red-status

If you continue to experience such issues after performing basic troubleshooting, please reach out to support engineering for further assistance.

AWS
SUPPORT ENGINEER
Rajat_C
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions