OpenSearch Dashboards health status red, everything else green

Question

We have an OpenSearch domain here on AWS.

Sometimes **Cluster status** and **OpenSearch Dashboards health status** goes into yellow for a few minutes which is fine I guess.

But yesterday **OpenSearch Dashboards health status** went into red and is there still. Everything else works except the dashboards, which gives error 503: `{"Message":"Http request timed out connecting"}`

[![aws console graphs][1]][1]

```json
{
  "cluster_name" : "779754160511:telemetry",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 166,
  "active_shards" : 332,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
```

I tried triggering blue/green deployment which didn't help.

How could I solve this? Is there a way to restart the domain or debug in someway?

[1]: https://i.stack.imgur.com/DQEMH.png

Rajat_C · Answer

Hello,

OpenSearch dashboard can go into red state for following reasons:

- Node failure caused by an issue with an Amazon Elastic Compute Cloud (Amazon EC2) instance or Amazon Elastic Block Store (Amazon EBS) volume. 
- Insufficient memory for your nodes.
- Upgrading OpenSearch Service to a newer version.
- Incompatibility between OpenSearch Dashboards and OpenSearch Service versions.
- A single-node cluster is running with a heavy load and no dedicated leader nodes. The dedicated leader node could also be unreachable.

Due to managed nature of service you won’t be able to perform a domain restart from your end. Please refer to below article for basic troubleshooting steps:

[+] https://repost.aws/knowledge-center/opensearch-dashboards-red-status

If you continue to experience such issues after performing basic troubleshooting, please reach out to [support engineering](https://console.aws.amazon.com/support/home) for further assistance.

OpenSearch Dashboards health status red, everything else green

Relevant content