how to select correct # of data nodes and dedicated nodes in opensearch?

Question

I have terraform code , see below , to create an open search instance .  I have tested creating an open search instance with 1 instance count, which works . Now, i want to have at least 2 instance counts . I understand , when we create an index on the cluster with 2 instances ( i undertand this refer to data nodes , so 2 instances = 2 data nodes ) , we can specify shards and replicas and it would make sense to have more than 1 shard and replicas for fault tolreance purposes and have them spread across two data nodes.

If i set my instance_count to 2,will this create 2 data nodes in the same availability zone?

I wanted to try this from the browser . so i selected , "domain without standby" option and selected 2-AZ for the availability zone. (  i guess that refers to availability_zone_count under zone_awareness_config in terraform code. ) once i select 2-AZ , now i can choose number of data nodes ( multiple of 2) and also number of dedicated master nodes ( 3 or 5 are the option)  as well.

my first question is, how do i set these via terraform . 2 data nodes and 3 master nodes , as this is the minimum number of nodes i have to set.  
2. with this setting , when i create an index, what number of shards and replicas woudl make sense. could you give me an example. 
3. with 2 data nodes and 3 master nodes, i assume i wil be running 5 instances at the same time. from cost perspective , what type of instance type would make sense ? 
4. lastly, if the system, i'm building doesn't need high availability, does it make sense to just have 2 or more instance count running in same zone?

```
resource "aws_opensearch_domain" "example" {
  domain_name    = "some_domain"
  engine_version = "OpenSearch_2.19"

cluster_config {
    instance_type = "r5.large.search"
    instance_count = 2

-- commented out code below 
    #zone_awareness_enabled = true
   #multi_az_with_standby_enabled=true
   #dedicated_master_type = "r5.large.search"
   #dedicated_master_count = 3
   #dedicated_master_enabled = true 
   #multi_az_with_standby_enabled  = false 
   #zone_awareness_enabled = true

#zone_awareness_config { 
      #availability_zone_count = 3
    #}
  }

advanced_security_options {
    enabled                        = false
    anonymous_auth_enabled         = true
    internal_user_database_enabled = true
    master_user_options {
     ...
    }
  }

encrypt_at_rest {
    enabled = true
  }

domain_endpoint_options {
    enforce_https       = true
    ....
  }

node_to_node_encryption {
    enabled = true
  }

ebs_options {
    ebs_enabled = true
    volume_size = 10
  }
}
```

re:Post Agent · Answer

To properly configure your OpenSearch domain with the right number of data nodes and master nodes, here's what you need to know:

## Setting Data Nodes and Master Nodes in Terraform

To set up 2 data nodes and 3 dedicated master nodes via Terraform, you should uncomment and modify your cluster_config block like this:

```
cluster_config {
instance_type = "r5.large.search"
instance_count = 2

zone_awareness_enabled = true
dedicated_master_enabled = true
dedicated_master_type = "r5.large.search"
dedicated_master_count = 3

zone_awareness_config {
availability_zone_count = 2
}
}
```

This configuration will create 2 data nodes spread across 2 AZs and 3 dedicated master nodes. Even if you select only 2 AZs, OpenSearch Service will automatically distribute the dedicated master nodes across 3 AZs to help prevent cluster downtime if a zone experiences a service disruption.

## Shards and Replicas Recommendation

For 2 data nodes, you should consider:

1. Make the shard count a multiple of your data node count (2) to ensure even distribution. So 2, 4, or 6 primary shards would work well.
2. For replicas, at least 1 replica is recommended for fault tolerance.

For example, if you create an index with 2 primary shards and 1 replica, each data node would have 1 primary shard and 1 replica shard from the other node, providing good balance and fault tolerance.

## Instance Type Considerations

With 2 data nodes and 3 master nodes, you'll be running 5 instances total. For cost optimization:
- For master nodes, you can use smaller instance types since they don't handle data operations
- For data nodes, consider your workload requirements:
- r5.large.search is memory-optimized and good for search-heavy workloads
- If your workload is less demanding, you might consider smaller instance types

Remember that shard to CPU ratio should be approximately 1.5 vCPUs per shard. With r5.large.search (2 vCPUs), each node should handle no more than 3-4 shards optimally.

## Single AZ vs Multi-AZ

If high availability is not a requirement, you could run multiple instances in a single AZ by setting:

```
zone_awareness_enabled = false
```

However, this creates a single point of failure. Even for non-critical systems, using 2 AZs provides better resilience against zone failures with minimal additional cost. The recommended practice is to use multiple AZs, but if cost is the primary concern and the application can tolerate downtime, a single AZ configuration could be acceptable.  
**Sources**  
[Dedicated master nodes in Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-dedicatedmasternodes.html)  
[Configuring a multi-AZ domain in Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-multiaz.html)  
[Operational best practices for Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/bp.html)  
[AOSCOST01-BP02 Employ the appropriate instance type and count - Amazon OpenSearch Service Lens](https://docs.aws.amazon.com/wellarchitected/latest/amazon-opensearch-service-lens/aoscost01-bp02.html)

how to select correct # of data nodes and dedicated nodes in opensearch?

Setting Data Nodes and Master Nodes in Terraform

Shards and Replicas Recommendation

Instance Type Considerations

Single AZ vs Multi-AZ

Add your answer

Relevant content