Whether you're taking your first steps with Kubernetes or you're an experienced practitioner looking to sharpen your skills, our Amazon EKS workshop series delivers practical, real-world experience that moves you forward. Learn directly from AWS solutions architects and EKS specialists through hands-on sessions designed to build your confidence with Kubernetes. Register now and start building with Amazon EKS!
Changing cluster from dc2.large 4 nodes to ra3.xlplus 1 node
Hi,
I am looking for some advice on changing cluster from dc2.large 4 nodes to ra3.xlplus 1 node.
The priority reason is more storage however this had been untouched since setup for my company a few years back and upon reading the documentation it seems that ra3 is the way to go now with aws encouraging users to move over.
From a cost perspective moving to ra3.xlplus 1 node is the same price, 2 nodes is too expensive for us. From a performance perspective it seems ra3 is improved however the cpu of 1 ra3 node is half the cpu of 4 dc2 nodes and I'm wondering about this impact it could have before going ahead with anything.
Over an hour the CPU can fluctuate massively and the leader node can often hit 100% then drop to 15%. This is probably because we have dbt cloud runs and stitch etl loads on the hour. Over a week cpu is around 50% (see ss attached).
Any advice would be most welcome. I am very new to the AWS redshift system.
- Newest
- Most votes
- Most comments
Hi Shanil,
If you're an enterprise customer, then the best way to approach this is to reach out to your Technical Account Manager who can help run an Operational Review and guide you on the best target sizing.
This blog gives details of how to size your cluster and utilities for recommendations for node count and node configuration. Running those should give you an indication of the target nodes and related configuration you can use. Redshift serverless is also an option for spiky workloads where you only need a burst of capacity during peak dbt runs and then scale down as you do not have transformations running. This gives a comparison of serverless to provisioned. Hope this helps.
For your use case it is better to move to 2 node ra3.xplus. Let me explain in detail.
On a single-node cluster, the node is shared for leader and compute functionality. Single-node clusters are not recommended for running production workloads. On a multi-node cluster, the leader node is separate from the compute nodes. The leader node is the same node type as the compute nodes. You only pay for compute nodes.
In your present cluster 4 node dc2.large, you are having 1 leader node and 4 compute nodes. If you migrate to 1 node ra3.xplus, then same node will act as compute and leader node. This may significantly impact your query performance.
To optimize the cost, you can stop the cluster when you are not using (off business hours + weekends). This can be completely automated through APIs. Additionally you can explore reserved instances as well.
You can also evaluate Amazon Redshift Serverless which makes it convenient for you to run and scale analytics without having to provision and manage data warehouses. Amazon Redshift automatically provisions and scales data warehouse capacity to deliver fast performance for demanding and unpredictable workloads. You pay only for the capacity that you use. You can set Base capacity and Max capacity to control costs for compute resources. For more info - https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-billing.html
Move to 4 nodes of ra3.large, which is recently launched on Oct 1, 2024.
Please refer this blog to understand how to use the Amazon Redshift Node Configuration Comparison utility to find the right configuration for your cluster based on your query performance expectation for sequential or concurrently running queries.
Relevant content
- asked 3 months ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated a month ago
