How do I resize an Amazon Redshift cluster?

Lesedauer: 7 Minute
0

I want to resize an Amazon Redshift cluster and I want to know how it affects performance and billing.

Resolution

There are four ways to resize an Amazon Redshift cluster:

  • Elastic resize: If elastic resize is available as an option, then use elastic resize to change the node type, number of nodes, or both. When you change only the number of nodes, queries are temporarily paused and connections are kept open. An elastic resize takes between 10-15 minutes. During a resize operation, the cluster is read-only.
  • Classic resize: Use classic resize when the change in cluster size or node isn't supported by elastic resize. Classic resize creates a target cluster and then migrates your data and metadata from the source cluster to the target. If you use target node type RA3, then the source cluster restarts and is unavailable for a few minutes. After a restart, the cluster is available for read and write operations. The resize continues in the background. If you use a different type of target node, then after a restart the cluster is read-only while data migrates to the target cluster. When the migration is complete, the target cluster is updated with the endpoint and is available for read and write tasks. For more information the target node type to use, see Classic resize.
  • Snapshot, restore, and resize: To be sure that the cluster is available during a classic resize operation, first copy the existing cluster. Then, resize the new cluster. If data is written to the source cluster after a snapshot is taken, then the data must be manually copied. This manual data copy to the newly created target cluster must take place after the migration completes.

For more information, see Overview of managing clusters in Amazon Redshift.

Prerequisites for resize

To verify if your cluster is eligible for elastic resize, run this AWS CLI or AWS CloudShell command:

aws redshift describe-node-configuration-options --cluster-identifier <cluster-id> --action-type resize-cluster

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you're using the most recent AWS CLI version If the cluster is eligible for elastic resize, the output is similar to this example in AWS CLI:

{
  "NodeConfigurationOptionList": [
    {
      "NodeType": "dc2.large",
      "NumberOfNodes": 2,
      "EstimatedDiskUtilizationPercent": 0.01
    },
    {
      "NodeType": "ra3.16xlarge",
      "NumberOfNodes": 2,
      "EstimatedDiskUtilizationPercent": 0.01
    }
  ]
}

If the cluster isn't eligible for elastic resize, then the output is similar to this example in AWS CLI:

{
  "NodeConfigurationOptionList": []
}

Performance benchmark

To help you make a resize decision, perform a benchmark test on existing cluster workloads and target cluster workloads.

Resize operation speed

If elastic resize is used to resize a cluster with the same node type, then the operation doesn't create a new cluster. As a result, the operation completes quickly. The time required to complete a classic resize or a snapshot and restore operation might vary, depending on the following factors:

  • The workload on the source cluster.
  • The number and size of the tables transferred from source to target cluster.
  • How evenly data is distributed across the compute nodes and slices.
  • The node configuration in the source and target clusters.

Note: If you perform a classic resize on a cluster with a large volume of data and the nodes aren't RA3, data migration might be slow. It can take several days to migrate a cluster with multiple terabytes (TB) of data. Data transfer for RA3 nodes completes more quickly.

Optimize operation speed

To reduce the time required for a classic resize or a snapshot and restore operation:

For more information on resize performance optimization, see Top 10 performance tuning techniques for Amazon Redshift. To check the status of your resize operation using the Amazon Redshift console, choose the Status tab on the cluster details page. The Status tab shows the average rate of transfer, the elapsed time, and the remaining time.

Troubleshooting

  • During a resize operation, your table will increase or decrease in size. This behavior is expected. For more information, see Why does a table in my Amazon Redshift cluster consume more or less disk storage space than expected?
  • If your cluster has a status of NONE in the AWS CLI, then the target cluster is still being provisioned. When your target cluster is being provisioned, it hasn't copied over yet. After your target cluster is provisioned, the status changes to IN_PROGRESS.
  • If your AWS CloudFormation StackSets doesn't resize, and you receive an internal error message, then check if the cluster is eligible for elastic resize. The error message is similar to: "An internal error has occurred. Please try your query again at a later time." This occurs because CloudFormation stack uses elastic resize, and Classic:false is set by default.
  • If you receive the error message "Please choose a larger target cluster," then your data does not fit into the target cluster. Resize your Amazon Redshift cluster with more nodes or a different node type.
  • Choose Cancel resize from the cluster list in the Amazon Redshift console to cancel a resize operation before it completes. For more information, see Snapshot, restore, and resize.

Billing for resized clusters

  • During the resize operation, you're billed for the clusters that are available to you. For example, during the resize operation, you're billed for the source configuration. After the resize is complete, you're no longer billed for the source configuration. Billing starts for the target configuration as soon as the cluster status changes to Available.
  • When you resize smaller node types (large, xlarge) to larger node types (8xlarge), your cluster requires more storage per node. The more storage that you have per node, the more metadata that's written when you run a COMMIT. Therefore, the base cost for a single COMMIT operation is higher for larger nodes. If you run several small COMMIT operations concurrently, you might see a decrease in performance. For improved performance, group multiple changes into a single COMMIT operation.
  • If you purchased Reserved Instances, then your billing depends on the resized cluster configuration, reserved node types, and reserved node count. For more information, see How reserved nodes work.

Related information

Resizing clusters in Amazon Redshift

Troubleshooting connection issues in Amazon Redshift

Building high-quality benchmark tests for Amazon Redshift using SQLWorkbench and psql

AWS OFFICIAL
AWS OFFICIALAktualisiert vor 8 Monaten
Keine Kommentare