How can I troubleshoot with the cpu throttle during the initial snapshot creation?

Question

Hi,
I’m experiencing high CPU usage after creating a Redshift cluster of the RA3 type, which has 2 nodes. 
The cluster is about a terabyte in size and was created from a snapshot. 
Now, the CPU usage has soared to 100%, and the snapshot creation process is stuck. 
The console indicates that there are about 12,000 seconds (3 and a half hours) left to complete.

Is there any way to speed up the snapshot creation process? Alternatively, it would be helpful to make it possible to query the cluster.

Cheers,

Hyunjoong_S · Accepted Answer

Hello,

As for the amount of time it takes to create a snapshot, various factors such as load on the Redshift cluster that you want to create the snapshot and encryption of the snapshot affect it.
In general, for creating Redshift snapshots, there are general guidelines below.

- If cluster encryption is going on, the actual snapshot creation won't start until encryption ends
- First snapshot takes longer time, but the later ones will be much faster. Coz first snapshot if a full backup, the later ones are incremental
- It is also possible that cluster resource was already highly used

You can monitor the creation of snapshots using the CLI commands below.

```
>> aws redshift describe-cluster-snapshots --cluster-identifier name --snapshot-type type --region region

[+] describe-cluster-snapshots
https://docs.aws.amazon.com/cli/latest/reference/redshift/describe-cluster-snapshots.html
```
In addition, we confirmed that your cluster's CPU usage is 100%.
If the CPU usage rate is high, it is recommended to reduce the load after checking the following contents.

- Review your Redshift cluster workload.
- Maintain your data hygiene.
- Update your table design.
- Check for maintenance updates.
- Check for spikes in your leader node CPU usage.
- Use Amazon CloudWatch to monitor spikes in CPU utilization.

[+] How do I troubleshoot high CPU usage in Amazon Redshift?
https://repost.aws/knowledge-center/redshift-high-cpu-usage

Therefore, it can take a long time to create a snapshot due to various factors, so it is necessary to create a case on the Support Team to know the clear reason.

Thanks

Thanniru Anil Kumar · Answer

**Monitor System Resources:** Use Amazon CloudWatch or any monitoring tool available to monitor system resources such as CPU, memory, disk I/O, and network activity. This can help you identify if the CPU usage is the bottleneck or if there are other resource constraints affecting the snapshot creation process.

**Check Redshift Performance Metrics:** Use Amazon CloudWatch to monitor Redshift-specific performance metrics such as cluster CPU utilization, disk space usage, and query throughput. This can provide insights into the overall health and performance of your Redshift cluster.

**Optimize Cluster Configuration: **Review your Redshift cluster configuration and consider optimizing parameters such as node type, number of nodes, and distribution style. Depending on your workload and data distribution, adjusting these parameters can help improve overall cluster performance and reduce resource contention during snapshot creation.

**Increase Node Size or Count: **If your cluster is under-provisioned in terms of compute resources, consider increasing the node size or adding more nodes to the cluster. This can provide additional CPU and memory resources to speed up the snapshot creation process.

**Check for Query Activity:** Ensure that there are no active queries running on the cluster that could be consuming CPU resources and affecting snapshot creation. You can use the Redshift console or query system views such as STV_INFLIGHT to monitor active queries on the cluster.

**Review Snapshot Settings**: Check the settings for the snapshot creation process, such as the snapshot retention period and whether the snapshot is being encrypted. Adjusting these settings may help speed up the snapshot creation process.

**Contact AWS Support: **If you continue to experience issues with high CPU usage and slow snapshot creation, consider reaching out to AWS Support for further assistance. They can provide expert guidance and help troubleshoot any underlying issues with your Redshift cluster.

How can I troubleshoot with the cpu throttle during the initial snapshot creation?

Relevant content