Cannot remove encryption from Redshift cluster

Question

When trying to restore an unencrypted snapshot via the AWS console, it seems to force the new cluster to enable encryption with no way of disabling it afterward.

Then, when we try to follow the instructions here to remove encryption via the CLI, we consistently get the following error even though the cluster seems to be fully available.

```
aws redshift modify-cluster --cluster-identifier  --no-encrypted
```

```
An error occurred (InvalidClusterState) when calling the ModifyCluster operation: Cluster cannot be resized when streaming restore is in progress
```

`"RestoreStatus": {
                "Status": "completed",
                "CurrentRestoreRateInMegaBytesPerSecond": 0.0,
                "SnapshotSizeInMegaBytes": 0,
                "ProgressInMegaBytes": 0,
                "ElapsedTimeInSeconds": 0,
                "EstimatedTimeToCompletionInSeconds": 0
            },`

https://docs.aws.amazon.com/redshift/latest/mgmt/changing-cluster-encryption.html

Is this a known issue with unencrypted clusters? Any help to work around this would be appreciated.

Riku_Kobayashi · Answer

Hello.

I tried it with my AWS account, and like you, when I restore the snapshot to the cluster in the management console, it is encrypted.  
How about restoring using the AWS CLI instead of the management console?  
You should be able to change the encryption settings here.  
You can restore from an unencrypted snapshot to an unencrypted cluster by adding the option "--no-encrypted" as shown below.  
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/redshift/restore-from-cluster-snapshot.html  
```
aws redshift restore-from-cluster-snapshot --cluster-identifier test1 --snapshot-identifier test1 --no-encrypted
```

By the way, as stated in re:Post Agent's answer, it is not possible to change an already encrypted cluster to an unencrypted cluster.

vitaly_il2 · Answer

"When you restore a snapshot, the new cluster inherits the encryption status of the snapshot. If the original snapshot was unencrypted, but you chose to encrypt the new cluster during restoration, this encryption cannot be removed." - IMHO it should work this way. And, according to my colleagues, it worked this way until now.
But today [AWS console] we cannot restore from unencrypted snapshot - there is *no way* to disable cluster encryption for the new cluster.

re:Post Agent · Answer

The issue you're experiencing is not a known problem with unencrypted clusters, but rather a result of how Amazon Redshift handles encryption and restoration processes. There are a few important points to consider:

1. Encryption is an immutable property of an Amazon Redshift cluster. Once a cluster is encrypted, it cannot be directly changed to an unencrypted state.

2. When you restore a snapshot, the new cluster inherits the encryption status of the snapshot. If the original snapshot was unencrypted, but you chose to encrypt the new cluster during restoration, this encryption cannot be removed.

3. The error message you're receiving suggests that the cluster is still in a state of streaming restore, even though the RestoreStatus shows as "completed". This could indicate a temporary inconsistency in the cluster's state.

To address this situation, you have a few options:

1. Wait for some time (possibly a few hours) and try the command again. Sometimes, internal processes need to complete before the cluster is fully ready for modifications.

2. If waiting doesn't resolve the issue, you may need to create a new unencrypted cluster from your original unencrypted snapshot. When restoring, make sure not to enable encryption during the process.

3. If you absolutely need an unencrypted cluster and can't restore it directly, you might need to:
   a. Create a new encrypted cluster from your snapshot
   b. Unload the data from this encrypted cluster to S3 (which will be unencrypted)
   c. Create a new unencrypted cluster
   d. Load the unencrypted data from S3 into the new cluster

Remember, moving from an encrypted state to an unencrypted state is not a straightforward process in Amazon Redshift due to security considerations. It's generally recommended to keep your data encrypted for better security. If you have specific reasons for needing an unencrypted cluster, you might want to consult with AWS support for the best approach in your particular case.  
**Sources**  
[Restoring a cluster from a snapshot - Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-snapshot-restore-cluster-from-snapshot.html)  
[Changing cluster encryption - Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/changing-cluster-encryption.html)  
[Amazon Redshift provisioned clusters overview - Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/overview.html)

Cannot remove encryption from Redshift cluster

Add your answer

Relevant content