Cloudhsm: HSMs are being rotated

Based on the information provided, it appears that the behavior you're observing with the CloudHSM instances being destroyed and recreated periodically is likely related to AWS's preventative maintenance processes for the CloudHSM service.

According to the AWS Best Practices documentation, HSMs may be replaced for reasons such as applying software or kernel patches as part of preventative maintenance activities. This process is designed to ensure the ongoing security, stability, and reliability of the CloudHSM service.

The documentation further states that when you have a cluster with multiple HSMs, the client will automatically route HSM requests away from the HSM that is being replaced during this maintenance process. This is intended to minimize any potential impact on customer applications that rely on the CloudHSM service.

The temporary increase in the HSM count to 3 during this process is likely due to AWS creating a new HSM instance, migrating data from the existing instance to the new one, and then decommissioning the old instance. This approach allows for a seamless transition without disrupting the availability of the CloudHSM cluster.

While this behavior may not be explicitly documented in detail, it appears to be an expected part of AWS's maintenance and operational processes for the CloudHSM service. As long as you have a cluster with multiple HSMs, the maintenance activities should not affect the availability of your applications that rely on the CloudHSM service.

However, if you have any concerns or notice any unusual behavior beyond what is described in the Best Practices documentation, it's always a good idea to reach out to AWS Support for further assistance and clarification.