Advice for Upgrading ElastiCache Redis Node Types

Reducing Upgrade Time

The upgrade time for node type modifications depends on several factors including the amount of data stored and the replication process. To reduce the time:

• Schedule the upgrade during low-traffic periods to minimize impact
• Ensure you're using the ApplyImmediately=true parameter when initiating the upgrade through the ModifyReplicationGroup API if you want the changes to take effect immediately rather than waiting for the next maintenance window
• Consider that vertical scaling creates new nodes and synchronizes data between old and new nodes, so having less data will result in faster upgrades

Impact of Connection Count

The higher number of connections (2000 vs 200) will likely impact the upgrade process. When upgrading the engine, ElastiCache terminates existing client connections, so more connections means more reconnection attempts. This could potentially extend the perceived downtime from the client perspective.

Ensuring Data Safety

The ElastiCache engine upgrade process is designed to make a best effort to retain your existing data and requires successful Redis replication. The vertical scaling process creates new nodes with the requested node type and synchronizes data from the old nodes to the new ones before switching over.

Minimizing Connection Disruption

To minimize connection disruption:

• Implement best practices for Redis clients with error retries and exponential backoff
• Follow best practices for minimizing downtime during maintenance
• Client applications should be designed to handle connection failures gracefully with automatic reconnection logic

The window when connections will be disrupted is typically during the DNS propagation phase. For Redis versions 5.0.6 and higher, this is designed to have minimal downtime. However, you should still expect a brief period (typically seconds) where connections will be terminated and need to reconnect.

General Advice

• Test the upgrade process thoroughly in a staging environment that closely mimics your production setup
• Monitor the cluster during the upgrade using CloudWatch metrics
• Ensure you have sufficient ENI (Elastic Network Interface) capacity in your VPC for the new node type
• If using reserved-memory parameter, make sure to update your parameter group to account for the new node type's memory
• Consider implementing a read-replica strategy to maintain read availability during the upgrade
• Schedule the upgrade during a time when you can monitor the process and respond to any issues
  Sources
  Upgrade considerations when working with node-based clusters - Amazon ElastiCache
  Online vertical scaling by modifying node type - Amazon ElastiCache
  Amazon ElastiCache for Redis Product Features - AWS

Hello.

As described in the following documentation, we recommend changing the node type during periods of low traffic.
In the past, we have tested changing node types and have observed temporary increases in latency during the change.
If your application can handle some downtime, it is recommended that you do so during this time.
https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/redis-cluster-vertical-scaling.html

While the vertical scaling process is designed to remain fully online, it does rely on synchronizing data between the old node and the new node. We recommend that you initiate scale up/down during hours when you expect data traffic to be at its minimum.

I think if you change the node type during times of high traffic, it takes time to synchronize data within the cluster.
https://repost.aws/knowledge-center/elasticache-redis-minimize-time-scaling

To minimize downtime during synchronization, avoid scaling when you have a high workload. If the cluster has a high workload and scaling is taking a long time, then reduce the incoming requests to Redis to prevent synchronization failure. If synchronization occurs, then check the SaveInProgress metric in Amazon CloudWatch to determine when the synchronization occurred. Note that the SaveInProgress metric collects data every minute and might not capture synchronization that finished under one minute. For more information, see Monitoring best practices with Amazon ElastiCache for Redis using Amazon CloudWatch.