- Newest
- Most votes
- Most comments
Hello,
Thank you for using AWS ElastiCache service.
Kindly note that this error is seen when the client application makes request to set a key or write some data by connecting to the replica node's endpoint.
Also, it is worth to consider here that during Engine Upgrade, the Redis engine version is updated on each node in the cluster one at a time. The primary node is upgraded first, followed by the replica nodes. During this process, Elasticache triggers a failover to switch to new primary and updates the DNS record to point to new primary. However, if the client has cached the old nodes IP and connects to the old primary, you will face the error "READONLY You can't write against a read only replica" as the old primary is no longer accepting writes.
Usual tendency of client application is to resolve the primary endpoint once and cache the primary IP address locally. During the failover the IP address on primary endpoint changes to new primary node. If your client does not detect this change, it will still try to connect to old IP and this will result in prolonged downtime.
To ensure less interruption, you should have proper retry configuration in place at client side to detect failover and pick new IPs. Also, you can consider to reduce the DNS caching TTL values (if used).
Relevant content
- asked 2 years ago
- Accepted Answerasked a year ago
- AWS OFFICIALUpdated 2 years ago