ElastiCache for Redis single node from Reader endpoint

During my initial tests I found a potential issue with scaling out ElastiCache Reader endpoint for reads. In my setup I used a set of 1 primary with 3 replicas in 2 availability zones (that’s cluster disabled mode).

Our code uses the primary endpoint for writes and reader endpoint for reads. With that in mind I expected the reader endpoint to distribute reads across all 3 reader nodes in round-robin fashion. The documentation confirms it:

Finding a Redis (Cluster Mode Disabled) Cluster's Endpoints (Console)

...
A reader endpoint will evenly split incoming connections to the endpoint between all read replicas in a ElastiCache for Redis cluster.
...
A reader endpoint is not a load balancer. It is a DNS record that will resolve to an IP address of one of the replica nodes in a round robin fashion.

Using a jump box host I noticed this is not the case (always getting the same DNS reply):


ubuntu@ip-10-5-33-208:~$ host react-region-redis-rep-group-dev-ro.dbvymf.ng.0001.use1.cache.amazonaws.com ns-699.awsdns-23.net
Using domain server:
Name: ns-699.awsdns-23.net
Address: 205.251.194.187#53
Aliases:

react-region-redis-rep-group-dev-ro.dbvymf.ng.0001.use1.cache.amazonaws.com is an alias for react-region-redis-rep-group-dev-002.dbvymf.0001.use1.cache.amazonaws.com.
react-region-redis-rep-group-dev-002.dbvymf.0001.use1.cache.amazonaws.com has address 10.5.27.242

I was expecting either :


• multiple “has address” ip addresses (3 to be exact) returned in random order for each query
• one “has address” ip, which is different for every DNS request.

What I see instead is the same IP address for every DNS queries, which is not inline with the documentation and it does not allow distributing reads using the reader endpoint. My tests are also confirmed by the metrics for ElastiCache service, where, despite using the Reader endpoints my application running in AWS ECS is only sending “Get Type commands” (like get, mget, hget etc..) to one of the replica:

Note all the reads are sent to the same replica node (react-region-redis-rep-group-dev-002) as returned by the dns query.


Thanks in advance for any pointers, Piotr