CloudFront Weighted R53 Registers don't work as expected

Question

Hi,

I've been trying to set up an r53 weighted record to flip between two CloudFront distributions. And it seems that the CloudFront internally somehow decides to pick the distribution that better fits the alias.

The setup I've done is: 
CloudFront 1: alias: *.mycompanydomain.com 
CloudFront 2: alias stage.mycompanydomain.com

In addition,

**R53 Weighted 1:**
type A;
record name "stage.mycompanydomain.com";
pointing to the first cf;
weight 100

**R53 Weighted 2:** 
type A;
record name "stage.mycompanydomain.com";
pointing to the second cf;
weight 100

Independently on how many requests I do. It always hit the cf 2. So I think is because of the record name for the cf 2 the alias name fits its value perfectly, but I am not 100% sure.

Someone can explain to me what the rule is here and also why this setup doesn't work. If possible, which is the way that I can split up traffic between two CloudFront? Thanks!

Note: The domain is not actual; I just made up one for the example.

Answer

Hey there,

I see that you are trying to equally distribute traffic between your two alias's using weighted records.

To assist with further investigation of this issue, please confirm the following:
1.	How was this record configured before the weighted routing? Was it initially pointing the distribution that the clients are connecting to?
2.	Are There any Health Checks on the record ? As it would affect the split of traffic.

A possible cause of this issue may be caching on the client side. Meaning that if the record was using a single distribution and now a second distribution was added with weight, the DNS cache on the clients side could possibly still be resolving to the original distribution that was initially in place.

With that being said I suggest that you to try resolving from a new client and see which IP you get, or try to Flush the DNS cache and then confirm if the issue persists.

As per AWS documentation, it mentions that if you set 0 as the weight for all weighted records then it will route traffic to all. Furthermore, having R53 Health Checks would make that case different [1]. Therefore, I would also suggest setting all weighted to 0 for even distribution and confirm if the issue persists

References:
[1] https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-values-weighted-alias.html

Answer

Route 53 weighted routing does distribute traffic equally when weights are same. However, it doesn't switch traffic between 2 values in a round robin fashion. You can see equal distribution when sample set is large(>1000 queries) and there is no caching at the intermediate resolver.

To test it , send 1000 queries to the domain's authoritative nameservers using the below commands. Hitting authoritative nameservers directly will eliminate the scope of caching at the intermediate resolver.

#!/bin/bash
for i in {1..10000}
do
domain=$(dig   @AuthoritativeNameserver_IP +short)
echo -e  "$domain" >> AuthoritativeNameServer_results.txt
done

Reference: https://repost.aws/knowledge-center/route-53-fix-dns-weighted-routing-issue

CloudFront Weighted R53 Registers don't work as expected

Relevant content