Are Route53 health checkers considered in combination with Route53 healthchecks based on CloudWatch alarms?

Question

I have a customer setup with regional DNS failover configured, using Route53 with 2 alias records that use a weighted policy.
The alias records point to network load balancers (NLB) in 2 different regions respectively.
There is a calculated health check specified for each record.  The calculation is based on individual health checks that check the status of Cloudwatch alarms which monitor HealthyHostCount >=1 for every NLB.

In CloudWatch, I see no alarms whatsoever and yet, the DNS failover is triggered.  In the Route53 healthcheck console, I see that my individual health checks fail at exactly the same time when the AWS regional health checkers indicate a failure (overall availability below 82%).  In CloudWatch there are no alarms or status changes to see for my HealthyHostCount metrics.  The individual registered targets in the target groups give no signs of excessive load.

Are the AWS regional health checkers somehow "AND"-ed with my own calculated health checks?  If so, can I disable them?
Since my Route53 healtchecks are CloudWatch based and not endpoint based, I was assuming that the AWS regional health checkers are running but not relevant nor impacting my use-case.

Ravi Bojja · Answer

Hello Kurt,

If you want Route 53 to make a Failover decision based on your attached custom health checks, you would need to disable to evaluate target health while creating Alias A record pointing to NLB's.

If you enable the evaluate target health while creating Alias A record to NLB and has custom health check attached,  now Route 53 Failover decision is based on the internal health of NLB and your custom health check health, both have to healthy to influence Route53  decision to route you requests to primary records.

Chethan SE · Answer

Hello Kurt Vanslambrouck,

This usually happens, when you have AND with the CloudWatch_Alarm based Health check and Regional Health Check. In order to verify the configuration, Navigate to the Calculated health checks ID and look for "Health checks to monitor" section to see list of Health-Checks added also, check the condition "Report healthy when".

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/health-checks-creating-values.html#health-checks-creating-values-calculated

Further, check what happens when you associate a health check with an alias record?

*When you set Evaluate target health to Yes for an alias record, Route 53 evaluates the health of the resource that the alias target value specifies. For an Application Load Balancer, Route 53 considers the health checks that are associated with the target groups behind the load balancer.*

[+] https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-complex-configs.html

[+] https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-complex-configs.html#dns-failover-complex-configs-hc-alias

Are Route53 health checkers considered in combination with Route53 healthchecks based on CloudWatch alarms?

Relevant content