Route 53 health check for API gateway token-based lambda authorizer

Question

I recently encountered a situation where lambda was down in my region and my API gateway endpoints were returning 500s because the [authorizer lambda](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html) wasn't working. I want to create a Route53 health check on the authorizer lambda so I can trigger a failover to my DR region if this happens again.

But I'm running into some issues while attempting to set this up:

1. I tried creating a simple `/health` resource on the API gateway + lambda which returns a 200 response after the lambda recognizes that health path. I can then integrate this with the R53 health check. However, the token-based lambda authorizer requires an `Authorization` header which cannot be configured for the R53 health check itself (not even a dummy value). I cannot convert the lambda from token-based to request-based as there are some requirements that prevent me from doing so. So this incompatibility seems to rule out this option.
2. I created a Cloudwatch alarm on the Lambda errors that can be used for the Route 53 health check. I decided to use an error rate (errors/invocations) as it's best suited for my use case. However, [Route 53 doesn't support alarms that use metric math to query multiple Cloudwatch alarms](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/health-checks-creating-values.html). For lambdas, [AWS recommends using Sum](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Best_Practice_Recommended_Alarms_AWS_Services.html#Lambda) as the statistic for the `Error` metric. But I don't like the idea of basing an alarm on a fixed number of failures, since I wouldn't want the expected traffic patterns and occasional error spikes from the lambda to trigger the alert. I only really want the alert + R53 failover to trigger if Lambda is DOWN down. Whether that's a continuous set of lambda errors or no invocations at all over a specified period (assuming lambda is suddenly unreachable).

Any tips/suggestions on how to set up this Route 53 health check given these challenges?

tom · Accepted Answer

Update: I ended up creating a second authorizer that uses the same lambda function, and I created a health resource on the API gateway for invoking it. The Route 53 health checks support query parameters, so by setting the second authorizer to request type you can configure Authorization as a query string and specify the same in the identitySource parameter. I just provided a dummy value for the authorization in the health check url. I was also able to keep the existing authorizer token-based. This totally worked out!

re:Post Agent · Answer

Given the challenges you've described, here are some suggestions for setting up a Route 53 health check for your API Gateway token-based Lambda authorizer:

1. Custom Lambda Health Check:
Instead of using the API Gateway directly, you could create a separate Lambda function specifically for health checks. This function would internally invoke your authorizer Lambda with the necessary token. You can then set up a Route 53 health check against an API Gateway endpoint that triggers this health check Lambda. This approach bypasses the need for the Authorization header in the Route 53 health check itself.

2. CloudWatch Metric Combination:
While Route 53 doesn't support metric math directly, you can create a custom metric that combines multiple Lambda metrics. For example, you could create a metric that represents the ratio of errors to invocations. Then, set up a CloudWatch alarm based on this custom metric. Route 53 can then use this alarm for health checking.

3. Multi-metric Alarm:
Create a CloudWatch composite alarm that considers both the error count and the invocation count. This alarm can trigger if there are too many errors OR if there are no invocations for a specified period. Route 53 can then use this composite alarm for health checking.

4. Synthetic Canary:
Use AWS CloudWatch Synthetics to create a canary that periodically invokes your API with the proper authorization token. Set up a CloudWatch alarm based on the canary's success rate, and use this alarm for your Route 53 health check.

5. External Monitoring:
Set up an external monitoring service that can make authenticated requests to your API. This service can then update a CloudWatch custom metric, which you can use for alarming and Route 53 health checks.

Each of these approaches has its own trade-offs in terms of complexity, cost, and accuracy. The best solution will depend on your specific requirements and constraints. Remember to thoroughly test your chosen solution to ensure it accurately reflects the health of your Lambda authorizer and triggers failover appropriately.

**Sources**  
[Input to an API Gateway Lambda authorizer - Amazon API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-lambda-authorizer-input.html)  
[How Amazon Route 53 determines whether a health check is healthy - Amazon Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-determining-health-of-endpoints.html)  
[HealthCheckConfig - Amazon Route 53](https://docs.aws.amazon.com/Route53/latest/APIReference/API_HealthCheckConfig.html)

Route 53 health check for API gateway token-based lambda authorizer

Relevant content