DDoS Resilience - Using AWS WAF to protect against DDoS Botnets

While this AWS WAF article applies to all AWS layer 7 services that are able to have an AWS WAF WebACL associated, for CloudFront distributions, please read this article in conjunction with fellow article DDoS Resilience - the unbelievable importance of HTTP caching, as preventing malicious requests from reaching your Origin is best done using a 'defence-in-depth' approach of using AWS WAF (to 'Block' clearly malicious requests) and HTTP caching, which serves requests on behalf of the origin and acts as a second layer of defence.

Rate-based rules

AWS strongly recommends protecting against HTTP request floods by using rate-based rules in AWS WAF to automatically 'Block' or 'Challenge' requests matching a specific aggregation key (which is by default 'IP address'), when the number of requests received in a configurable sliding window exceeds a threshold that you define. Offending client IP addresses will receive an HTTP 403 forbidden response (or configured block error response code/body) and remain blocked until request rates drop below the threshold.

It’s recommended to layer rate-based rules to provide enhanced protection so that you have:

• a blanket 'catch-all' rate-based rule to protect your application from large HTTP floods.
• a rule with a custom aggregation key e.g. 'Header (Host) + URI + query string + IP', at more restrictive rates than the blanket rate-based rule - this is particularly effective against DDoS as the 'modus operandi' of DDoS botnets is to hit the same URL over and over

rule name: rbr_host_uri_ip_ddos
rule type: rate-based
scope-down:  none
limit: <a low number>
evaluation window: 300
aggregation key (custom): Header(Host) + URI + querystring + IP
action: BLOCK

• one or more rate-based rules to protect specific URIs, especially ones that cannot be cached, at more restrictive rates than the blanket rate-based rule

rule name: rbr_api
rule type: rate-based
scope-down:  if uri_path starts_with '/api'
limit: <a low number>
evaluation window: 300
aggregation key (default):  IP
action: BLOCK

For instance you may choose a blanket rate-based rule (no scope-down statement) with a limit of 500 requests within a 5-minute period, and then create one or more of the following rate-based rules with lower limits than 500 (as low as 10 requests in a 10-minute period) using scope-down statements:

• Protect your web pages with a scope-down statement like "if NOT uri_path contains '.'" so that requests for resources without a file extension are further protected. This also protects your homepage (/) which is a frequently targeted URI path.
• Protect dynamic endpoints with a scope-down statement like "if method exactly matches 'post' (convert lowercase)"
• Protect 'heavy' requests that reach your database or invoke a one-time password (OTP) with a scope-down like "if uri_path starts_with '/login' OR uri_path starts_with '/signup' OR uri_path starts_with '/forgotpassword'"

Together with IP Reputation rulegroups (discussed below), rate-based rules in 'Block' mode are the cornerstone of a defense-in-depth WAF configuration to protect against request floods, and are a requirement for AWS Shield Advanced cost protection requests to be approved.

IP Reputation rulegroups

To prevent attacks based on IP address reputation, you can create rules that match IPs making HTTP requests to custom IPSets that you maintain, or use AWS WAF Managed IP Reputation rulegroups. From a DDoS perspective, the most important 2 rules are:

1. ‘AWSManagedIPDDoSList’ rule within the 'AWSManagedRulesAmazonIpReputationList' rulegroup
2. ‘AnonymousIPList’ rule with the 'AWSManagedRulesAmazonIpReputationList' rulegroup

AWSManagedIPDDoSList rule within the 'AWSManagedRulesAmazonIpReputationList' rulegroup

Amazon's IP reputation list rule-group 'AWSManagedRulesAmazonIpReputationList' includes rules based on various Amazon internal threat intelligence teams. These rules look for IP addresses that are bots, performing reconnaissance against AWS resources, or actively engaging in DDoS activities against AWS resources. The 'AWSManagedIPDDoSList' rule is curated based on IPs recently seen particpating, , which is in 'Count' mode by default, has been observed matching over 90% of requests during layer 7 attacks. Surges in traffic are a strong indicator of a DDoS attempt and AWS recommends adding the 'AWSManagedRulesAmazonIpReputationList' rulegroup to your WebACL and take one of the following actions with regards to the 'AWSManagedIPDDoSList' rule:

1. Move it into 'Block' or 'Challenge' mode OR
2. Leave it in 'Count' mode and create a subsequent rate-based rule using a custom aggregation key using the WAF label namespace 'awswaf:managed:aws:amazon-ip-list:', applied to requests matching this rulegroup

The best option for you from the 2 options above will depend on what constitutes 'normal' for your online property, and your ability to absorb small floods of malicious traffic. We tend to see that websites either constantly receive a steady trickle of traffic from 'AWSManagedIPDDoSList' rule, or never see any traffic from it unless they are under attack. In order to determine your 'normal', add the 'AWSManagedRulesAmazonIpReputationList' rulegroup into your WebACL and use CloudWatch AWS/WAFv2 'CountedRequests' metrics for the 'AWSManagedRulesAmazonIpReputationList' rulegroup, to identify if your web application receives traffic from this rulegroup during non-attack periods ('peacetime'). You may want to use WAF logs to pivot to the specific IPs that matched rules within the rulegroup to see if their behaviour in terms of the requests that they made, was in keeping with what 'normal' user behavior looks like. Then you can make an informed decision to 'Block' or 'Challenge' actions, or create a rate-based rule.

If you choose to use a rate-based rule rather than block this traffic outright (the majority of customers choose to block this outright) it would look like the following (using configuration 'pseudo-code'):

rule name: rbr_ipddoslist
rule type: rate-based
scope-down: if label matches 'awswaf:managed:aws:amazon-ip-list:AWSManagedIPDDoSList'
limit: <a low number>
evaluation window: 300
aggregation key (custom): label namespace 'awswaf:managed:aws:amazon-ip-list:'
action: BLOCK

‘AnonymousIPList’ rule within the 'AWSManagedRulesAnonymousIpList' rulegroup

The 'AWSManagedRulesAnonymousIpList' rule group contains rules to block requests from services that allow the obfuscation of viewer identity. These include requests from VPNs, proxies, Tor nodes, and cloud platforms (excluding AWS).

Requests from clients matching the ‘AnonymousIPList’ rule are potential candidates for being used for malicious purposes like DDoS, and you should consider moving this rule to ‘Block’, but be careful of moving the ‘HostingProviderIPList’ rule to ‘Block’ - it’s a very blunt instrument, and customers frequently choose to move this rule to ‘Count’ mode within the rulegroup.

Challenge rule action

DDoS botnets are built to perform maximum damage to targets, at the least cost to the botnet operator - they are generally unsophisticated script-based bots who cannot process javascript. The 'Challenge' rule action asks the client to perform a javascript 'proof of work' and to present evidence of this in the form of a token (cookie) 'aws-waf-token', before the request is permitted. It has a high efficacy rate with DDoS botnets at preventing malicious requests, although it does have a slightly higher cost per request than 'standard' AWS WAF rule actions, so you may want to combine it with rate-based rules or only create a rule with 'Challenge' as action, during an active attack event where availability is being impacted.

The token can only be acquired via HTTP 'GET' requests with the 'Accept' header containing 'text/html' i.e. 'page' requests (unless there has been prior clientside integration). Once the token is acquired it will be presented by the client on subsequent requests for the same domain until the TTL in the form of cookie expiration, expires. Do not configure a 'Challenge' response for a non-page request, unless you have already acquired the 'aws-waf-token'.

Block access to unused URI paths

Block access to unused URI paths using AWS WAF - the most obvious example here would be the '/' URI path for an API endpoint, which would typically return an HTTP 404 (Not Found).

Likewise consider blocking access to file extensions that you do not use e.g.

if NOT uri_path matches regex .*\.(css|js|png|jpg|svg|woff2|html|json)$ AND uri_path contains ‘.’

Blocking access to such paths prevents sudden high-volume request flood on a non-existent URL or URLs, from overwhelming your target group (for ALB endpoints) or origin (for CloudFront).