How do I analyze AWS WAF logs in CloudWatch?

Resolution

To analyze and filter specific log requests in CloudWatch, use CloudWatch Logs Insights or the CloudWatch query generator.

Use CloudWatch Logs Insights to analyze AWS WAF access logs

You can use CloudWatch Logs Insights from the CloudWatch console or from the CloudWatch Logs Insights tab in the AWS WAF console.

From AWS WAF

Complete the following steps:

1. Open the AWS WAF console.
2. In the navigation pane, under AWS WAF, choose Web ACLs.
3. For Region, select the AWS Region that contains your web access control list (web ACL).
   Note: If your web ACL is set up for Amazon CloudFront, then select Global.
4. Select your web ACL.
5. From the tabs at the top of the page, navigate to CloudWatch Logs Insights.
6. In the Query editor, enter your query. Use query syntax to design your queries. You can also select queries from the Most frequently used queries list.
7. Choose Run query.
8. To view your results, in the navigation pane, choose Logs.

From CloudWatch

Complete the following steps:

1. Open the CloudWatch console.
2. In the navigation pane, under Logs, choose Logs Insights.
3. For Select log group(s), select one or more log groups to query from the list. Or, choose Browse log groups, and then select your query.
4. (Optional) Choose a time range for the period that you want to query.
5. Use query syntax to design your queries.
6. To view your results, choose Run query.

To filter for specific information, use the following example queries:

Top client IP addresses

To count the top client IP addresses that make requests to your protected resource, run the following query:

fields httpRequest.clientIp
| stats count(*) as requestCount by httpRequest.clientIp
| sort requestCount desc

Top countries

To count the top countries from where requests are sent to your protected resource, run the following query:

stats count(*) as RequestCount by httpRequest.country as Country
| sort RequestCount desc

Top hosts

To count the top hosts headers that make requests to your protected resource, run the following query:

parse @message /\"name\":\"[Hh]ost\",\"value\":\"(?Host[^\"]*)\"/
| stats count(*) as RequestCount by Host
| sort RequestCount desc

Top methods

To count the top HTTP methods used to make requests to your protected resource, run the following query:

stats count(*)as RequestCount by httpRequest.httpMethod as Method
| sort RequestCount desc

Top user agents

To count the top user agents that make requests to your protected resource, run the following query:

parse @message /\"name\":\"[Uu]ser\-[Aa]gent\",\"value\":\"(?UserAgent[^\"]*)\"/
| stats count(*) as RequestCount by UserAgent
| sort RequestCount desc

Top URI paths

To list the top most visited URI paths on your protected resources, run the following query:

fields httpRequest.uri 
| stats count(*) as uriVisits by httpRequest.uri 
| sort uriVisits desc 
| limit 50

Top terminating rules

To count the top terminating rules in your logs, run the following query:

stats count(*) as RequestCount by terminatingRuleId
| sort RequestCount desc

Filter blocked requests

To filter for all blocked requests, and their terminating rule, URI path, and client IP address, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule
| filter action = "BLOCK"
| sort @timestamp desc

Filter by host

To filter your logs by a specific host header value, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI
| parse @message /\{"name":"[Hh]ost\",\"value":\"(?Host[^\"]*)\"/
| filter Host = "www.example.com"

Note: Replace www.example.com with the name of your host.

Filter for a specific user agent

To filter your logs by a specific user agent, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method,httpRequest.uri as URI
| parse @message /\"name\":\"[Hh]ost\",\"value\":\"(?Host[^\"]*)\"/
| parse @message /\"name\":\"[Uu]ser\-[Aa]gent\",\"value\":\"(?UserAgent[^\"]*)\"/
| filter @message like "Postman"
| display Rule, action, Country, ClientIP, Method, URI, Host, UserAgent
| sort action, URI desc

Note: Replace Postman with your user agent.

Filter on POST requests

To filter for POST requests, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.httpMethod as Method, httpRequest.uri as URI
| parse @message /\"name\":\"[Hh]ost\",\"value\":\"(?Host[^\"]*)\"/
| parse @message /\"name\":\"[Uu]ser\-[Aa]gent\",\"value\":\"(?UserAgent[^\"]*)\"/
| filter httpRequest.httpMethod ="POST"
| display Rule, action, Country, ClientIP, Method, URI, Host, UserAgent
| sort @timestamp desc

Filter by country

To filter out requests that don't originate from a specific country, run the following query:

fields terminatingRuleId as Rule, action, httpRequest.country as Country, httpRequest.clientIp as ClientIP, httpRequest.uri as URI
| parse @message /\"name\":\"[Hh]ost\",\"value\":\"(?Host[^\"]*)\"/
| parse @message /\"name\":\"[Uu]ser\-[Aa]gent\",\"value\":\"(?UserAgent[^\"]*)\"/
| filter Country != "US"
| sort Country, action desc

Note: Replace US with the country code that you want to filter out.

Filter requests blocked by rate-based rules

To filter for requests blocked by a rate-based rule, run the following query:

fields @timestamp, httpRequest.clientIp as ClientIP, httpRequest.uri as URI, terminatingRuleId as rule, httpRequest.country as Country
| filter action = "BLOCK"
| filter terminatingRuleType = "RATE_BASED"
| sort @timestamp desc

Cross-Site Scripting (XSS) or SQL Injection

To find patterns that cause XSS or SQL Injection in the terminating rule for a custom rule or AWS Managed Rules rule group, run the following query. The query returns entries with a timestamp, client IP address, country of origin, match details, and the request ID:

fields @timestamp
| parse @message ',"terminatingRuleMatchDetails":[*],' as terminatingRuleMatchData
| filter (terminatingRuleMatchData like /XSS/ or terminatingRuleMatchData like /SQL/)
| display @timestamp, httpRequest.clientIp, httpRequest.country, terminatingRuleMatchData, httpRequest.requestId

Filter requests counted by a specific rule in a rule group

To filter for requests that are counted by a specific rule in a rule group, then terminated by the default action, run the following query:

fields @timestamp
| filter (@message like 'excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"NoUserAgent_HEADER"}]}' and @message like 'terminatingRuleId":"Default_Action"')
| parse @message '"ruleId":*}]}' as ruleMatchDetails
| display @timestamp, httpRequest.clientIp, httpRequest.country, ruleMatchDetails, httpRequest.requestId

Note: Replace ruleId with your rule ID.

Filter requests with a CAPTCHA that's not valid

To filter for the top 100 requests with a CAPTCHA that's not valid, run the following query. This query returns the time that the request was made, the IP address, the request ID, the response code, and the entire message:

fields @timestamp, httpRequest.clientIp, httpRequest.requestId, captchaResponse.failureReason, @message
| filter captchaResponse.failureReason ='TOKEN_MISSING'
| sort @timestamp desc
| limit 100

Note: Replace 100 with the number of requests that you want to filter for.

Use CloudWatch query generator to analyze AWS WAF access logs

To use generative AI to analyze your access logs, run the query generator in CloudWatch.