How can I use CloudWatch Logs Insights queries with my VPC flow log?

Short description

When you turn on VPC flow logs that target CloudWatch Logs, you see one log stream for each elastic network interface. If an issue occurs, then use CloudWatch Logs Insights to identify potential causes and validate deployed fixes.

For more information about the log types that CloudWatch Logs Insights supports, see Supported logs and discovered fields.

Resolution

Run a query

Use the CloudWatch console to either run a sample query, or run a custom query. To run a query that you previously ran, choose History. To export your results, choose Export results, and then choose a format.

Example queries

Scenario 1

You have a webserver, application server, and database server. You receive a timeout or HTTP 503 error, and you want to determine the cause of the error.

Run a query with the following example variables:

• Action is set to REJECT so that only rejected connections are returned.
• The query includes only internal networks.
• The list of server IP addresses shows both inbound and outbound connections (srcAddr and dstAddr).
• The Limit is set to 5 so that only the first five entries are shown.
• The web server IP address is 10.0.0.4.
• The app server IP address is 10.0.0.5.
• The database server IP address is 10.0.0.6.

Example query:

filter(
   action="REJECT" and
   dstAddr like /^(10\.|192\.168\.)/ and  
   srcAddr like /^(10\.|192\.168\.)/ and
   (srcAddr = "10.0.0.4" or dstAddr = "10.0.0.4" or srcAddr = "10.0.0.5" or dstAddr = "10.0.0.5" or srcAddr = "10.0.0.6" or dstAddr = "10.0.0.6")
)
| stats count(*) as records by srcAddr,dstAddr,dstPort,protocol 
| sort records desc 
| limit 5

Scenario 2

You experience intermittent timeouts on a network interface. To checks for rejects on the network interface over a period of time, run the following query:

fields @timestamp, interfaceId, srcAddr, dstAddr, action
| filter (interfaceId = 'eni-05012345abcd' and action = 'REJECT')
| sort @timestamp desc
| limit 5

Scenario 3

To analyze VPC flow logs and produce a report on a specific network interface, run the following query:

fields @timestamp, @message
 | stats count(*) as records by dstPort, srcAddr, dstAddr as Destination
 | filter interfaceId="eni-05012345abcd"
 | filter dstPort="80" or dstPort="443" or dstPort="22" or dstPort="25"
 | sort HitCount desc
 | limit 10

The preceding query checks the amount of traffic that's sent to different ports.

Scenario 4

To filter VPC flow logs and list IP addresses that try to connect to a specific IP address or CIDR in your VPC, run one of the following queries.

Specific IP address:

fields @timestamp, srcAddr, dstAddr
 | sort @timestamp desc
 | limit 5
 | filter srcAddr like "172.31."

Specific CIDR:

fields @timestamp, srcAddr, dstAddr
 | sort @timestamp desc
 | limit 5
 | filter isIpv4InSubnet(srcAddr,"172.31.0.0/16")

For more example queries, see Sample queries.