How can I send AWS WAF log to both CloudWatch logs and S3?

3 minute read
Content level: Intermediate
6

When you enable logging in AWS WAF, you could choose to send WAF logs to Cloudwatch logs or S3 bucket, but you cannot choose both at the same time. This article provides you a workaround.

Short description

When you enable logging in AWS WAF, you could choose to send WAF logs to Cloudwatch logs or S3 bucket, but you cannot choose both at the same time. 

Resolution

To send AWS WAF logs to both CloudWatch Logs log group and S3 bucket, you would follow the following steps:

  1. Configure WAF send logs to Cloudwatch Logs
  2. Create Amazon Data Firehose stream and set stream destination to S3 bucket
  3. Configure Cloudwatch Log subscription to stream logs to Amazon Data Firehose stream

Configure WAF send logs to Cloudwatch Logs log group

  1. In CloudWatch console, go to Log groups tab.
  2. Create a log group with prefix “aws-waf-logs” (without this prefix, you wouldn’t be able to select it in the next step)
  3. In WAF&Shield Console, go to Web ACLs tab under AWS WAF.
  4. Choose the webACL you want to enable logging, then go to Logging and metrics tab.
  5. In Logging section, click Edit.
  6. Choose the log group you just created.

eidt logging

Create Amazon Data Firehose stream and set destination to s3

  1. In Amazon Data Firehose Console, under Firehose streams, click Create Stream
  2. Choose source and destination
  3. Source should be Direct PUT
  4. Destination should be Amazon S3
  5. Firehose stream name is <your stream name>
  6. Decompress source records from Amazon CloudWatch Logs should be checked (Turn on decompression)
  7. Choose S3 bucket that you want to put logs s3://<your bucket>
  8. Leave rest settings to default value

stream

Configure Cloudwatch Log subscription to stream logs to Amazon Data Firehose stream

  1. Go to IAM console
  2. Create new policy PermissionsForCWL, switch to JSON view, under Specify permissions paste the following permission:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "firehose:PutRecord",
            "Resource": "arn:aws:firehose:*:<your account number>:deliverystream/*"
        }
    ]
}

Replace <your account number> to your account number

  1. Create a new role
  2. Under Select trusted entity, select Custom trust policy, paste the following:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
  1. Select Next
  2. Choose permission you just created PermissionsForCWL
  3. Select Next
  4. Give the new role name CWLtoKinesisFirehoseRole
  5. Go to CloudWatch console, go to Log groups tab.
  6. Select the waf log group
  7. Select Subscription filters tab
  8. In Create drop down menu, select Create Data Firehose subscription filter
  9. Under Amazon Data Firehose stream, choose the stream
  10. Under Grant permission, choose IAM role CWLtoKinesisFirehoseRole
  11. Under Configure log format and filters, choose log format JSON and Subscription filter pattern, paste:
{$.action = \* }

This pattern matches any log event where the action field contains any value. 
Or you could only stream BLOCKed request by using the following pattern:

{$.action = "BLOCK"}
  1. Click Start Streaming

subscription

Related information

Subscription filters with Kinesis Data Streams

1 Comment

very helpful!

profile pictureAWS
Peter
replied a month ago