How do I create and troubleshoot an EventBridge rule that triggers when changes to S3 objects or operations occur?

Resolution

To use EventBridge to capture Amazon S3 events, you can either use AWS CloudTrail or capture events directly from Amazon S3.

Each option has its own set of metadata. Events offered by CloudTrail include details from AWS API, AWS Identity and Access Management (IAM), and userAgent. Events sent from Amazon S3 include object details, for example, size.

Capture S3 events through CloudTrail

You can use CloudTrail to monitor and capture changes made to S3 buckets, such as creation, modification, or deletion. CloudTrail automatically forwards these events to EventBridge.

To capture S3 events through CloudTrail, complete the following steps:

1. Create a CloudTrail trail to log events on the relevant S3 buckets.
2. Create a resource to process the S3 events.
   Note: The resource that you create can be any supported EventBridge target. For example, AWS Lambda, an Amazon Elastic Container Service (Amazon ECS) task, an API destination, or an Amazon Kinesis Data Firehose delivery stream.
3. Create an EventBridge rule with an event pattern to match the S3 events. The following example shows an event pattern that matches PutObject, DeleteObject, and DeleteObjects API calls made against a specific S3 bucket:

   {  "source": [
       "aws.s3"
     ],
     "detail-type": [
       "AWS API Call via CloudTrail"
     ],
     "detail": {
       "eventName": [
         "PutObject",
         "DeleteObject",
         "DeleteObjects"
       ],
       "requestParameters": {
         "bucketName": [
           "123456789012-prod-app"
         ],
         "key": [
           {
             "prefix": "feature1/"
           },
           {
             "prefix": "feature2/"
           }
         ]
       }
     }
   }

   Note: The keys start with feature1/ and feature2/.
4. Associate the resource that you created with the EventBridge rule as a target.

For more information about how to capture data events, see Tutorial: Log Amazon S3 object-level operations using EventBridge.

Use Amazon S3 to capture S3 events

This option doesn't require CloudTrail. The S3 bucket is configured to send all operations on that resource to the EventBridge service.

Activate S3 events to EventBridge

1. Open the Amazon S3 console.
2. In the Buckets list, choose the name of the bucket that you want to activate events for.
   Note: For more information, see Enabling Amazon EventBridge.
3. Choose Properties.
4. Navigate to the Event Notifications section, and then find Amazon EventBridge. Choose Edit.
5. For Send notifications to Amazon EventBridge for all events in this bucket, choose On.

Create a resource to process S3 events

The resource that you create can be any supported EventBridge target.

Create an EventBridge rule with an event pattern to match the S3 events

Create an EventBridge rule with an event pattern to match the S3 events. The following is an example of an event pattern that initiates when an object is created or deleted within a specific S3 bucket:

{  "source": [
    "aws.s3"
  ],
  "detail-type": [
    "Object Created",
    "Object Deleted"
  ],
  "detail": {
    "bucket": {
      "name": [
        "123456789012-prod-app"
      ]
    },
    "object": {
      "size": [
        {
          "numeric": [
            ">=",
            15728640
          ]
        }
      ],
      "key": [
        {
          "suffix": ".zip"
        }
      ]
    }
  }
}

Note: The pattern is a filter on file name parameters such as a size larger than 15 MB, and with an extension of zip.

Associate the resource with the rule

Associate the resource with the rule that you created with the EventBridge rule as a target.

Troubleshoot errors

Make sure that the necessary CloudTrail trails or S3 Event Notifications are properly configured.

Use EventBridge TriggeredRules and Invocations metrics to confirm successful matches to the rule and successful target invocations.

TriggeredRules data points indicate that the rule matched the incoming event. If there are no data points, then re-evaluate the event pattern.

The FailedInvocations metric indicates that there's a problem with how EventBridge forwards events to the target. This might be related to IAM or resource policy permissions, or target configurations.

Because the event pattern is case-sensitive, "FAILED" and "failed" are processed differently. The EventBridge Sandbox tool offers example events from AWS services and EventBridge partners. The tool validates incoming events against the event pattern.

To match an event, all the fields in an event pattern must match. To troubleshoot the event pattern, remove fields to create a more generic event pattern. For more information, see How can I troubleshoot issues with EventBridge rules?

EventBridge rules can have up to five targets. To troubleshoot problems, temporarily attach a CloudWatch Logs group target or an Amazon Simple Notification Service (Amazon SNS) topic. Both the log group and notifications capture the event forwarded to the target and help isolate misconfigurations.

Related information

How do I create an EventBridge rule to automate responses to CloudTrail API calls?

Use Amazon S3 Event Notifications with Amazon EventBridge

Using dynamic Amazon S3 event handling with Amazon EventBridge