How do I troubleshoot an EventBridge rule with Amazon ECS?

Short description

When you use an EventBridge rule to invoke an Amazon ECS task, EventBridge calls the RunTask API to run the tasks.

The EventBridge rule might fail to invoke your ECS task because of the following reasons:

• The EventBridge rule configuration is incorrect.
• The EventBridge rule's target configuration is incorrect.
• The AWS Identity and Management (IAM) role doesn't have the permissions to run the tasks.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Check the EventBridge rule configuration

Check whether the EventBridge rule is in the ENABLED status. If it's in the DISABLED status, then reactivate the rule and perform tests to determine whether the rule can create a task.

To check the status of the rule, you can use the EventBridge console or the AWS CLI.

EventBridge console

Complete the following steps:

1. Open the EventBridge console.
2. In the navigation pane, choose Rules.
3. Under Select Event bus, select the event bus that's associated with the rule.
4. Choose the rule to view its status.

AWS CLI

Run the following describe-rule command:

aws events describe-rule --name "DailyLambdaFunction" --region "us-east-1"

Note: Replace us-east-1 with your AWS Region.

In the command's output, check the State parameter to identify the rule's status. If the rule is in the DISABLED status, then check the AWS CloudTrail logs to identify why the rule was deactivated.

Make sure that the event pattern is correct and matches the events that are from the event source. To verify the event pattern, use the EventBridge Sandbox.

Check the EventBridge rule's target configuration

Take the following actions:

• Check the ECS Cluster, TaskDefinition, and Network configuration parameters for the EventBridge rule's target to make sure that they're correct.
• Verify that the EventBridge IAM role has the required permissions to run the task.

Use the following Amazon CloudWatch metrics to help you troubleshoot:

• Check for the TriggeredRules metric. CloudWatch shows the metric only when an event triggers a rule.
• Use the Invocations metric to check how many times EventBridge invoked a rule and then tried to run the ECS task.
• Use the FailedInvocations metric to determine the number failed invocations. If the value is high value, then your target might be incorrectly configured.
  Note: The Sum statistic might be the most useful statistic to check for the FailedInvocations metric.

Determine why the RunTask API call failed

Complete the following steps:

1. Open the CloudTrail console.
2. In the navigation pane, choose Event history.
3. In the Lookup attributes dropdown list, choose Event name.
4. In the text box, enter RunTask to filter out the events that are related to the RunTask API.
5. Filter the time range based on the expected ECS task runtime.
   Note: The default values for the time range are 30 minutes, 1 hour, 3 hours, and 12 hours. To specify a custom time range, choose Absolute range.
6. Choose the event, and then review the JSON event record in Event record section. Look for a failure reason under errorCode and errorMessage.

Use EventBridge DLQ

EventBridge rules support dead-letter queues (DLQs). DLQs uses the Amazon Simple Queue Service (Amazon SQS) to store failed invocation events in a standard queue that you specify. When the event rule doesn't invoke its target, EventBridge delivers a JSON payload that contains invocation details and responses from the target to the DLQ. You can analyze the failed event delivery in the DLQ to resolve the issue.

Related information

How can I troubleshoot issues with Amazon EventBridge rules?

How can I use a dead-letter queue to troubleshoot FailedInvocations for EventBridge rules?