Why is my EventBridge Rule silently failing every week?

0

I have two, almost identical, EventBridge Rules and one runs but the other fails and I don't know why. These rules both run a CodeBuild job to rebuild our database every Saturday morning. They are generated with almost identical CloudFormation stacks doing almost the exact same thing. If I run the CodeBuild jobs manually, it works fine, so I don't think it's the job itself. However, one EventBridge Rule runs every week and the other fails.

Since EventBridge doesn't generate any logs of any kind, how can I determine what the problem is? I've tried moving the cron() schedule around to see if there is some sort of issue, but nothing seems to work. The Rule fires and fails, every time.

RNHurt
asked 2 years ago729 views
2 Answers
1
Accepted Answer

EventBridge works with AWS CloudTrail. CloudTrail captures API calls made by or on behalf of your AWS account from the EventBridge console and to EventBridge API operations. Using the information collected by CloudTrail, you can determine what request was made to EventBridge, the IP address from which the request was made, who made the request, when it was made, and more. Alternatively consider using CloudWatch to log you EventBridge activity. For example, setup a new rule that will match all events and as the target define a CloudWatch Log Group. When events are now raised the rules will be captured to the log group. This may be enough to start the debugging exercise and get some insight into what is going on.

AWS documentation also has the following Troubleshooting Amazon EventBridge which may offer some additional avenues to explore.

RoB
answered 2 years ago
  • It would be nice if some of the AWS resources would surface these types of results from CloudTrail. For example, EventBridge has a "Monitoring" link in the Rule details. It would be helpful to have a "CloudTrail" link as well, giving a shortcut to the relevant CloudTrail results.

0

Answering this for posterity.


CloudTrail provided the answer. In my case the IAM permissions were incorrect. The role used by the EventBridge Rule didn't allow the codebuild:StartBuild action for the CodeBuild project name. In a previous CFN update, I had updated the name of the CodeBuild project but forgot to update the project name in the IAM rule.

:facepalm:

RNHurt
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions