Eventbridge Rule not receiving failed events from glue crawler

Question

Hi all, I created a EventBridge rule with the following event pattern that is suppose to match all glue crawler state changes and send them to a lambda function, however, the rule is only only sending "Started" (when the crawler starts) and "Succeeded" (when the crawler succeeds) states, and never received "Failed" (when the crawler fails) state.

The doc [here](https://docs.aws.amazon.com/glue/latest/dg/automating-awsglue-with-cloudwatch-events.html) says: Events for "detail-type":"Glue Crawler State Change" are generated for Started, Succeeded, and Failed.

{
        "source": ["aws.glue"],
        "detail-type": ["Glue Crawler State Change"],
        "detail": {
            "state": ["Started", "Succeeded", "Failed"]
        }
    }

Am I missing something or is the "Failed" state not available as an event for "detail-type":"Glue Crawler State Change"?

Answer

I have the same requirement in my environment and the above code works perfectly fine for me. If "Started" and "Succeeded" states works fine for you then you can follow these steps to troubleshoot -

1. Check if Eventbridge rule is getting triggered - from the monitoring tab in your eventbridge rule, see if your rule has been invoked. If it is invoked then lambda has received the event. If not eventbridge has not received the config change.
2. Check your permissions - If you are using custom event bus check its permissions
3.  GO with aws config - develop the event pattern as source as aws config

Answer

Hi, 
as per https://docs.aws.amazon.com/glue/latest/dg/automating-awsglue-with-cloudwatch-events.html, FAILED is available for `Glue Crawler State Change`

```
Events for "detail-type":"Glue Job State Change" are generated for SUCCEEDED, FAILED, TIMEOUT
, and STOPPED.
```

You may want to follow the guidance of https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-troubleshooting.html to investigate your current issue.

This article also shows how to trigger jobs based on your "Glue Job State Change" event: https://repost.aws/knowledge-center/glue-cross-account-job-trigger
You may compare your own settings with this example.

Best,

Didier

Eventbridge Rule not receiving failed events from glue crawler

Relevant content