getting failedinvocations in aws eventbridge

Question

I have programmatically defined a eventbridge rule to send an event when a crawler completes.
response = event_client.put_rule(
            Name="newmyrule",
            EventPattern='{"detail-type": ["Glue Crawler State Change"],"source": ["aws.glue"],"detail": {"crawlerName":["'+crawler_name+'"],"state": ["Succeeded"]}}'
        ) 
        print("put_rule="+str(response))
        put_target_response = event_client.put_targets(
            Rule='newmyrule',
            Targets=[{
                'Id': 'mylambdafn',
                'Arn': 'arn:aws:lambda:us-west-1:xxxxxxxxxxxxx:function:mylambdafn'
            }]
        )
enable_rule_response = event_client.enable_rule(Name='newmyrule')

I have also defined the crawler through boto3.

create_crawler_response =glue.create_crawler(
                    Name=crawler_name,
                    Role='arn:aws:iam::xxxxxxxxxx:role/ravi-glue-access',
                    DatabaseName='noah-ingest',
                    #TablePrefix="",
                    Targets={'S3Targets': [{'Path': s3_target}]},
                    SchemaChangePolicy={
            		    'UpdateBehavior': 'UPDATE_IN_DATABASE',
            		    'DeleteBehavior': 'DELETE_FROM_DATABASE'
        		    }
        	    )
It looks similar to rules defined through the console but results in failedinvocations. How do I fix this.

thanks,
Ravi.

Answer

I suspect your lambda doesn't have the appropriate permissions that would allow eventbridge to trigger it.

If you're using boto3 to set this up, you probably need to use `add_permission()` (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html#Lambda.Client.add_permission) something like

```
response = lambda.add_permission(
  functionName = '_insertFunctionNameHere_',
  statementId = 'AllowExecutionFromEventbridge',
  action='lambda:invokeFunction',
  principal='events.amazonaws.com',
  sourceArn='_insertARNofEventBridgeRule_'
)
```

replacing the values beginning _ appropriately.

See https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-troubleshooting.html#eb-lam-function-not-invoked for more info

Answer

Try out the code below. Replace xxxxx with youractual aws account ID. Also provide correct Role and Target values. If you still encounter any issues, try and check the IAM roles and permissions for your lambda function and Glue crawler.
Hope it helps.

import boto3
import json

# Create an EventBridge client
event_client = boto3.client('events')

# Define the crawler name
crawler_name = 'my-crawler'

# Define the event pattern
event_pattern = {
    "source": ["aws.glue"],
    "detail-type": ["Glue Crawler State Change"],
    "detail": {
        "crawlerName": [crawler_name],
        "state": ["Succeeded"]
    }
}

# Define the target
target = {
    "Arn": "arn:aws:lambda:us-west-1:xxxxxxxxxxxxx:function:mylambdafn",
    "Id": "mylambdafn"
}

# Define the rule name
rule_name = "my-crawler-completed-rule"

# Create the rule
create_rule_response = event_client.put_rule(
    Name=rule_name,
    EventPattern=json.dumps(event_pattern)
)

# Add the target to the rule
put_targets_response = event_client.put_targets(
    Rule=rule_name,
    Targets=[target]
)

# Enable the rule
enable_rule_response = event_client.enable_rule(
    Name=rule_name
)

# Define the Glue crawler
glue = boto3.client('glue')
create_crawler_response = glue.create_crawler(
    Name=crawler_name,
    Role='arn:aws:iam::xxxxxxxxxx:role/ravi-glue-access',
    DatabaseName='noah-ingest',
    Targets={'S3Targets': [{'Path': s3_target}]},
    SchemaChangePolicy={
        'UpdateBehavior': 'UPDATE_IN_DATABASE',
        'DeleteBehavior': 'DELETE_FROM_DATABASE'
    }
)

print("EventBridge rule created successfully")

getting failedinvocations in aws eventbridge

Create an EventBridge client

Define the crawler name

Define the event pattern

Define the target

Define the rule name

Create the rule

Add the target to the rule

Enable the rule

Define the Glue crawler

관련 콘텐츠