getting failedinvocations in aws eventbridge

0

I have programmatically defined a eventbridge rule to send an event when a crawler completes. response = event_client.put_rule( Name="newmyrule", EventPattern='{"detail-type": ["Glue Crawler State Change"],"source": ["aws.glue"],"detail": {"crawlerName":["'+crawler_name+'"],"state": ["Succeeded"]}}' ) print("put_rule="+str(response)) put_target_response = event_client.put_targets( Rule='newmyrule', Targets=[{ 'Id': 'mylambdafn', 'Arn': 'arn:aws:lambda:us-west-1:xxxxxxxxxxxxx:function:mylambdafn' }] ) enable_rule_response = event_client.enable_rule(Name='newmyrule')

I have also defined the crawler through boto3.

create_crawler_response =glue.create_crawler( Name=crawler_name, Role='arn:aws:iam::xxxxxxxxxx:role/ravi-glue-access', DatabaseName='noah-ingest', #TablePrefix="", Targets={'S3Targets': [{'Path': s3_target}]}, SchemaChangePolicy={ 'UpdateBehavior': 'UPDATE_IN_DATABASE', 'DeleteBehavior': 'DELETE_FROM_DATABASE' } ) It looks similar to rules defined through the console but results in failedinvocations. How do I fix this.

thanks, Ravi.

asked a year ago358 views
2 Answers
1

I suspect your lambda doesn't have the appropriate permissions that would allow eventbridge to trigger it.

If you're using boto3 to set this up, you probably need to use add_permission() (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html#Lambda.Client.add_permission) something like

response = lambda.add_permission(
  functionName = '_insertFunctionNameHere_',
  statementId = 'AllowExecutionFromEventbridge',
  action='lambda:invokeFunction',
  principal='events.amazonaws.com',
  sourceArn='_insertARNofEventBridgeRule_'
)

replacing the values beginning _ appropriately.

See https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-troubleshooting.html#eb-lam-function-not-invoked for more info

profile picture
answered a year ago
0

Try out the code below. Replace xxxxx with youractual aws account ID. Also provide correct Role and Target values. If you still encounter any issues, try and check the IAM roles and permissions for your lambda function and Glue crawler. Hope it helps.

import boto3 import json

Create an EventBridge client

event_client = boto3.client('events')

Define the crawler name

crawler_name = 'my-crawler'

Define the event pattern

event_pattern = { "source": ["aws.glue"], "detail-type": ["Glue Crawler State Change"], "detail": { "crawlerName": [crawler_name], "state": ["Succeeded"] } }

Define the target

target = { "Arn": "arn:aws:lambda:us-west-1:xxxxxxxxxxxxx:function:mylambdafn", "Id": "mylambdafn" }

Define the rule name

rule_name = "my-crawler-completed-rule"

Create the rule

create_rule_response = event_client.put_rule( Name=rule_name, EventPattern=json.dumps(event_pattern) )

Add the target to the rule

put_targets_response = event_client.put_targets( Rule=rule_name, Targets=[target] )

Enable the rule

enable_rule_response = event_client.enable_rule( Name=rule_name )

Define the Glue crawler

glue = boto3.client('glue') create_crawler_response = glue.create_crawler( Name=crawler_name, Role='arn:aws:iam::xxxxxxxxxx:role/ravi-glue-access', DatabaseName='noah-ingest', Targets={'S3Targets': [{'Path': s3_target}]}, SchemaChangePolicy={ 'UpdateBehavior': 'UPDATE_IN_DATABASE', 'DeleteBehavior': 'DELETE_FROM_DATABASE' } )

print("EventBridge rule created successfully")

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions