FailedInvocation when connecting eventbridge to glue workflow

0

I set up the resources to trigger glue job through eventbridge. But when I tested in console, Invocations == FailedInvocations == TriggeredRules == 1.

What can I do to fix it?

######### AWS Glue Workflow ############
# Create a Glue workflow that triggers the Glue job
resource "aws_glue_workflow" "example_glue_workflow" {
  name        = "example_glue_workflow"
  description = "Glue workflow that triggers the example_glue_job"
}

resource "aws_glue_trigger" "example_glue_trigger" {
  name = "example_glue_trigger"
  workflow_name = aws_glue_workflow.example_glue_workflow.name
  type = "EVENT"
  actions {
    job_name = aws_glue_job.example_glue_job.name
  }
}


######### AWS EventBridge ##############
resource "aws_cloudwatch_event_rule" "example_etl_trigger" {
  name        = "example_etl_trigger"
  description = "Trigger Glue job when a request is made to the API endpoint"

  event_pattern = jsonencode({
    "source": ["example_api"]
  })
}

resource "aws_cloudwatch_event_target" "glue_job_target" {
  rule      = aws_cloudwatch_event_rule.example_etl_trigger.name
  target_id = "example_event_target"

  arn = aws_glue_workflow.example_glue_workflow.arn
  role_arn = local.example_role_arn
}
asked a year ago580 views
1 Answer
0

Hello,

I understand that you are trying to trigger Glue Workflow using EventBridge, however rule was failing to invoke the target.

  • TriggeredRules metric indicate that the rule matched an event and forwarded the event to the target.
  • Invocations metric indicate that the target was invoked by the rule
  • FailedInvocations metric indicates that there is an issue invoking the target.

FailedInvocations represent a permanent failure and might be the result of incorrect permissions or a misconfiguration of the target. I kindly request you to make sure the target IAM role has required permissions. Please refer the following documentation to learn more about IAM permissions.

[+] Starting an AWS Glue workflow with an Amazon EventBridge event

Further, make sure your IAM role also has following "Trust relationship" policy attached.


{
"Version": "2012-10-17",
"Statement": [{
    "Effect": "Allow",
    "Principal": {
        "Service": [
            "glue.amazonaws.com",
            "events.amazonaws.com"
        ]
    },
    "Action": "sts:AssumeRole"
}]
}

You can also consider configuring a dead-letter queue (DLQ) to troubleshoot/ debug the failed invocations issues/ avoid losing events after they fail to be delivered to a target and send all failed events to it for processing later.

[+] Event retry policy and using dead-letter queues

For detailed investigation, we would need to check the resources and the permissions in your account. If you still have queries regarding this, I would like to request you to reach out to our support team by opening a support case with all the resource details, and we will investigate the issue further.

Let me know if you have any further queries. Thank you for your interest in re:Post community.

Best Regards, Ashish

References:

  1. How can I troubleshoot issues with EventBridge rules?

  2. Troubleshooting Amazon EventBridge

  3. Build a serverless event-driven workflow with AWS Glue and Amazon EventBridge

AWS
SUPPORT ENGINEER
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions