By using AWS re:Post, you agree to the Terms of Use
/adding etl workflow to event rule/

adding etl workflow to event rule

0

Hi team,

I want to add a glue workflow as a target to my rule

import * as events from "@aws-cdk/aws-events";

const rule = new events.Rule(
      this,
      "object_created_event",
      {
        description:
          "description here....",
        ruleName: "newObj",
        enabled: true,
        eventPattern: {
          source: ["aws.s3"],
          detailType: ["Object Created"],
          detail: {
            bucket: {
              name: ["test_bucket"],
            },
          },
        },
      }
    );

rule.addTarget(new targets.....);

I can't find glue workflow as a target on the targets list so that I can add it as a target to my rule.

is there any other way to do it other than using ..addTarget method?

thank you.

2 Answers
0
Accepted Answer

I was able to do it using the CfnRule construct :

new events.CfnRule(
      this,
      "rule",
      {
       ....
        targets: [
          {
            arn: `arn:aws:glue:${Aws.REGION}:${Aws.ACCOUNT_ID}:workflow/${myworkFlow.name}`,
            id: "some_id",
            roleArn: myRole.roleArn,
          },
        ],
      }
    );
answered 24 days ago
0

Hello,

Unfortunately, Glue workflow is not a supported target for AWS CDK aws_events module as shown here https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events-readme.html I could also see a feature request was opened recently here https://github.com/aws/aws-cdk/issues/18853

An alternative is to write a simple Lambda function to start a Glue workflow using boto3 client and choose this lambda function as a target.

Glue start workflow boto3 client https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.start_workflow_run

However , both AWS events cli and events boto3 does support Glue workflow as a target.

References:

1. AWS events cli put-targets https://awscli.amazonaws.com/v2/documentation/api/latest/reference/events/put-targets.html

2. AWS events boto3 put-targets https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/events.html#EventBridge.Client.put_targets

3. Starting AWS Glue workflow with eventbridge CLI https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html

answered 25 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions