Passing parameters to the Glue Workflow using Python

0

I'm trying to run a Glue Workflow from the lambda. According to this documentation there is a parameter RunProperties. So, I'm trying to run the workflow like this:

newRun = glue.start_workflow_run(Name = 'MyWorkflow', RunProperties = { 'key1': 'value1' })

But getting the error:

[ERROR] ParamValidationError: Parameter validation failed:
Unknown parameter in input: "RunProperties", must be one of: Name

It looks like it waits only one parameter Name but not RunProperties. The question is: How can I run my workflow with parameters from a lambda function?

Some additional information:
What I'm trying to achieve here is to run a Glue Workflow when a zip file uploaded to S3. I don't want to run a crawler because it's redundant here. When a zip file is uploaded to S3, a lambda is started and from it I want to start the Glue Workflow.

Alex
asked 2 years ago1959 views
1 Answer
1
Accepted Answer

Hi, @Alex

I tried and encountered the same problem as you.

As a workaround, you can put the parameters immediately after using the RunId of the workflow started without properties.

newRun = glue.start_workflow_run(Name = 'MyWorkflow')
updateRun = glue.put_workflow_run_properties(
    Name = 'MyWorkflow',
    RunId = newRun['RunId'],
    RunProperties = { 'key1': 'value1'}
    )

Maybe this is a boto3 issue.
I think I can report this issue...

profile picture
EXPERT
iwasa
answered 2 years ago
profile picture
EXPERT
reviewed 21 days ago
  • @Alex

    I then looked into Boto3.
    Apparently, the Run Properties parameter is available in the latest version of Boto3.
    However, since the Boto3 version of Lambda is old, you can pass parameters with start_workflow_run by creating and using a Lambda layer that packages the latest Boto3.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions