By using AWS re:Post, you agree to the Terms of Use
/Passing parameters to the Glue Workflow using Python/

Passing parameters to the Glue Workflow using Python


I'm trying to run a Glue Workflow from the lambda. According to this documentation there is a parameter RunProperties. So, I'm trying to run the workflow like this:

newRun = glue.start_workflow_run(Name = 'MyWorkflow', RunProperties = { 'key1': 'value1' })

But getting the error:

[ERROR] ParamValidationError: Parameter validation failed:
Unknown parameter in input: "RunProperties", must be one of: Name

It looks like it waits only one parameter Name but not RunProperties. The question is: How can I run my workflow with parameters from a lambda function?

Some additional information:
What I'm trying to achieve here is to run a Glue Workflow when a zip file uploaded to S3. I don't want to run a crawler because it's redundant here. When a zip file is uploaded to S3, a lambda is started and from it I want to start the Glue Workflow.

asked 15 days ago40 views
1 Answers
Accepted Answer

Hi, @Alex

I tried and encountered the same problem as you.

As a workaround, you can put the parameters immediately after using the RunId of the workflow started without properties.

newRun = glue.start_workflow_run(Name = 'MyWorkflow')
updateRun = glue.put_workflow_run_properties(
    Name = 'MyWorkflow',
    RunId = newRun['RunId'],
    RunProperties = { 'key1': 'value1'}

Maybe this is a boto3 issue.
I think I can report this issue...

answered 15 days ago
  • @Alex

    I then looked into Boto3.
    Apparently, the Run Properties parameter is available in the latest version of Boto3.
    However, since the Boto3 version of Lambda is old, you can pass parameters with start_workflow_run by creating and using a Lambda layer that packages the latest Boto3.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions