Passing parameters to the Glue Workflow using Python

0

I'm trying to run a Glue Workflow from the lambda. According to this documentation there is a parameter RunProperties. So, I'm trying to run the workflow like this:

newRun = glue.start_workflow_run(Name = 'MyWorkflow', RunProperties = { 'key1': 'value1' })

But getting the error:

[ERROR] ParamValidationError: Parameter validation failed:
Unknown parameter in input: "RunProperties", must be one of: Name

It looks like it waits only one parameter Name but not RunProperties. The question is: How can I run my workflow with parameters from a lambda function?

Some additional information:
What I'm trying to achieve here is to run a Glue Workflow when a zip file uploaded to S3. I don't want to run a crawler because it's redundant here. When a zip file is uploaded to S3, a lambda is started and from it I want to start the Glue Workflow.

Alex
feita há 2 anos1981 visualizações
1 Resposta
1
Resposta aceita

Hi, @Alex

I tried and encountered the same problem as you.

As a workaround, you can put the parameters immediately after using the RunId of the workflow started without properties.

newRun = glue.start_workflow_run(Name = 'MyWorkflow')
updateRun = glue.put_workflow_run_properties(
    Name = 'MyWorkflow',
    RunId = newRun['RunId'],
    RunProperties = { 'key1': 'value1'}
    )

Maybe this is a boto3 issue.
I think I can report this issue...

profile picture
ESPECIALISTA
iwasa
respondido há 2 anos
profile picture
ESPECIALISTA
avaliado há um mês
  • @Alex

    I then looked into Boto3.
    Apparently, the Run Properties parameter is available in the latest version of Boto3.
    However, since the Boto3 version of Lambda is old, you can pass parameters with start_workflow_run by creating and using a Lambda layer that packages the latest Boto3.

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas