Passing parameters to the Glue Workflow using Python

0

I'm trying to run a Glue Workflow from the lambda. According to this documentation there is a parameter RunProperties. So, I'm trying to run the workflow like this:

newRun = glue.start_workflow_run(Name = 'MyWorkflow', RunProperties = { 'key1': 'value1' })

But getting the error:

[ERROR] ParamValidationError: Parameter validation failed:
Unknown parameter in input: "RunProperties", must be one of: Name

It looks like it waits only one parameter Name but not RunProperties. The question is: How can I run my workflow with parameters from a lambda function?

Some additional information:
What I'm trying to achieve here is to run a Glue Workflow when a zip file uploaded to S3. I don't want to run a crawler because it's redundant here. When a zip file is uploaded to S3, a lambda is started and from it I want to start the Glue Workflow.

Alex
posta 2 anni fa1986 visualizzazioni
1 Risposta
1
Risposta accettata

Hi, @Alex

I tried and encountered the same problem as you.

As a workaround, you can put the parameters immediately after using the RunId of the workflow started without properties.

newRun = glue.start_workflow_run(Name = 'MyWorkflow')
updateRun = glue.put_workflow_run_properties(
    Name = 'MyWorkflow',
    RunId = newRun['RunId'],
    RunProperties = { 'key1': 'value1'}
    )

Maybe this is a boto3 issue.
I think I can report this issue...

profile picture
ESPERTO
iwasa
con risposta 2 anni fa
profile picture
ESPERTO
verificato un mese fa
  • @Alex

    I then looked into Boto3.
    Apparently, the Run Properties parameter is available in the latest version of Boto3.
    However, since the Boto3 version of Lambda is old, you can pass parameters with start_workflow_run by creating and using a Lambda layer that packages the latest Boto3.

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande