ValueError - Please resolve the pending DynamicFrame choices before using this transformation - AWS Visual Glue Job

Question

I am creating an ETL pipeline over AWS Visual Glue Jobs going from a Glue Catalog => Change Schema => Flatten => S3 Bucket. However, I am getting the same error consistently during this process:

```
ValueError - Please resolve the pending DynamicFrame choices before using this transformation
```

The transformation being referred to is the gs_flatten() step in my script, but I am not sure how to resolve the DynamicFrame choices without ResolveChoices which would turn my job into a script and no longer a visual job. Any help to resolve this issue would be greatly appreciated.

Giovanni Lauria · Accepted Answer

The error you’re encountering in your AWS Glue job is due to ambiguous types within a DynamicFrame. Before certain transformations can be used, these ambiguities, represented as choice types, need to be resolved. This can be done using the ResolveChoice class in AWS Glue, specifically the DynamicFrame.resolveChoice() method. You can specify resolution strategies such as cast, make_cols, make_struct, and project to handle fields that contain multiple types. For example, df = df.resolveChoice(specs = [("ColumnB[].ColumnD", "cast:double")])

ValueError - Please resolve the pending DynamicFrame choices before using this transformation - AWS Visual Glue Job

Ajoutez votre réponse

Contenus pertinents