Interactive Testing of Glue Spark Scripts

Question

Is there a way to test Glue scripts interactively on single/few entries of data base tables? I am looking for an option to print out stuff in transformation steps, which would simplify my debugging.

Answer

Depending on the region you are using you could use the[ Glue Studio notebook ](https://docs.aws.amazon.com/glue/latest/ug/notebooks-chapter.html).

Alternatively [Interactive sessions ](https://docs.aws.amazon.com/glue/latest/dg/interactive-sessions-chapter.html)will allow you to connect any Jupyter notebook to Glue and test interactively your script.

With Glue Studio you could use the [data preview feature](https://docs.aws.amazon.com/glue/latest/ug/job-editor-features.html) to check data in each transform.

and finally, legacy method (using Glue 1.0) would be to [use dev endpoints](https://docs.aws.amazon.com/glue/latest/dg/dev-endpoint.html).

hope this helps

Answer

You can also develop Glue scripts locally, if that's easier for you to debug. Instructions here. 
[https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.html#develop-local-python ]()

Interactive Testing of Glue Spark Scripts

相关内容