By using AWS re:Post, you agree to the Terms of Use

Interactive Testing of Glue Spark Scripts

0

Is there a way to test Glue scripts interactively on single/few entries of data base tables? I am looking for an option to print out stuff in transformation steps, which would simplify my debugging.

asked 8 months ago101 views
2 Answers
1

You can also develop Glue scripts locally, if that's easier for you to debug. Instructions here. https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.html#develop-local-python

answered 8 months ago
EXPERT
reviewed 6 months ago
1

Depending on the region you are using you could use the Glue Studio notebook .

Alternatively Interactive sessions will allow you to connect any Jupyter notebook to Glue and test interactively your script.

With Glue Studio you could use the data preview feature to check data in each transform.

and finally, legacy method (using Glue 1.0) would be to use dev endpoints.

hope this helps

EXPERT
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions