By using AWS re:Post, you agree to the Terms of Use

How to see log of query made by Glue pyspark script against JDBC DB

0

Is there a way to see or log the SQL query made by glue DynamicFrame calls against a JDBC database source? We are trying to debug a long-running DynamicFrame Query and we are unable to see how the SQL call is being constructed. Would like to know what SQL query statement is being sent to the DB server

asked 2 months ago29 views
1 Answer
0

Please try enabling the continuous logging feature for the glue job and see whether the required logs are visible or not. With the help of this feature, it is possible to view the real time logs in your Cloud watch console. Within these logs you can find executor logs, driver logs and spark job progress bar. I hope that these documentations might be of some help:

  1. https://docs.aws.amazon.com/glue/latest/dg/monitor-continuous-logging.html
  2. https://docs.aws.amazon.com/glue/latest/dg/monitor-profile-glue-job-cloudwatch-metrics.html
SUPPORT ENGINEER
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions