By using AWS re:Post, you agree to the Terms of Use
/How to retrieve job run id from the script itself in python shell glue job/

How to retrieve job run id from the script itself in python shell glue job

0

I need to print my python shell glue job run id from the script itself into cloudwatch logs. But I didn’t find a way to find the current job run id, I have found a partial way using boto3 functions but its giving all glue job run ids and if we want to find out current job run id then its saying to filter with job run status, but this procedure works only there is one job run at a time, in my scenario I am expecting concurrent runs. Can you help me on how to find out the current job run id??

asked 2 months ago50 views
1 Answers
0

You can pass this in your AWS Glue Scripts. See if this helps

import sys 
from awsglue.utils import getResolvedOptions 
args = getResolvedOptions(sys.argv, ['JOB_NAME']) 
job_run_id = args['JOB_RUN_ID']
answered 2 months ago
  • I have tried this, but the job is throwing an error -> KeyError: JOB_RUN_ID, and as per your message it seems like job is trying to retrieve a job parameter which is not even passed. Without passing job run id in the job parameters how can we retrieve it from script?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions