Updating a Glue Script without affecting other job parameters using boto3

0

Hi , I am trying to build a Glue Deployment System using Boto3 . I could upload a Glue Script Python file to Glue Sources S3 bucket and create a job. No problem there.

The next step is , when I want to update the script file, and not tamper any of the other job parameters. I was trying to re-upload the updated script file ( same name as the previous one ) and was assuming that the job will show the new version when checking it from the Glue UI , but it doesnt . Still shows the older version .

I can delete and recreate the job. but doing that my job-bookmarks will be lost . I am not sure if the job update can be done this way or will the update_job() function can do this or will be the right way to do this . Any help ?

asked 2 years ago1210 views
2 Answers
1
Accepted Answer

By updating the S3 location of the script, you are only creating a new version of the code in S3. To completely replace the code in Glue, you would have to use boto3 to update Glue job with the location of the new script- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.update_job. This will not reset any of the other parameters.

response = client.update_job({
   "JobName": "string",
   "JobUpdate": { 
      "Command": { 
         "Name": "string",
         "PythonVersion": "string",
         "ScriptLocation": "string"
      }
   }
})
profile pictureAWS
answered 2 years ago
0

I followed ananthtm's answer and could update the jobs.It however overwrote the other parameters like Connections etc when I passed the basic ones as mentioned . I then added the other parameters in the update statement and it worked fine.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions