Updating a Glue Script without affecting other job parameters using boto3

Question

Hi , 
I am trying to build a Glue Deployment System using Boto3 . I could upload a Glue Script Python file to Glue Sources S3 bucket and create a job. No problem there.

The next step is , when I want to update the script file, and not tamper any of the other job parameters. I was trying to re-upload the updated script file ( same name as the previous one ) and was assuming that the job will show the new version when checking it from the Glue UI , but it doesnt . Still shows the older  version .

I can delete and recreate the job. but doing that my job-bookmarks will be lost . 
I am not sure if the job update can be done this way or will the update_job() function can do this or will be the right way to do this . Any help ?

Accepted Answer

By updating the S3 location of the script, you are only creating a new version of the code in S3. To completely replace the code in Glue, you would have to use boto3 to update Glue job with the location of the new script- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.update_job. This will not reset any of the other parameters.

```
response = client.update_job({
   "JobName": "string",
   "JobUpdate": { 
      "Command": { 
         "Name": "string",
         "PythonVersion": "string",
         "ScriptLocation": "string"
      }
   }
})
```

Answer

I followed ananthtm's answer and could update the jobs.It however overwrote the other parameters like Connections etc  when I passed the basic ones as mentioned . I then added the other parameters in the update statement and it worked fine.

Updating a Glue Script without affecting other job parameters using boto3

관련 콘텐츠