Call a glue Job from within another without using vpc endppoints or SG

Question

Hi all,

Is it possible to call a glue job/ or python script from within another glue job without passing by **glue endpoint** and adding a new rule in SG?

the following code uses glue endpoint and doesn't work without adding extra rules in the Security Group :

```
client = boto3.client('glue', 'AWS_REGION')
client.start_job_run(JobName='test_call_script')
```
I want to call an extra script that sends an email from within my original glue job without the need to use SES or glue VPC endpoints or opening new rules in the SG.
just call another glue script that can send an email out of the box

Answer

Hi ,

if the first job has a connection and thus is linked to a VPC that will not be possible.
The best way to do it , it is a full separation , exit from the initial job with an explicit exit code , and message or just writing the message ou want to send via email in a file on S3.

Then use a Glue Workflow or just a cloudwatch event to trigger a second job that read the output message from the first job and send the email.
with the second job not attached to a VPC you do not need the VPC endpoints or SG rules.

[Cloudwatch event for Glue Failed job:](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types)

```
{
    "version":"0",
    "id":"abcdef01-1234-5678-9abc-def012345678",
    "detail-type":"Glue Job State Change",
    "source":"aws.glue",
    "account":"123456789012",
    "time":"2017-09-07T06:02:03Z",
    "region":"us-west-2",
    "resources":[],
    "detail":{
        "jobName":"MyJob",
        "severity":"ERROR",
        "state":"FAILED",
        "jobRunId":"jr_0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
        "message":"JobName:MyJob and JobRunId:jr_0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef failed to execute with exception Role arn:aws:iam::123456789012:role/Glue_Role should be given assume role permissions for Glue Service."
    }
}
```

in the Glue script just use a python exit statement passing the message you want as exception and it should appear as meesage.

hope this is clear

Call a glue Job from within another without using vpc endppoints or SG

相关内容