Send customised notification of Glue job using SNS

0

I need to send notification having ** job execution time**, number of records processed and the rest of the things using SNS. Below is the template I am using, but I need these two inputs too in my notification: { "detail-type": ["Glue Job State Change"], "source": ["aws.glue"], "detail": { "jobName": ["job_name1","job_name2"], "severity": ["ERROR"], "state": ["FAILED", "ERROR", "TIMEOUT", "STOPPED"] } } Can you please help me.

1 Answer
0

Hi ,

the 2 additional inputs (job execution time, number of records processed) you require are not generated automatically by Glue.

The job execution time you can retrieve it using the Glue APIs get_job_run passing the latest run_id.

The number of record preprocessed you would need to calculate it in the job and then export it in and external file or db table you can query to retrieve it and add it to the SNS message before you send it.

hope this helps.

AWS
EXPERT
answered 2 years ago
  • Hi, thanks for the response. How do I use the API and the querying file/table part, should I use a Lambda function for this?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions