Cloudwatch alarms with Airflow MWAA

1

Hello,

I am setting up a managed Airflow service (MWAA), and would like to setup alerting.

I am looking at alerting via Cloudwatch metrics. I would like to be alerted when any task fails within a Dag. (TaskInstanceFailures) Unfortunately, it seems to exist only at the '1 dag'/'1 task' level, or at the 'All dags'/'All tasks' level when creating an alarm. Whenever I try to select 'All tasks' for 1 specific Dag, nothing shows up. (Even through the query builder 'Graph with SQL query' option).

I have found a way to use the 'DAGDurationFailed' metric but it really does not seem like an ideal method. Is there any other way to set up an alarm for one dag, without having to write one alarm per task ?

Furthermore, my alarm would send a notification with SNS, to send emails. The email message does not contain much information about the error. We were previously using Sentry for alerting on Airflow task errors, is it possible to send more detailed information about the actual task error in the email alert via cloudwatch ?

Thank you.

1 回答
1

One option is using a callback on your task failures that emit CloudWatch metrics or alarms.

AWS
John_J
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则