Cloudwatch alarms with Airflow MWAA

1

Hello,

I am setting up a managed Airflow service (MWAA), and would like to setup alerting.

I am looking at alerting via Cloudwatch metrics. I would like to be alerted when any task fails within a Dag. (TaskInstanceFailures) Unfortunately, it seems to exist only at the '1 dag'/'1 task' level, or at the 'All dags'/'All tasks' level when creating an alarm. Whenever I try to select 'All tasks' for 1 specific Dag, nothing shows up. (Even through the query builder 'Graph with SQL query' option).

I have found a way to use the 'DAGDurationFailed' metric but it really does not seem like an ideal method. Is there any other way to set up an alarm for one dag, without having to write one alarm per task ?

Furthermore, my alarm would send a notification with SNS, to send emails. The email message does not contain much information about the error. We were previously using Sentry for alerting on Airflow task errors, is it possible to send more detailed information about the actual task error in the email alert via cloudwatch ?

Thank you.

1回答
1

One option is using a callback on your task failures that emit CloudWatch metrics or alarms.

AWS
John_J
回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ