Cloudwatch alarms with Airflow MWAA

1

Hello,

I am setting up a managed Airflow service (MWAA), and would like to setup alerting.

I am looking at alerting via Cloudwatch metrics. I would like to be alerted when any task fails within a Dag. (TaskInstanceFailures) Unfortunately, it seems to exist only at the '1 dag'/'1 task' level, or at the 'All dags'/'All tasks' level when creating an alarm. Whenever I try to select 'All tasks' for 1 specific Dag, nothing shows up. (Even through the query builder 'Graph with SQL query' option).

I have found a way to use the 'DAGDurationFailed' metric but it really does not seem like an ideal method. Is there any other way to set up an alarm for one dag, without having to write one alarm per task ?

Furthermore, my alarm would send a notification with SNS, to send emails. The email message does not contain much information about the error. We were previously using Sentry for alerting on Airflow task errors, is it possible to send more detailed information about the actual task error in the email alert via cloudwatch ?

Thank you.

1개 답변
1

One option is using a callback on your task failures that emit CloudWatch metrics or alarms.

AWS
John_J
답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인