Cloudwatch alarms with Airflow MWAA

1

Hello,

I am setting up a managed Airflow service (MWAA), and would like to setup alerting.

I am looking at alerting via Cloudwatch metrics. I would like to be alerted when any task fails within a Dag. (TaskInstanceFailures) Unfortunately, it seems to exist only at the '1 dag'/'1 task' level, or at the 'All dags'/'All tasks' level when creating an alarm. Whenever I try to select 'All tasks' for 1 specific Dag, nothing shows up. (Even through the query builder 'Graph with SQL query' option).

I have found a way to use the 'DAGDurationFailed' metric but it really does not seem like an ideal method. Is there any other way to set up an alarm for one dag, without having to write one alarm per task ?

Furthermore, my alarm would send a notification with SNS, to send emails. The email message does not contain much information about the error. We were previously using Sentry for alerting on Airflow task errors, is it possible to send more detailed information about the actual task error in the email alert via cloudwatch ?

Thank you.

1 réponse
1

One option is using a callback on your task failures that emit CloudWatch metrics or alarms.

AWS
John_J
répondu il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions