MWAA inconsistent tasks states

0

Hi,

We are using MWAA with 2.6.3 version of Airflow, mw1.small, 2 schedulers, 1-12 workers. Every now and then, maybe once a month, in a DAG with multiple tasks, there is something strange happening - an upstream task is marked with success status, but the following downstream task fails with upstream_failed status. Strange task states

There is nothing in the Airflow task logs indicating why it happened. Also, there are other users who observe similar behaviour, for example, here: https://github.com/apache/airflow/discussions/33528 . The Airflow devs suggest that it is specific to MWAA. How to investigate the issue? It is not possible to reproduce it intentionally, and there is no pattern for such behavior.

No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions