MWAA DAGs are disappearing continously in a loop - only if environment not in "UPDATING" state

0

Hey together,

my company wants to use the service "MWAA", the deployment in our shared VPC etc. went well. I can also access the UI. But when I tried to upload 2 DAGs I get the problem, that they are constantly appearing for 5 seconds and then disappear for like 1 minute, this loop is constantly. (The DAG code is fine and runs on local Airflow, its just some example DAGs)

The logs of the "dag_processor_manager" are also saying, that there are 2 files. dag_processor_manager Logstream

The funny thing is, when I'm updating the MWAA environment (for example the startup script), the DAGs appear on the UI and I can run them normally. As soon as the "update" is finished, the DAGs get to the loop again. MWAA environment status

What could lead to this behavior? Any ideas, how I could fix that?

Thank you very much!

Kind regards Daniel

Dan
asked a month ago70 views
1 Answer
0

Hello Daniel,


Based on the details you have provided, there are several potential root causes that need to be explored:

  1. Browser Cache Issue: Intermittent issues with the display of DAGs can sometimes be attributed to browser caching. Clearing your browser cache and cookies may help resolve this type of problem.

  2. Synchronization Issue: There may be a synchronization problem between the MWAA environment and the S3 bucket. If the MWAA environment is unable to reliably monitor and detect changes in the S3 bucket, it could lead to the observed cyclic behavior of the DAGs appearing and disappearing.

  3. Airflow Configuration or Compatibility Issue: Specific Airflow configuration settings or compatibility problems with the Airflow version used by your MWAA environment could cause inconsistencies in the way the DAGs are recognized and loaded.

  4. S3 Permissions Issue: The MWAA environment requires appropriate permissions to access the S3 bucket where your DAGs are stored. Any issues with the IAM role or S3 bucket policy could prevent the MWAA environment from consistently detecting and loading the DAGs.


Given that, this issue appears to be relatively uncommon, which makes it challenging to accurately pinpoint the cause or provide effective mitigation steps without a detailed investigation. It would be difficult to give a definitive solution without access to complete logs and case tools to review the environment resources thoroughly.


Therefore, I kindly request you to raise a case with AWS Support team. This will enable our team to conduct a thorough investigation and efficiently identify the root cause of the problem.


Link to raise case: https://support.console.aws.amazon.com/support/home#/case/create


Thanks!

AWS
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions