DAG import errors are not disappearing in the UI even after resolving them

Question

I'm facing an issue with the Airflow Web UI in the Managed Airflow environment. I uploaded a DAG file in the Airflow designated bucket in S3. It had few import errors. I fixed them and re-uploaded the file. But I still see the errors in the UI. The DAG ran fine without any issues.   
  
I thought since the web server process is not getting restarted, it's unaware of the fixes done. So, I made a minor change in the Airflow configuration options of the environment, thereby forcing the services to restart. But it did not help.  
  
I have a couple of questions:  
1. Why there is no option to restart or stop the Airflow environment? It would be helpful in reducing the costs for the customer as well as in restarting UI after making any changes to the configuration  
2. How to let the webserver process know that the DAG import errors have been resolved so it does not show the errors in the UI anymore?  
  
Edited by: ArjunAnaji on May 30, 2021 10:49 AM

Accepted Answer

Hi!  
  
1. While no dedicated restart option exists, adding/removing/altering an arbitrary configuration override or log level option will force a restart.   
  
Stopping an environment is not an option as there are numerous dependencies in Airflow such as the meta database that cannot be easily stopped. As an alternative, environments can be created and destroyed via API or CloudFormation.  
  
2. MWAA is running open source Airflow, either 1.10.12 or 2.0.2, and does not alter it's behavior.  If the web server is not being updated to the lack of errors, it is likely that the scheduler responsible for serializing the DAGs has not completed or still sees errors.  See your DAG processing logs in CloudWatch (may require INFO level logging).  You can also debug locally using https://github.com/aws/aws-mwaa-local-runner   
  
Thanks!

DAG import errors are not disappearing in the UI even after resolving them

Relevant content