Restart MWAA environment programmatically

Question

Hi,  
  
We use custom packages as dependencies of some of our PythonOperators. Therefore, when we update these packages, we'd like to reflect this update when used by a PythonOperator in MWAA.  
  
The current solution today is to "restart" the MWAA in the UI (by simply editing anything in the MWAA environment configuration).  
  
But we wondered if there was any solution in order to trigger this restart of the MWAA env programmatically, so it is possible to integrate this an extra step of a CodePipeline for instance.  
  
Thanks

Answer

Hi!  
  
You can call update-environment, either with the new S3 version ID for your new plugins/requirements.txt or with an additional configuration override, such as custom.lastupdate = "", as part of your CI/CD pipeline that will force MWAA to restart.  
  
Thanks!

Answer

Note that, as of Airflow 2.0, you do not need to package custom operators as plugins and should simply place them somewhere in your S3 dags folder.  These will be synced automatically by MWAA to each container and no restart is required.  See https://airflow.apache.org/docs/apache-airflow/stable/plugins.html .

Restart MWAA environment programmatically

Contenuto pertinente