MWAA legacy airflow and python versions

Question

Hi all! I am new to MWAA service. I would like to use it to manage my dags, which I deployed by myself on ec2 instances with airflow running in a docker. It would be much simpler for me to use managed airflow :)
But: I already use Dynamic Task Mapping which is from Airflow 2.3 where MWAA Airflow is 2.2 + my packages are python 3.9 where MWAA python is 3.7.
1) Would you know the roadmap of AWS concerning the updates of Airflow and Python?
2) Would you know how to run my airflow tasks in a 3.9 environment (I read it could be possible using docker, but I don't find any tutorial to do this).
Thanks a lot for your help!

Answer

For the tasks that will be running inside a container, you can utilize the ecs operator to execute a task on the Amazon Elastic Container Service (ECS).   We have some sample information to assist you in connection to the Amazon ECS utilizing the ECSOperator:
https://docs.aws.amazon.com/mwaa/latest/userguide/samples-ecs-operator.html

This link contains the information on how to host your container within Amazon ECS and also the example DAG for invoking the ECS Operator.

Answer

Ok thanks. Finally we are going to downgrade our packages and to unuse Dynamic Task Mapping, which is not available in Airflow 2.2.2.
Would you have information on the timeline for versions upgrades or Python and Airflow?
Thanks a lot!

MWAA legacy airflow and python versions

相關內容