MWAA legacy airflow and python versions

0

Hi all! I am new to MWAA service. I would like to use it to manage my dags, which I deployed by myself on ec2 instances with airflow running in a docker. It would be much simpler for me to use managed airflow :) But: I already use Dynamic Task Mapping which is from Airflow 2.3 where MWAA Airflow is 2.2 + my packages are python 3.9 where MWAA python is 3.7.

  1. Would you know the roadmap of AWS concerning the updates of Airflow and Python?
  2. Would you know how to run my airflow tasks in a 3.9 environment (I read it could be possible using docker, but I don't find any tutorial to do this). Thanks a lot for your help!
asked 4 months ago32 views
2 Answers
0

For the tasks that will be running inside a container, you can utilize the ecs operator to execute a task on the Amazon Elastic Container Service (ECS). We have some sample information to assist you in connection to the Amazon ECS utilizing the ECSOperator: https://docs.aws.amazon.com/mwaa/latest/userguide/samples-ecs-operator.html

This link contains the information on how to host your container within Amazon ECS and also the example DAG for invoking the ECS Operator.

answered 3 months ago
0

Ok thanks. Finally we are going to downgrade our packages and to unuse Dynamic Task Mapping, which is not available in Airflow 2.2.2. Would you have information on the timeline for versions upgrades or Python and Airflow? Thanks a lot!

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions