ModuleNotFoundError: No module named 'airflow.providers.cncf' on MWAA version 2.8.1

0

Hi all, I'm currently working with MWAA service version 2.8.1(Latest) upgraded from 2.7.2, and encountered an issue while trying to utilize the KubernetesPodOperator class in a Python DAG. I'm facing this error: ModuleNotFoundError: No module named 'airflow.providers.cncf'. I'm curious to know if anyone has experienced a similar issue or if there's a known packaging problem with this specific version of MWAA that could be affecting the airflow.providers.cncf module.

Hermes
asked a month ago499 views
1 Answer
0

Hi,

As the above error could be either because of dependency conflicts or requirements and constraints issue, it would be better to use mwaa-local-runner to test the requirements file and plugins [if any]. Please refer the github link[1] for more details on local runner.

For Airflow version 2.8.1, we should pass the requirement as "apache-airflow[cncf-kubernetes]". It would be ideal to also use the constraints file [2] and check if that resolves the issue. You can also check the requirements-install logs from scheduler log group to find the conflicts if any. If you are still facing the issue, please raise a support case[3] with us along with the requirements file, plugins file [if any] and the error message.

Thank you.

[1] https://github.com/aws/aws-mwaa-local-runner

[2] https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags-dependencies.html#working-dags-dependencies-test-create

[3] https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case

AWS
SUPPORT ENGINEER
answered a month ago
profile picture
EXPERT
reviewed a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions