MWAA environment variables are not accessible in requirements.txt?

0

Hi,

when trying to install private python package as a dependency via requirements.txt, it looks like usual mwaa environment variables are not accessible in the requirements.txt.

My requirements.txt file (does not work):
--extra-index-url=https://${AIRFLOW__GEMFURY__TOKEN}:@pypi.fury.io/myusername/
myprivatepackage==0.2

However, if I don't use environment variables inside requirements.txt file, everything works fine, dependencies are installed:
--extra-index-url=https://privatetoken:@pypi.fury.io/myusername/
myprivatepackage==0.2

One more thing to mention, if I try to print out this environment variable inside the DAG – it is accessible and has the defined value, without any problems.

Example how to use mwaa environment variables inside requirements.txt file is here –https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html#best-practices-dependencies-different-ways
(Option three: Python dependencies hosted on a private PyPi/PEP-503 Compliant Repo)

Does anyone have/had similar issue?

demandé il y a 3 ans938 vues
1 réponse
0

From version Airflow 2.0 this problem no longer exist!

répondu il y a 3 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions