MWAA environment variables are not accessible in requirements.txt?

0

Hi,

when trying to install private python package as a dependency via requirements.txt, it looks like usual mwaa environment variables are not accessible in the requirements.txt.

My requirements.txt file (does not work):
--extra-index-url=https://${AIRFLOW__GEMFURY__TOKEN}:@pypi.fury.io/myusername/
myprivatepackage==0.2

However, if I don't use environment variables inside requirements.txt file, everything works fine, dependencies are installed:
--extra-index-url=https://privatetoken:@pypi.fury.io/myusername/
myprivatepackage==0.2

One more thing to mention, if I try to print out this environment variable inside the DAG – it is accessible and has the defined value, without any problems.

Example how to use mwaa environment variables inside requirements.txt file is here –https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html#best-practices-dependencies-different-ways
(Option three: Python dependencies hosted on a private PyPi/PEP-503 Compliant Repo)

Does anyone have/had similar issue?

posta 3 anni fa938 visualizzazioni
1 Risposta
0

From version Airflow 2.0 this problem no longer exist!

con risposta 3 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande