MWAA environment variables are not accessible in requirements.txt?

0

Hi,

when trying to install private python package as a dependency via requirements.txt, it looks like usual mwaa environment variables are not accessible in the requirements.txt.

My requirements.txt file (does not work):
--extra-index-url=https://${AIRFLOW__GEMFURY__TOKEN}:@pypi.fury.io/myusername/
myprivatepackage==0.2

However, if I don't use environment variables inside requirements.txt file, everything works fine, dependencies are installed:
--extra-index-url=https://privatetoken:@pypi.fury.io/myusername/
myprivatepackage==0.2

One more thing to mention, if I try to print out this environment variable inside the DAG – it is accessible and has the defined value, without any problems.

Example how to use mwaa environment variables inside requirements.txt file is here –https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html#best-practices-dependencies-different-ways
(Option three: Python dependencies hosted on a private PyPi/PEP-503 Compliant Repo)

Does anyone have/had similar issue?

1개 답변
0

From version Airflow 2.0 this problem no longer exist!

답변함 3년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠