- Newest
- Most votes
- Most comments
Outbound internet access was removed from the private web server option.
If you wish to continue to install requirements from public repositories such as http://pypi.org on the webserver that is private, you may do so by downloading and packaging Python WHL files in plugins.zip, or changing your webserver to public.
It turns out the issue may have been due to the webserver being in private mode. After switching it to private mode, the package installation succeeded. I think when private network mode is selected for the Airflow webserver, the AWS-managed service VPC in which it runs does not also allow outbound Internet access - hence the connection timeouts
Hi, we are trying to setup MWAA in private mode. Even for us the same issue is happening with python dependency installation. The pypi.org is accessible from other EC2 on the same subnet so there should not be any network/FW issues. Does anyone has any idea what might be wrong or what other configuration may be needed to make this work? Happy to provide additional details if required.
I have the same problem, MWAA woudn't update totally! after updating requrements.txt file. My webserver access is set to public but it doesn't work!
Perhaps we don't need the packages installed on the Airflow Web Server, only on the Workers and Schedulers, so the pip install timeouts on the web server could probably be ignored. Unless we need some Airflow UI plugins installed. Has anyone verified this? Does the Web Server also need all the packages in the requirements.txt file installed? Surely the Scheduler and Workers do, but the Web Server?
Faced the same issue with version 2.2.2 . After investigating further, I found that webserver is using python -m pip install -r requirements.txt. While the rest (worker and scheduler) are using pip install -r requirements.txt.
Relevant content
- asked 2 years ago
- asked 9 months ago
- asked a year ago
AWS OFFICIALUpdated 8 months ago- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
Is there any way to enable this? It seems like a regression to me because in many disconnected environments, folks host their own PyPI repositories. We already have a cordoned off MWAA instance behind an AWS network firewall and are explicitly allowing outbound connection to our private PyPI repository.
Was this mentioned in a changelog somewhere? And if so, was the fact that our previous deployment that had the Private network enabled, but was able to install remote dependencies over the Internet, a security bug?