1 個回答
- 最新
- 最多得票
- 最多評論
0
Glue shell (unlike Glue ETL) doesn't allow installing packages from s3 directly, but what you can do is create a Pypi repo in s3 with the things you need and ask the shell to use it instead of the internet one.
To do that you need:
- Create a repo on s3 with your dependencies, for instance using https://github.com/wolever/pip2pi
- In the --additional-python-modules parameter you can specify pip flags, so you can use -i to point to the repository index on s3
相關內容
- AWS 官方已更新 2 年前
- AWS 官方已更新 2 年前
- AWS 官方已更新 2 年前
- AWS 官方已更新 3 個月前
Hi Gonzalo, Thanks for your revert. Could you add more details on the step, I tried --additional-python-modules where I have passed pypi libraries like keras==2.15.0 etc and also used --python-modules-installer-option" : "--no-index --find-links=" but this didn't work either. I had all my wheel files present inside S3 in a package folder.
I think that --python-modules-installer-option is for Glue ETL, try adding the flags directly inside the --additional-python-modules . Check the logs to see if it picks it up