1 Antwort
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
0
Glue shell (unlike Glue ETL) doesn't allow installing packages from s3 directly, but what you can do is create a Pypi repo in s3 with the things you need and ask the shell to use it instead of the internet one.
To do that you need:
- Create a repo on s3 with your dependencies, for instance using https://github.com/wolever/pip2pi
- In the --additional-python-modules parameter you can specify pip flags, so you can use -i to point to the repository index on s3
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
Hi Gonzalo, Thanks for your revert. Could you add more details on the step, I tried --additional-python-modules where I have passed pypi libraries like keras==2.15.0 etc and also used --python-modules-installer-option" : "--no-index --find-links=" but this didn't work either. I had all my wheel files present inside S3 in a package folder.
I think that --python-modules-installer-option is for Glue ETL, try adding the flags directly inside the --additional-python-modules . Check the logs to see if it picks it up