2 réponses
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
To provide custom Python modules you need to put the module(s) in a zip or whl and then add it to the job as extra python files.
Verify the module is correctly defined with a init.py file and you can import it locally.
0
- Create a folder with Name zip_folder
- place your python file in it , lets assume the name of the file is r_file.py -> So your folder structure now is zip_folder/r_file.py
- Zip the folder and upload it to a s3 bucket , zipped folder name is now zip_folder.zip
- In the Python library path setting of Glue jobs , give the path of the zip file , eg : s3://bucket/prefix/zip_folder.zip
- In your glue script , import r_file ( this should be the same name as your .py file name )
Some More details : https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue20-modules-provided
répondu il y a 8 mois
Contenus pertinents
- demandé il y a un an
- demandé il y a 7 jours
- demandé il y a 7 mois
- AWS OFFICIELA mis à jour il y a 4 mois
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 9 mois
- AWS OFFICIELA mis à jour il y a 2 ans
Same thing I followed what you are mentioned in your answer, I have mentioned my custom wheel package in extra python files only. I noticed redshift connector installed successfully message in Jenkins pipeline which I have added that redshift package in --additional-python-modules. But it not consider the redshift_connector import statement. One thing I need to share you, redshift connector is used in one of python file in custom package. Please let me know the changes if any.