2개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
To provide custom Python modules you need to put the module(s) in a zip or whl and then add it to the job as extra python files.
Verify the module is correctly defined with a init.py file and you can import it locally.
0
- Create a folder with Name zip_folder
- place your python file in it , lets assume the name of the file is r_file.py -> So your folder structure now is zip_folder/r_file.py
- Zip the folder and upload it to a s3 bucket , zipped folder name is now zip_folder.zip
- In the Python library path setting of Glue jobs , give the path of the zip file , eg : s3://bucket/prefix/zip_folder.zip
- In your glue script , import r_file ( this should be the same name as your .py file name )
Some More details : https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue20-modules-provided
답변함 9달 전
관련 콘텐츠
- AWS 공식업데이트됨 2년 전
- AWS 공식업데이트됨 일 년 전
- AWS 공식업데이트됨 2년 전
Same thing I followed what you are mentioned in your answer, I have mentioned my custom wheel package in extra python files only. I noticed redshift connector installed successfully message in Jenkins pipeline which I have added that redshift package in --additional-python-modules. But it not consider the redshift_connector import statement. One thing I need to share you, redshift connector is used in one of python file in custom package. Please let me know the changes if any.