2 Risposte
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
0
- Your UDF file should be at the root level of your zip file , if you are not making a py package .
- Instead of importing like from mylibrary import udf_uppercase , just use import udf_uppercase given that udf_uppercase.py is your UDF file name
- Try to use the Python Library Path text box in the job properties tab. give the full path like : s3://BucketName/utils/utils.zip
Refer : https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html
con risposta un anno fa
0
That should work (assuming that your file is named mylibrary.py) so that import can work.
The issue I see there is that you convert to DataFrame, add a column (still DataFrame) and then try to pass it to write_dynamic_frame which is intended for DynamicFrame.
You should receive an error like this: TypeError: frame_or_dfc must be DynamicFrame orDynamicFrameCollection. Got <class 'pyspark.sql.dataframe.DataFrame'>
BTW, I guess that's just an example, it's much more efficient if you don't use udfs and do upper case using a SQL function.
Contenuto pertinente
- AWS UFFICIALEAggiornata 4 mesi fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 3 anni fa