- Le plus récent
- Le plus de votes
- La plupart des commentaires
Hello,
I would like to inform above exception generally occurs when spark is not able to find conditional columns in dataset.
To confirm , I have tested sort and orderBy function in Glue job and it is working absolutely fine. Please find the sample code below:
++++++++++ datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "testdb", table_name = "nycflights13_csv", transformation_ctx = "datasource0")
datasource0.toDF().sort('year','month').show(5)
datasource0.toDF(). orderBy('year','month').show(5) ++++++++++
I would request you please verify schema once again and try to print sample data after creating the dynamic frame and then use sort or orderBy function:
+++++++++ DyanmicFrame.printSchema()
##Above function should print the columns which you would like to use in sort or orderBY
DyanmicFrame.toDF().show()
##Above function should return values
DyanmicFrame.toDF().sort('year','month').show(5)
DyanmicFrame.toDF(). orderBy('year','month').show(5) +++++++++++
If you still face any issue, Please feel free to reach out to AWS Premium Support with sample data and we will be happy to help.
Have a Nice day!
Contenus pertinents
- demandé il y a un an
- demandé il y a 2 mois
- demandé il y a un an
- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 3 ans
- AWS OFFICIELA mis à jour il y a un an
Hi @Shubham_P, is there a way to sort() or orderBy() a Dynamic Dataframe avoiding going .toDF() ?
Thanks