- 最新
- 最多得票
- 最多評論
Hello,
I would like to inform above exception generally occurs when spark is not able to find conditional columns in dataset.
To confirm , I have tested sort and orderBy function in Glue job and it is working absolutely fine. Please find the sample code below:
++++++++++ datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "testdb", table_name = "nycflights13_csv", transformation_ctx = "datasource0")
datasource0.toDF().sort('year','month').show(5)
datasource0.toDF(). orderBy('year','month').show(5) ++++++++++
I would request you please verify schema once again and try to print sample data after creating the dynamic frame and then use sort or orderBy function:
+++++++++ DyanmicFrame.printSchema()
##Above function should print the columns which you would like to use in sort or orderBY
DyanmicFrame.toDF().show()
##Above function should return values
DyanmicFrame.toDF().sort('year','month').show(5)
DyanmicFrame.toDF(). orderBy('year','month').show(5) +++++++++++
If you still face any issue, Please feel free to reach out to AWS Premium Support with sample data and we will be happy to help.
Have a Nice day!
相關內容
- AWS 官方已更新 3 個月前
- AWS 官方已更新 2 年前
- AWS 官方已更新 3 年前
Hi @Shubham_P, is there a way to sort() or orderBy() a Dynamic Dataframe avoiding going .toDF() ?
Thanks