- 최신
- 최다 투표
- 가장 많은 댓글
Hello,
I would like to inform above exception generally occurs when spark is not able to find conditional columns in dataset.
To confirm , I have tested sort and orderBy function in Glue job and it is working absolutely fine. Please find the sample code below:
++++++++++ datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "testdb", table_name = "nycflights13_csv", transformation_ctx = "datasource0")
datasource0.toDF().sort('year','month').show(5)
datasource0.toDF(). orderBy('year','month').show(5) ++++++++++
I would request you please verify schema once again and try to print sample data after creating the dynamic frame and then use sort or orderBy function:
+++++++++ DyanmicFrame.printSchema()
##Above function should print the columns which you would like to use in sort or orderBY
DyanmicFrame.toDF().show()
##Above function should return values
DyanmicFrame.toDF().sort('year','month').show(5)
DyanmicFrame.toDF(). orderBy('year','month').show(5) +++++++++++
If you still face any issue, Please feel free to reach out to AWS Premium Support with sample data and we will be happy to help.
Have a Nice day!
관련 콘텐츠
- AWS 공식업데이트됨 3년 전
- AWS 공식업데이트됨 3달 전
- AWS 공식업데이트됨 일 년 전
Hi @Shubham_P, is there a way to sort() or orderBy() a Dynamic Dataframe avoiding going .toDF() ?
Thanks