1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
Hello,
I would like to inform DynamicFrame is similar to a DataFrame, except that each record is self-describing, so no schema is required initially. Instead, AWS Glue computes a schema on-the-fly when required. Basically Glue DynamicFrame is based on RDD due to which show() method does not work directly and you need to convert dynamic frame to dataframe first to check the data in tabular format.
dyf.printSchema()
dyf.toDF().show()
답변함 2년 전
관련 콘텐츠
- AWS 공식업데이트됨 3년 전
- AWS 공식업데이트됨 일 년 전
Converting the Glue DynamicFrame to a Spark DataFrame and using the show method is from my point of view a workaround. As you can see in the AWS Documentation, Glue DynamicFrames are supposed to have a show method as well: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame.html#aws-glue-api-crawler-pyspark-extensions-dynamic-frame-show
But this method does not work so this seems to be a bug. Will AWS provide a fix for that?