1 Antwort
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
0
Hello,
I would like to inform DynamicFrame is similar to a DataFrame, except that each record is self-describing, so no schema is required initially. Instead, AWS Glue computes a schema on-the-fly when required. Basically Glue DynamicFrame is based on RDD due to which show() method does not work directly and you need to convert dynamic frame to dataframe first to check the data in tabular format.
dyf.printSchema()
dyf.toDF().show()
beantwortet vor 2 Jahren
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 3 Jahren
Converting the Glue DynamicFrame to a Spark DataFrame and using the show method is from my point of view a workaround. As you can see in the AWS Documentation, Glue DynamicFrames are supposed to have a show method as well: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame.html#aws-glue-api-crawler-pyspark-extensions-dynamic-frame-show
But this method does not work so this seems to be a bug. Will AWS provide a fix for that?