1 Respuesta
- Más nuevo
- Más votos
- Más comentarios
0
Hello,
I would like to inform DynamicFrame is similar to a DataFrame, except that each record is self-describing, so no schema is required initially. Instead, AWS Glue computes a schema on-the-fly when required. Basically Glue DynamicFrame is based on RDD due to which show() method does not work directly and you need to convert dynamic frame to dataframe first to check the data in tabular format.
dyf.printSchema()
dyf.toDF().show()
respondido hace 2 años
Contenido relevante
- OFICIAL DE AWSActualizada hace 2 años
- OFICIAL DE AWSActualizada hace 3 años
- OFICIAL DE AWSActualizada hace 3 años
Converting the Glue DynamicFrame to a Spark DataFrame and using the show method is from my point of view a workaround. As you can see in the AWS Documentation, Glue DynamicFrames are supposed to have a show method as well: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-dynamic-frame.html#aws-glue-api-crawler-pyspark-extensions-dynamic-frame-show
But this method does not work so this seems to be a bug. Will AWS provide a fix for that?