Glue DynamicFrame show method yields nothing

Question

I'm using a Notebook together with a Glue Dev Endpoint to load data from S3 into a Glue DynamicFrame. The `printSchema` method works fine but the `show` method yields nothing although the dataframe is not empty. Converting the DynamicFrame into a Spark DataFrame actually yields a result (`df.toDF().show()`). Here the dummy code that I'm using

```python
glueContext = GlueContext(spark.sparkContext)

df = glueContext.create_dynamic_frame_from_options(
    connection_type="s3", 
    connection_options = {
        "paths": [f"s3://bucketname/filename"]},
    format="json",
    format_options={"multiline": True}
)

df.printSchema()
df.show()
```
This problem was posted on Stackoverflow before: https://stackoverflow.com/questions/56013334/spark-dynamic-frame-show-method-yields-nothing

Why does the `show` method yield nothing? Is this a bug or is there something that I'm missing to make this work?

Answer

Hello,

I would like to inform DynamicFrame is similar to a DataFrame, except that each record is self-describing, so no schema is required initially. Instead, AWS Glue computes a schema on-the-fly when required. Basically Glue DynamicFrame is based on RDD due to which show() method does not work directly and you need to convert dynamic frame to dataframe first to check the data in tabular format.

```
dyf.printSchema()
dyf.toDF().show()

```

Glue DynamicFrame show method yields nothing

Relevant questions

Sagemaker Notebook from Dev Endpoint

Can an Glue Crawler use a S3 Lambda Access Point as a data store?

Decimal Precision issues when writing from DynamicFrame

How to connect a Sagemaker Notebook to Glue Catalog

Glue DynamicFrame show method yields nothing

same glue job running differently when used sample method

What is the best practice to load data to redshift with aws glue ?

Glue + SageMaker Pip Packages

Cleaning a bucket with purge_s3_path with Glue console.

Aws glue script toDF().sort() method gives exception