Data Preview not showing any outputs, even with "Data Catalog update options" not working
0
Hi team, I have a Glue Job which uses a Data Source - Data Catalog -> Transform - SQL Query -> Data Target - S3 Bucket.
I am able to see the Data Preview for "Transform - SQL Query", but when I try to see the Data Preview for "Data Target - S3 Bucket" it doesn't shows any rows and also no columns.
asked 3 years ago1.5K views
1 Answer
- Newest
- Most votes
- Most comments
0
The Data Preview shows the output of a node, in the case of a sink there is no output.
To see the data that would be stored on the sink, you can see the preview of the node immediately before the sink.
I agree it could be more friendly and just so the input in the case of a sink, will open a suggestion to the team.
Relevant content
- asked 4 years ago
- asked 2 years ago
Not only this, if I use "Data Catalog update option" -> "Create a table in the Data Catalog and on subsequent runs, update the schema and add new partitions", the catalog gets updated, but when you run the query in Athena, it shows blank records...
The weird thing is that the files in S3 does has data, which can be viewed using S3 Select. But the same is not visible via Athena.
if you see rows in Athena but the columns are blank (when you know they have data), the most likely cause is that the files on s3 use a different casing to name the columns, check that just in case.
Can you please explain what do you mean by "files on s3 use a different casing to name the columns"? All the files are generated by the Glue Job as output and as part of the process they are updating the data catalog as well. What should I be looking at or what changes do I need to make.