Error reading Kinesis table in Zeppelin

0

My application writes text data to a Kinesis Stream using pipe symbol as field separator. I've defined a table in Glue and specified CSV with the | separator. It's defined as a 3 column table and each column is string. However, when running interactive queries in Zeppelin using %flink.ssql, it chucks the following errors:

Caused by: java.io.IOException: Failed to deserialize CSV row 'DBSERV-1-REPO1|2022-02-24 11:55:34|WARNING: Heavy swapping detected '.

Caused by: java.lang.RuntimeException: Row length mismatch. 3 fields expected but was 1.

If I test writing to Kinesis using comma as separator and set the Glue table as CSV with comma separator, the query works correctly and I can read the data. Do I have to separately instruct the Zeppelin Note when the separator is not a comma?

asked 9 months ago26 views