Hi team,
I have a glue job that read from an S3 CSV file and inject it to DB,
I have the following error while running the job,
I think it's related to the file encoding,
the original file encoding is : ** ISO-8859-1**
if I change manually the file encoding to be UTF-8, the glue job passes.
do I need to have the CSV file encoded in utf-8 to be able to run successfully the job way?
is there any way to go around this?
Thank you!
com.amazonaws.services.glue.util.FatalException: Unable to parse file: myFile.csv\n\n\tat com.amazonaws.services.glue.readers.JacksonReader.hasNextFailSafe(JacksonReader.scala:94)\n\tat com.amazonaws.services.glue.readers.JacksonReader.hasNext(JacksonReader.scala:38)\n\tat com.amazonaws.services.glue.readers.CSVReader.hasNext(CSVReader.scala:169)\n\tat com.amazonaws.services.glue.hadoop.TapeHadoopRecordReaderSplittable.nextKeyValue(TapeHadoopRecordReaderSplittable.scala:97)\n\tat org.apache.spark.rdd.NewHadoopRDD$$anon$1.hasNext(NewHadoopRDD.scala:247)\n\tat org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)\n\tat