How to escape a comma in a csv file in AWS Glue?

Question

Customer is using Glue to parse a CSV file.

The issue happens when trying to load a CSV file and one of its text column EQUIPMENT_DESCRIPTION has “,” (Comma) value in it. This column value is identified with text qualifier (double quotes). But the way we currently have our crawler/glue configured, we are not able to parse the column with the text qualifier to ignore comma values in it.  
 
This is causing all the other columns following EQUIPMENT_DESCRIPTION, to be filled with incorrect data including the DATE_COLLECTED column.

How do you configure Glue to ignore this comma?

JDB · Accepted Answer

You should confirm that the quote character is configured appropriately in the SerDe properties of the table?

Have a look at *Working with CSV Files* in [Best Practices When Using Athena With AWS Glue][1].

[1]: https://docs.aws.amazon.com/athena/latest/ug/glue-best-practices.html

How to escape a comma in a csv file in AWS Glue?

Relevant content