Customer is using Glue to parse a CSV file.
The issue happens when trying to load a CSV file and one of its text column EQUIPMENT_DESCRIPTION has “,” (Comma) value in it. This column value is identified with text qualifier (double quotes). But the way we currently have our crawler/glue configured, we are not able to parse the column with the text qualifier to ignore comma values in it.
This is causing all the other columns following EQUIPMENT_DESCRIPTION, to be filled with incorrect data including the DATE_COLLECTED column.
How do you configure Glue to ignore this comma?