How to escape a comma in a csv file in AWS Glue?
Customer is using Glue to parse a CSV file.
The issue happens when trying to load a CSV file and one of its text column EQUIPMENT_DESCRIPTION has “,” (Comma) value in it. This column value is identified with text qualifier (double quotes). But the way we currently have our crawler/glue configured, we are not able to parse the column with the text qualifier to ignore comma values in it.
This is causing all the other columns following EQUIPMENT_DESCRIPTION, to be filled with incorrect data including the DATE_COLLECTED column.
How do you configure Glue to ignore this comma?
- Newest
- Most votes
- Most comments
You should confirm that the quote character is configured appropriately in the SerDe properties of the table?
Have a look at Working with CSV Files in Best Practices When Using Athena With AWS Glue.
Relevant content
- asked 5 years ago
- asked 3 years ago
