By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Glue Crawler and Classifiers - Supported file encodings: is UTF16 supported?

0

Hi, AWS Glue Crawlers with CSV and XML Classifiers and works well with files encoded in UTF-8 but not with file encoded in UTF-16.

Public documentation does not clarify this point:

  • Do Glue crawler and classifier support UTF-16?
  • Is there please an available documentation on supported encodings with Glue crawlers and classifiers?

Best regards,

Nicolas.

Topics
AnalyticsStorage
Tags
AWS GlueData LakesAWS Lake Formation
Language
English
AWS
Nicolas
asked 4 years ago1101 views
1 Answer
0
Accepted Answer

Glue at the moment supports UTF-8 encoded files only [1]. If UTF-16 files are passed in, you may encounter the "Internal Service Exception" error message. The most feasible method would be to programatically convert the utf-16 files to utf-8 before passing it through Glue Crawler.

[1] - https://docs.aws.amazon.com/glue/latest/dg/components-key-concepts.html

AWS
EXPERT
Prabhakar_C
answered 4 years ago
profile picture
EXPERT
Giovanni Lauria
reviewed 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content