Stay up to date with the latest from the Knowledge Center. See all new Knowledge Center articles published in the last month, and re:Post's top contributors.
How do I create a fixed-width classifier manually for a Glue crawler?
I have a few text files on S3 that I need to add to the Glue Catalog in order to use them in a job. None of them have separators, they are all fixed-width files. I have the schemas, but the crawler naturally fails to recognize them. I tried processing the files using a Lambda function, but since they are too big (one is 11GB), execution times out, even at the maximum available (15min).
I couldn't find any tutorials about dealing with these type of files.
- Newest
- Most votes
- Most comments
Hi,
Go to this article to find a fully explained example: https://medium.com/@avj70380/how-to-use-grok-custom-classifier-to-read-flat-text-files-using-aws-glue-657c7d55bd57
Best,
Didier
Relevant content
- asked 3 years ago
- AWS OFFICIALUpdated 4 months ago
