How could we have Glue to get data from csv as String?

0

I have csv data uploaded to S3 bucket, letting Glue to set them up as tables for later use. I want all the columns loaded as string without designating each column name by name. How could we configure Glue to load all columns as string, not transforming into bigint or so?

質問済み 2年前2003ビュー
1回答
0
承認された回答

Hello,

In Glue we use crawlers to automatically detect the schema from file and create a table in Glue catalog. For CSV files, the crawler reads either the first 100 records or the first 1 MB of data, whatever comes first to detect the schema. [1]

Having said that with this approach it is not possible to load all csv columns as string in Glue catalog directly. You can consider two approach for your use case:

  1. Create a crawler and run on csv data. Once it create the table in Glue catalog with correct datatype , you can modify the table schema to string for all columns.

  2. Directly read the data from csv files using Glue ETL job and in applymapping change schema to string and write the table into catalog with enableUpdateCatalog option. [2]

--Reference:

[1] https://aws.amazon.com/premiumsupport/knowledge-center/glue-crawler-detect-schema/ [2] https://docs.aws.amazon.com/glue/latest/dg/update-from-job.html

AWS
回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ