Can AWS Glue crawlers crawl parquet files to iceberg type table?

0

We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?

duwan
已提问 6 个月前593 查看次数
2 回答
1
已接受的回答

The crawler doesn't do any conversion on the data, it registers what you have there.
So if you just have parquet files and not an Iceberg table, the crawler cannot do that. What the crawler can do is register an Iceberg table in S3 you already have.
Iceberg has utilities to create a table based on existing files, then you can ask the crawler to register it:
https://iceberg.apache.org/docs/latest/spark-procedures/#table-migration

profile pictureAWS
专家
已回答 6 个月前
0
profile picture
专家
已回答 6 个月前
  • Notice the question says "some parquet files", there is no Iceberg table for the crawler to find

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则