Can AWS Glue crawlers crawl parquet files to iceberg type table?

0

We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?

duwan
asked 5 months ago560 views
2 Answers
1
Accepted Answer

The crawler doesn't do any conversion on the data, it registers what you have there.
So if you just have parquet files and not an Iceberg table, the crawler cannot do that. What the crawler can do is register an Iceberg table in S3 you already have.
Iceberg has utilities to create a table based on existing files, then you can ask the crawler to register it:
https://iceberg.apache.org/docs/latest/spark-procedures/#table-migration

profile pictureAWS
EXPERT
answered 5 months ago
0
profile picture
EXPERT
answered 5 months ago
  • Notice the question says "some parquet files", there is no Iceberg table for the crawler to find

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions