Can AWS Glue crawlers crawl parquet files to iceberg type table?

0

We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?

duwan
feita há 6 meses589 visualizações
2 Respostas
1
Resposta aceita

The crawler doesn't do any conversion on the data, it registers what you have there.
So if you just have parquet files and not an Iceberg table, the crawler cannot do that. What the crawler can do is register an Iceberg table in S3 you already have.
Iceberg has utilities to create a table based on existing files, then you can ask the crawler to register it:
https://iceberg.apache.org/docs/latest/spark-procedures/#table-migration

profile pictureAWS
ESPECIALISTA
respondido há 6 meses
0
profile picture
ESPECIALISTA
respondido há 6 meses
  • Notice the question says "some parquet files", there is no Iceberg table for the crawler to find

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas