Can AWS Glue crawlers crawl parquet files to iceberg type table?

0

We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?

duwan
preguntada hace 6 meses590 visualizaciones
2 Respuestas
1
Respuesta aceptada

The crawler doesn't do any conversion on the data, it registers what you have there.
So if you just have parquet files and not an Iceberg table, the crawler cannot do that. What the crawler can do is register an Iceberg table in S3 you already have.
Iceberg has utilities to create a table based on existing files, then you can ask the crawler to register it:
https://iceberg.apache.org/docs/latest/spark-procedures/#table-migration

profile pictureAWS
EXPERTO
respondido hace 6 meses
0
profile picture
EXPERTO
respondido hace 6 meses
  • Notice the question says "some parquet files", there is no Iceberg table for the crawler to find

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas