1 Answer
- Newest
- Most votes
- Most comments
0
You shouldn't create Iceberg tables manually, configure the Iceberg library to use the Glue catalog and register a reference when writing the data.
https://iceberg.apache.org/docs/latest/aws/#spark
Iceberg doesn't use standard table partitions, the partitions in Iceberg are handled internally by the metadata (so they are dynamic).
Relevant content
- asked 8 months ago
- asked a year ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated a year ago
There's literally an API to create iceberg tables with glue, and I'm not using spark at all here so I'm not sure how any of this is really relevant.
I believe that API registers an existing table on the catalog but you still need an engine to create the actual table like Spark or Athena, it's easier if you do both things at the same time (create the table on s3 and register it on the catalog)
You don't need to do anything when creating the table via the glue API. You can query the (empty) table, or insert into it like normal. It's a fully functional iceberg table, it's just not clear how to do the call to set it up with partitions like PARTITION BY in athena, which is all the question is about really.