3 Antworten
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
0
You can either run a step function in loops in order to VACUUM again and again (https://devopstar.com/2023/07/28/vacuuming-iceberg-with-aws-step-functions/) or you can run a Spark job that expires snapshots and remove orphan files: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html
beantwortet vor 9 Monaten
0
At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the vacuum run. Can anyone tell me what to do ?. we have metadata size around 4.5 gb in icerberg table
beantwortet vor 3 Monaten
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
Are you sure that looping step function is working, I tried with same but still I am getting same error.