Got error when running VACUUM to removed expired files for iceberg in Athena

3

Error message: ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 1000 files in this round of vacuum, but there are more files remaining. Please run another VACUUM command to process the remaining files

How can I run VACUUM to delete more files, or event all expired files?

已提問 1 年前檢視次數 452 次
3 個答案
0

You can either run a step function in loops in order to VACUUM again and again (https://devopstar.com/2023/07/28/vacuuming-iceberg-with-aws-step-functions/) or you can run a Spark job that expires snapshots and remove orphan files: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html

已回答 9 個月前
  • Are you sure that looping step function is working, I tried with same but still I am getting same error.

0

It would be amazing to get an answer from AWS about this topic.

已回答 1 年前
0

At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the vacuum run. Can anyone tell me what to do ?. we have metadata size around 4.5 gb in icerberg table

Naresh
已回答 3 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南