By using AWS re:Post, you agree to the Terms of Use
AWS re:Postre:Post

Got error when running VACUUM to removed expired files for iceberg in Athena

2

Error message: ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 1000 files in this round of vacuum, but there are more files remaining. Please run another VACUUM command to process the remaining files

How can I run VACUUM to delete more files, or event all expired files?

Topics
Analytics
Tags
Amazon Athena
Language
English
rePost-User-2280829
asked 4 months ago183 views
2 Answers
0

It would be amazing to get an answer from AWS about this topic.

mattia23
answered 2 months ago
0

You can either run a step function in loops in order to VACUUM again and again (https://devopstar.com/2023/07/28/vacuuming-iceberg-with-aws-step-functions/) or you can run a Spark job that expires snapshots and remove orphan files: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html

Eran Levy
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content