S3 Tables Compaction jobs and delete-files/DELETE_FILE_THRESHOLD

In S3 Tables, compaction is enabled by default to improve query performance by combining smaller objects into larger ones. While the documentation for S3 Tables doesn't explicitly mention the DELETE_FILE_THRESHOLD parameter, this is an important consideration for Iceberg table maintenance.

The DELETE_FILE_THRESHOLD parameter specifies the minimum number of delete operations that must be present in a data file before it's included in compaction. By default, Iceberg compaction doesn't combine delete files with data files unless this threshold is met.

Currently, the S3 Tables documentation doesn't indicate a direct way to configure the DELETE_FILE_THRESHOLD parameter for its automatic compaction jobs. The S3 Tables compaction configuration allows you to set parameters like target file size (between 64MB and 512MB) and choose compaction strategies (auto, binpack, sort, or z-order), but doesn't explicitly mention configuration options for handling delete files.

If you need more control over delete file handling in your Iceberg tables, you might consider using other AWS services that provide more granular configuration options for Iceberg compaction:

1. AWS Glue offers an IcebergCompactionConfiguration with a deleteFileThreshold parameter that serves the same purpose as DELETE_FILE_THRESHOLD. This parameter sets the minimum number of deletes that must be present in a data file to make it eligible for compaction, with a default value of 1.

2. Amazon Athena supports manual compaction through the OPTIMIZE command, which can rewrite data files into a more optimized layout based on their size and number of associated delete files.

For S3 Tables specifically, you may need to contact AWS Support for more information about configuring delete file handling in the automatic compaction process if this functionality is critical for your use case.
Sources
Maintaining tables by using compaction - AWS Prescriptive Guidance
IcebergCompactionConfiguration - AWS Glue
Maintenance for tables - Amazon Simple Storage Service
Optimize Iceberg tables - Amazon Athena