S3 Iceberg Table Import into Glue Catalog

To import Iceberg tables from S3 Tables buckets into the AWS Glue Data Catalog using a crawler, you need to understand how to properly reference the S3 Tables paths.

When working with S3 Tables (not general-purpose S3 buckets), you need to use a specific format to reference the table location. For S3 Tables buckets, the warehouse location format is:

<accountid>:s3tablescatalog/<table-bucket-name>

To set up your Glue crawler to access these Iceberg tables:

1. In the AWS Glue console, create a new crawler
2. When configuring the data source, select S3 as the source type
3. For the Include path, you'll need to browse to or enter the path to your Iceberg table data
4. Make sure to select Iceberg as the format when configuring the crawler
5. Create or select an IAM role with appropriate permissions to access both AWS Glue and the S3 Tables bucket
6. Select your target Glue database where the table metadata will be stored
7. Run the crawler to import the table schema into the Glue Data Catalog

If you're having trouble locating the exact path, you can also access S3 Tables using AWS Glue with Spark. In AWS Glue 5.0 and later, there's native integration with S3 Tables Bucket using the AWS analytics services integration.

Make sure your IAM role has the necessary permissions to access both AWS Glue and the S3 Tables bucket. You may need to configure Lake Formation permissions as well to ensure proper access to the table resources.
Sources
Accessing Amazon S3 tables using the AWS Glue Iceberg REST endpoint - Amazon Simple Storage Service
Introducing AWS Glue 5.0 for Apache Spark | AWS Big Data Blog
Community | Redshift Queries Iceberg Tables
Using the Iceberg framework in AWS Glue - AWS Glue

Hi Amit,

Great question. This is one of those subtle areas where understanding how AWS Glue interacts with Iceberg metadata really helps. When you’re dealing with Iceberg tables stored in S3 Tables (not general-purpose S3 buckets), the key is to remember that Glue doesn’t crawl the raw S3 file locations directly for Iceberg; instead, it connects through the Iceberg table’s metadata location. That metadata path points to the Iceberg table definition and schema evolution files, not to the data files themselves.

Here’s how you can identify the correct table path and get your crawler working properly:

Locate the Iceberg Metadata Directory In S3 Tables, each Iceberg table has a metadata directory where the table definition, snapshots, and manifest files are stored. The path typically looks like:

(perl) s3://<your-s3-table-bucket>/<namespace>/<table-name>/metadata/

or in some cases:

(pgsql) s3://<your-s3-table-bucket>/tables/<namespace>/<table-name>/metadata/

The exact structure depends on whether you created the tables via AWS Glue, Lake Formation, or an external engine like Spark or Athena Iceberg.

Use the Table Metadata File (.metadata.json) Inside that metadata folder, you’ll find files ending in .metadata.json. That file represents the active metadata pointer for the Iceberg table. When configuring your Glue Crawler, use the parent folder of this metadata file as the table path.

Confirm the Path in the Catalog (if it exists) If the Iceberg table was previously registered in a Glue or Lake Formation catalog, you can confirm the location property by using the AWS CLI or SDK:

(bash) aws glue get-table --database-name <your_db> --name <your_table>

Look for the StorageDescriptor.Location field in the output. That path is the same one your crawler needs.

Crawler Configuration When setting up your Glue Crawler, choose “Iceberg” as the source type and then provide the S3 path that points to the metadata directory for each table. Make sure the IAM role used by the crawler has permissions to access both the table bucket and any referenced metadata or manifest files.

If You’re Using S3 Tables (Lake Formation Managed) For S3 Tables created through Lake Formation, you can also find the exact path through the Lake Formation console. Navigate to the “Tables” section, select your table, and review the “Location” value. This is the authoritative path for your Iceberg table.

One important note: the S3 Table bucket has a slightly different internal layout than a general-purpose S3 bucket, so you won’t always see the full folder hierarchy when browsing in the S3 console. Using the CLI or the Lake Formation APIs is usually more reliable for confirming metadata paths.

If you’d like, I can share a short Terraform example that automates the creation of an Iceberg-aware Glue Crawler and dynamically picks up Iceberg metadata paths from Lake Formation or Glue. That’s a useful pattern if you’re scaling this across multiple tables or environments. Would that be helpful?