A quick-startup guide to connect DuckDB Shell to Amazon S3 Table Bucket

Introduction

Amazon S3 Tables provides fully managed Apache Iceberg tables on S3. One of the most powerful aspects of open table formats like Iceberg is engine independence - your data stays in place, and you bring any compatible engine to it. No data copying, no format conversion, no lock-in.

In this post, I'll show you how to connect DuckDB ,a fast, open-source analytical SQL engine to your Amazon S3 Tables bucket and query your data directly from a browser tab. No EC2, no install, no AWS additional compute cost. Just a browser, Amazon S3 Table bucket, temporary credentials, and SQL.

Why DuckDB?

DuckDB is:

• Its open source (MIT licensed) and rapidly evolving
• Portable - runs on your laptop, in Python/R notebooks, inside CI/CD pipelines, and in the browser via WebAssembly
• Iceberg-native - built-in support for Iceberg REST Catalogs, including Amazon S3 Tables
• Instant - no cluster, no cold start, query runs the moment you hit enter

DuckDB's WebAssembly module (shell.duckdb.org) runs a full SQL engine inside your browser. All computation is local,your credentials are only sent to the S3 Tables API endpoint, not to any third party.

Prerequisites

• An AWS account with an S3 Tables bucket containing at least one table
• Access to AWS CloudShell (or any terminal with AWS CLI configured)
• A modern browser (Chrome, Firefox, Safari, Edge)
• We chose us-east-1 as the region in command samples, but this can be any region where your Amazon S3 Table bucket resides.

Disclaimer This article demonstrates the capability of connecting DuckDB to S3 Tables (from a browser) - it is not a recommendation for production access patterns. Generating temporary credentials and using them in a browser-based tool involves inherent risks (credential exposure in browser memory, shared workstations, etc.). For production workflows, install DuckDB locally on your machine, use IAM roles on EC2, or use PROVIDER credential_chain with your existing AWS CLI profile. The browser approach shown here is ideal for quick exploration, demos, and learning, not for handling sensitive production data at scale.

Step 1: Create a scoped IAM policy

Create a least-privilege read-only policy granting only the S3 Tables permissions DuckDB needs to connect and query. No broad s3:* access required - the S3 Tables API handles data-plane access internally.

In the IAM console → Policies → Create Policy, switch to the JSON tab and paste:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3TablesReadOnly",
      "Effect": "Allow",
      "Action": [
        "s3tables:GetTableBucket",
        "s3tables:ListNamespaces",
        "s3tables:ListTables",
        "s3tables:GetTable",
        "s3tables:GetTableMetadataLocation",
        "s3tables:GetTableData",
        "s3tables:ListTableBuckets"
      ],
      "Resource": [
        "arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/<TABLE_BUCKET_NAME>",
        "arn:aws:s3tables:<REGION>:<ACCOUNT_ID>:bucket/<TABLE_BUCKET_NAME>/*"
      ]
    }
  ]
}

Replace <REGION>, <ACCOUNT_ID>, and <TABLE_BUCKET_NAME> with your values (That would be the S3 Tables bucket ARN). Name the policy S3Tables-DuckDB-ReadOnly.

Tip: You can find your table bucket ARN in the S3 console → Table buckets - click your table bucket and copy the ARN from the info panel. The ARN format is arn:aws:s3tables:<region>:<account-id>:bucket/<bucket-name>.

Why these specific actions? They are the minimum required to connect via the Iceberg REST Catalog, list tables, and read data. No write permissions included.

Step 2: Create an IAM role

Create a role that you (or your team) can assume to get temporary credentials:

1. IAM console → Roles → Create Role
2. Trusted entity type: AWS account → This account
3. Attach the policy: S3Tables-DuckDB-ReadOnly
4. Name it: DuckDBRole

The trust policy ("this account") means any authenticated IAM identity in your account can assume this role if they have sts:AssumeRole permission.

Step 3: Generate temporary credentials

In CloudShell (or your terminal), run this script to assume the role and extract credentials cleanly:

CREDS=$(aws sts assume-role \
  --role-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text --no-cli-pager):role/DuckDBRole \
  --role-session-name duckdb \
  --duration-seconds 3600 \
  --no-cli-pager --output json)

export KEY_ID=$(echo "$CREDS" | python3 -c "import sys,json;print(json.load(sys.stdin)['Credentials']['AccessKeyId'])")
export SECRET_KEY=$(echo "$CREDS" | python3 -c "import sys,json;print(json.load(sys.stdin)['Credentials']['SecretAccessKey'])")
export SESSION_TOKEN=$(echo "$CREDS" | python3 -c "import sys,json;print(json.load(sys.stdin)['Credentials']['SessionToken'])")

echo "KEY_ID=$KEY_ID"
echo "SECRET_KEY length=${#SECRET_KEY}"
echo "TOKEN length=${#SESSION_TOKEN}"

Expected output would look like this:

KEY_ID=AKIAIOSFODNN7EXAMPLE
SECRET_KEY length=40
TOKEN length=736

Verify: KEY_ID starts with A*IA, SECRET_KEY is 40 chars, TOKEN is ~736 chars.

Now generate the SQL commands you'll paste into DuckDB:

# Generate the CREATE SECRET command
echo "CREATE SECRET (TYPE S3, KEY_ID '${KEY_ID}', SECRET '${SECRET_KEY}', SESSION_TOKEN '${SESSION_TOKEN}', REGION 'us-east-1');"


This will connect the DuckDB Shell to you provisioned role credentials.

# Generate the ATTACH command
echo "ATTACH '<TABLE_BUCKET_ARN>' AS my_tables (TYPE iceberg, ENDPOINT_TYPE s3_tables);"

This will connect DuckDB to the specific table bucket.

Expected output:

CREATE SECRET (TYPE S3, KEY_ID 'AKIAIOSFODNN7EXAMPLE', SECRET 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY+/', SESSION_TOKEN 'IQoJb3JpZ2lu...< ~736 characters >...Wg==', REGION 'us-east-1');

ATTACH 'arn:aws:s3tables:us-east-1:123456789012:bucket/amzn-s3-demo-table-bucket' AS my_tables (TYPE iceberg, ENDPOINT_TYPE s3_tables);

Follow below, to copy each line and paste into the DuckDB Shell one at a time.

Why this approach? The session token is ~736 characters. Manually copy-pasting from JSON is extremely error-prone - a single missing character causes authentication failure. This script eliminates that risk of typos.

Step 4: Open DuckDB in your browser

Navigate to https://shell.duckdb.org - DuckDB loads as a WebAssembly module. No install, no sign-up.

Step 5: Connect to S3 Tables

Paste each command one at a time into the DuckDB Shell:

Load extensions

INSTALL httpfs;
LOAD httpfs;
INSTALL iceberg;
LOAD iceberg;

httpfs provides S3/HTTP networking. iceberg adds Iceberg REST Catalog support. Order matters — httpfs registers the S3 secret type.

Create the secret (paste from Step 3 output)

CREATE SECRET (TYPE S3, KEY_ID 'AKIA...', SECRET '...', SESSION_TOKEN '...', REGION 'us-east-1');

Note: TYPE S3 configures credentials (how DuckDB authenticates). This is NOT the catalog type.

Attach your table bucket (paste from Step 3 output)

ATTACH '<TABLE_BUCKET_ARN>' AS my_tables (TYPE iceberg, ENDPOINT_TYPE s3_tables);

Note: TYPE iceberg configures the catalog protocol (how DuckDB talks to the Iceberg REST API). This is NOT the credential type.

Step 6: Query your data

SHOW ALL TABLES;

Expected output:

┌───────────┬──────────┬─────────────────┬──────────────┬──────────────┬───────────┐
│ database  │  schema  │      name       │ column_names │ column_types │ temporary │
├───────────┼──────────┼─────────────────┼──────────────┼──────────────┼───────────┤
│ my_tables │ my_schema │ sensor_readings │ [...]        │ [...]        │ false     │
└───────────┴──────────┴─────────────────┴──────────────┴──────────────┴───────────┘

Then query:

SELECT * FROM my_tables.<namespace>.<table_name> LIMIT 10;

Example output:

┌──────────────┬─────────────────────┬───────────────┬──────┬─────────┬─────────────────┬──────────────┬─────────────┬─────────────┬─────────────┬────────┬──────┐
│  event_id    │     timestamp       │ datacenter_id │ hall │ rack_id │ compute_unit_id │ compute_type │   service   │  sensor_id  │ sensor_type │ value  │ unit │
├──────────────┼─────────────────────┼───────────────┼──────┼─────────┼─────────────────┼──────────────┼─────────────┼─────────────┼─────────────┼────────┼──────┤
│ evt-a1b2c3d4 │ 2026-08-12 14:05:00 │ dc-eu-west-1  │ H-03 │ R-12    │ CU-0847         │ gpu-training │ ml-platform │ sensor-2841 │ gpu_temp    │  72.4  │ °C   │
│ evt-e5f6a7b8 │ 2026-08-12 14:05:00 │ dc-us-east-1  │ H-01 │ R-05    │ CU-0192         │ cpu-general  │ web-serving │ sensor-0441 │ cpu_temp    │  58.1  │ °C   │
│ ...          │ ...                 │ ...           │ ...  │ ...     │ ...             │ ...          │ ...         │ ...         │ ...         │  ...   │ ...  │
└──────────────┴─────────────────────┴───────────────┴──────┴─────────┴─────────────────┴──────────────┴─────────────┴─────────────┴─────────────┴────────┴──────┘

You're now querying yout managed Iceberg table on S3 Tables - from your browser, using temporary scoped credentials.

Performance considerations

DuckDB WASM runs entirely in your browser and fetches data over HTTPS. It's excellent for:

• Schema exploration (SHOW ALL TABLES, metadata queries)
• Filtered lookups with LIMIT (partition pruning keeps it fast)
• Query plan analysis (EXPLAIN)
• Small-to-moderate queries on focused subsets

For full-table aggregations across hundreds of millions of rows, use Athena or install DuckDB locally on your laptop or EC2. The browser WASM module has limited memory and network bandwidth.

First query latency: The first query can take up to ~20–30 seconds (downloading Iceberg metadata over HTTPS). Subsequent queries should be significantly faster as metadata is cached.

What we demonstrated

In this walkthrough, we connected an open-source SQL engine to a fully managed Iceberg table on S3 Tables without provisioning a single resource. No EC2, no cluster, no server. Just a browser tab, a scoped IAM role, and standard SQL.

This is what open table formats on managed infrastructure are all about: the data stays in place, and any compatible engine can read it. What we showed with DuckDB, here, is very similar to the way you'de integrate with Spark, Trino, Flink, Redshift Spectrum, or any engine that speaks the Iceberg (REST) Catalog protocol. No data copying. No format conversion.

About DuckDB

DuckDB is developed by DuckDB Labs and released under the MIT license . It runs as a single binary with no external dependencies, making it easy to embed in applications, notebooks, CI/CD pipelines, or as we showed here, a browser via WebAssembly. You can install DuckDB locally on macOS, Linux, or Windows for a faster experience with larger datasets.

Cleanup

You may want to keep the IAM role and policy, if you plan to reuse them, they're scoped to one table bucket and only useful with temporary credentials.

If you want to remove them:

aws iam detach-role-policy --role-name DuckDBRole --policy-arn arn:aws:iam::<ACCOUNT_ID>:policy/S3Tables-DuckDB-ReadOnly
aws iam delete-role --role-name DuckDBRole
aws iam delete-policy --policy-arn arn:aws:iam::<ACCOUNT_ID>:policy/S3Tables-DuckDB-ReadOnly

References

• Amazon S3 Tables documentation
• DuckDB Iceberg extension - S3 Tables
• DuckDB - "Iceberg in the Browser" (Dec 2025)
• AWS Blog - Streamlining access to S3 Tables with DuckDB
• DuckDB Shell (WebAssembly)

---

Ran Pergamin is a Senior Specialist Solutions Architect for Storage & Data at AWS.