PyIceberg in AWS Lambda gets 403 on S3 Tables backing bucket — Glue Iceberg REST returns no vended S3 credentials

Is Lake Formation credential vending through the Glue Iceberg REST endpoint (https://glue.<region>.amazonaws.com/iceberg) supported for AWS Lambda execution roles when reading and writing S3 Tables?

• If yes, what configuration enables vending?
• If no, what is the recommended PyIceberg-in-Lambda pattern for federated S3 Tables catalogs?

Symptom

PyIceberg 0.11.1 in a Lambda function calls catalog.load_table(...) against a federated S3 Tables catalog. The call succeeds, but LoadTableResponse.config contains no S3 credentials: s3.access-key-id, s3.secret-access-key, and s3.session-token are all absent from table.io.properties.

The catalog's advertised endpoint list also omits:

• POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan
• POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/credentials

PyArrow then falls back to the Lambda role's default credential chain, and HeadObject on the <uuid>--table-s3 backing bucket returns 403 AccessDenied, even with AdministratorAccess attached to the role.

Catalog config used

type                               = rest
uri                                = https://glue.<region>.amazonaws.com/iceberg
warehouse                          = <account>:s3tablescatalog/<bucket-name>
rest.sigv4-enabled                 = true
rest.signing-name                  = glue
rest.signing-region                = <region>
header.X-Iceberg-Access-Delegation = vended-credentials
rest-scan-planning-enabled         = true

Key diagnostic: plain boto3 reproduces the 403

A diagnostic Lambda using the same execution role and VPC config runs:

boto3.client("s3").head_object(
    Bucket="<uuid>--table-s3",
    Key="metadata/<n>.metadata.json",
)

Same key, same minute:

• From a developer laptop under an admin IAM user → 200 OK
• From Lambda under the scoped role → 403 Forbidden

This rules out PyIceberg and PyArrow entirely. The backing bucket appears to admit admin-equivalent account principals but reject scoped IAM roles, in some way beyond s3:*. GetBucketPolicy on <uuid>--table-s3 returns MethodNotAllowed, so the policy is not inspectable.

What is already configured

Lake Formation

• DataLakeAdmins set; AllowFullTableExternalDataAccess: true
• Integrate with AWS analytics services enabled on the table bucket; federated catalog visible
• Backing bucket registered with the Lake Formation service-linked role (LF SLR)
• LF grants to the Lambda role:
  • Database: DESCRIBE, CREATE_TABLE, ALTER, DROP
  • Table (plus TableWildcard): SELECT, INSERT, DELETE, DESCRIBE, ALTER
  • DATA_LOCATION_ACCESS on the registered location
• IAM_ALLOWED_PRINCIPALS ALL revoked on the affected tables and databases

IAM on the Lambda role

• lakeformation:GetDataAccess, GetResourceLFTags, ListLFTags, RegisterResource, DescribeResource
• glue:GetCatalog(s), GetDatabase(s), GetTable(s), Create/Update/Delete Table, partitions, GetUnfilteredTableMetadata, GetUnfilteredPartition(s)Metadata
• s3tables:* on the relevant table bucket, namespaces, and tables
• iam:PassRole on the LF service-linked role with condition iam:PassedToService=lakeformation.amazonaws.com

What has been ruled out

• AmazonS3FullAccess and AdministratorAccess (*:*) on the Lambda role: still 403
• KMS: buckets are SSE-S3, no KMS in the auth path
• Cross-account ownership: all resources in the same account
• S3 Access Grants: instance not configured (AccessGrantsInstanceNotExistsError)
• Explicit credentials in catalog config: passing s3.access-key-id / s3.secret-access-key / s3.session-token resolved from boto3.Session().get_credentials() directly into the catalog config still 403, ruling out PyArrow credential-resolution bugs
• PyIceberg type=glue native catalog: uses boto3.glue.get_table with no vending API; same failure mode
• LF enforcement itself works: Athena against the federated catalog correctly denies SELECT before the LF grant and succeeds after

Specific questions

1. Is LF credential vending through Glue Iceberg REST supported today for arbitrary IAM principals such as Lambda execution roles?
2. If yes, what setting or permission causes LoadTableResponse.config to actually include s3.access-key-id / s3.secret-access-key / s3.session-token?
3. If no, what is the recommended pattern for PyIceberg in Lambda under a scoped role against federated S3 Tables catalogs? Is the only supported access path through Athena, Glue, or EMR?
4. Should the /plan and /credentials endpoints be advertised by Glue Iceberg REST for federated S3 Tables catalogs?

Happy to provide additional diagnostic detail privately if needed.