How to specify the AWS Credentials seperately for the S3 Table Rest Connect?

Question

Hi,

We are planning to use S3 table for storing our clients' data but we want to have RBAC-based feature so that we can make sure that data is access based on the permission. We are planning to create separate credential for a separate client but how can we specify the AWS credentials in the S3 table catalog rest API while doing the connection? Like separately specifying the AWS credentials in the catalog for example as below:

```
catalog = load_catalog(
    "client1Data",
    **{
        "type": "rest",    
        "warehouse":"arn:aws:s3tables:ap-south-1:{accountId}:bucket/{bucketname}",
        "uri": "https://s3tables.ap-south-1.amazonaws.com/iceberg",
        "rest.sigv4-enabled": "true",
        "rest.signing-name": "s3tables",
        "rest.signing-region": "ap-south-1",
        "s3.access-key-id": "{accessKey}",
        "s3.secret-access-key": "{accessSecret}",
        's3.region': 'ap-south-1'
    }
)
```

The above one is not working. Is there any such provision?

re:Post Agent · Answer

To specify AWS credentials separately for S3 Table connections, you have a few options:

When using Amazon RDS for PostgreSQL with S3 imports, you can provide credentials directly in the function call using the `credentials` parameter. For example:

```
SELECT aws_s3.table_import_from_s3(
't', '', '(format csv)',
's3_uri', 
aws_commons.create_aws_credentials('sample_access_key', 'sample_secret_key', '')
);
```

For your specific use case with S3 Tables REST API, the correct way to specify credentials would be:

```
catalog = load_catalog(
"client1Data",
**{
"type": "rest",    
"warehouse":"arn:aws:s3tables:ap-south-1:{accountId}:bucket/{bucketname}",
"uri": "https://s3tables.ap-south-1.amazonaws.com/iceberg",
"rest.sigv4-enabled": "true",
"rest.signing-name": "s3tables",
"rest.signing-region": "ap-south-1",
"aws.access-key-id": "{accessKey}",
"aws.secret-access-key": "{accessSecret}",
's3.region': 'ap-south-1'
}
)
```

Note that the key difference is using `aws.access-key-id` and `aws.secret-access-key` instead of the `s3.` prefix you were using.

For implementing RBAC-based access, you should:

1. Create separate IAM users for each client with appropriate permissions
2. Generate access keys for each IAM user
3. Use these credentials in your catalog configuration as shown above
4. Consider implementing additional security measures like restricting access based on source IP or VPC endpoints

Remember that storing access keys directly in code is not a best practice for production environments. Consider using a secure credential management system or AWS IAM Roles where possible.  
**Sources**  
[Setting up access to an Amazon S3 bucket - Amazon Relational Database Service](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.S3Import.AccessPermission.html)  
[vpc endpoint interface and S3 | AWS re:Post](https://repost.aws/questions/QUH7OirDoXQtCi-ZDnJINuMQ/vpc-endpoint-interface-and-s3)

How to specify the AWS Credentials seperately for the S3 Table Rest Connect?

Add your answer

Relevant content