LakeFormation assumed role cannot access s3 objects during Athena requests ("Permission denied on S3 path" )
I have Delta Lake tables (using Symlink text input format) catalogued in Glue, stored in a S3 bucket, with all its resources tagged with LakeFormation Tags (for tag-based governance). The problem is that, although the users can see the database, tables, and metadata within Athena's catalogue, they cannot perform queries against the specific tables because of "Permission denied on S3 path" errors.
LakeFormation has the data location registered for the datalake bucket, with AWSServiceRoleForLakeFormationDataAccess role. And this role has IAM permissions automatically added to the resources:
LakeFormationDataAccessServiceRolePolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
and
LakeFormationDataAccessPolicyForS3
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "LakeFormationDataAccessPermissionsForS3",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::MYBUCKET/*"
]
},
{
"Sid": "LakeFormationDataAccessPermissionsForS3ListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::MYBUCKET"
]
}
]
}
I have also tried registering the data location with a role with Admin permissions (Action "" and Resources ""), but even so the same error is thrown.
Looking through the CloudTrail logs, I found that LakeFormation passes custom policies to the role when running AssumeRole:
"policy": "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Action\": [\"s3:GetObject\"],\n \"Effect\": \"Allow\",\n \"Resource\": [\"arn:aws:s3:::MYBUCKET\",\n \"Condition\": {\"ForAnyValue:StringLike\":{\"s3:prefix\":[\"MYTABLE/_symlink_format_manifest\",\"MYTABLE/_symlink_format_manifest/*\"]}}\n },\n {\n \"Action\": [\"kms:Decrypt\"],\n \"Effect\": \"Allow\",\n \"Resource\": [\"*\"],\n \"Condition\": {\"StringEquals\":{\"kms:ViaService\":[\"s3.us-east-2.amazonaws.com\"]}}\n } ]\n}"
}
This seems like a malformatted json string that is being passed to the assumed role. Can this be causing the errors I'm having? And does anyone have had this issue before?
PS: I have manually removed ACL control over the S3 bucket and objects. Still same behavior. The error is not shown if I remove the data location, and Athena ignores Lake Formation.
No, it's pointing to a folder with parquet files, and manifests in subfolders.
Just saw in the Lake Formation documentation that it does not support queries that use manifests.
facing the same issue and my steps are exactly the same as yours and being completely stucked with it...
Are your bucket and objects encrypted? If your bucket and objects are encrypted, I would make sure Lakeformation has the proper access to use the KMS Key (KMS Key Policy).
They are encrypted with AWS managed keys. I've added the specific KMS permissions to the access role I'm using, but still ends up denying the access. I've tried adding full KMS access, still nothing.
Relevant questions
AWS Lake Formation: (AccessDeniedException) when calling the GetTable operation: Insufficient Lake Formation permission(s) on table
asked 2 months agoSagemake couldn't access S3 vis athena query
asked 2 months agoAthena query: Insufficient Lake Formation permission(s): Illegal permission combination
asked 4 months agoAccess denied when trying to GET objects uploaded to s3 bucket via aws sdk using cloudfront
asked 5 months agoError Running Glue Crawler
Accepted Answerasked 2 years agoAWS Wrangler Error HIVE_METASTORE_ERROR: Table is missing storage descriptor
asked 2 months agoCross account access from Athena to S3
Accepted Answerasked 2 years ago403 Access denied error from S3 in Glue
Accepted Answerasked 5 years agoWhat permissions configurations are required on an S3 bucket for Athena to be able to Preview View on an object?
asked 2 months agoLakeFormation assumed role cannot access s3 objects during Athena requests ("Permission denied on S3 path" )
asked 3 months ago
Do you mean that you have a table in AWS Lake Formation with the "location" pointing to manifest file(s)?