LakeFormation assumed role cannot access s3 objects during Athena requests ("Permission denied on S3 path" )

0

I have Delta Lake tables (using Symlink text input format) catalogued in Glue, stored in a S3 bucket, with all its resources tagged with LakeFormation Tags (for tag-based governance). The problem is that, although the users can see the database, tables, and metadata within Athena's catalogue, they cannot perform queries against the specific tables because of "Permission denied on S3 path" errors.

LakeFormation has the data location registered for the datalake bucket, with AWSServiceRoleForLakeFormationDataAccess role. And this role has IAM permissions automatically added to the resources:

LakeFormationDataAccessServiceRolePolicy
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

and

LakeFormationDataAccessPolicyForS3
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LakeFormationDataAccessPermissionsForS3",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::MYBUCKET/*"
            ]
        },
        {
            "Sid": "LakeFormationDataAccessPermissionsForS3ListBucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::MYBUCKET"
            ]
        }
    ]
}

I have also tried registering the data location with a role with Admin permissions (Action "" and Resources ""), but even so the same error is thrown.

Looking through the CloudTrail logs, I found that LakeFormation passes custom policies to the role when running AssumeRole:

"policy": "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n         {\n           \"Action\": [\"s3:GetObject\"],\n           \"Effect\": \"Allow\",\n           \"Resource\": [\"arn:aws:s3:::MYBUCKET\",\n           \"Condition\": {\"ForAnyValue:StringLike\":{\"s3:prefix\":[\"MYTABLE/_symlink_format_manifest\",\"MYTABLE/_symlink_format_manifest/*\"]}}\n         },\n         {\n           \"Action\": [\"kms:Decrypt\"],\n           \"Effect\": \"Allow\",\n           \"Resource\": [\"*\"],\n           \"Condition\": {\"StringEquals\":{\"kms:ViaService\":[\"s3.us-east-2.amazonaws.com\"]}}\n         }  ]\n}"
    }

This seems like a malformatted json string that is being passed to the assumed role. Can this be causing the errors I'm having? And does anyone have had this issue before?

PS: I have manually removed ACL control over the S3 bucket and objects. Still same behavior. The error is not shown if I remove the data location, and Athena ignores Lake Formation.

  • Do you mean that you have a table in AWS Lake Formation with the "location" pointing to manifest file(s)?

  • No, it's pointing to a folder with parquet files, and manifests in subfolders.

  • Just saw in the Lake Formation documentation that it does not support queries that use manifests.

  • facing the same issue and my steps are exactly the same as yours and being completely stucked with it...

asked 2 years ago1998 views
1 Answer
0

Are your bucket and objects encrypted? If your bucket and objects are encrypted, I would make sure Lakeformation has the proper access to use the KMS Key (KMS Key Policy).

jsonc
answered 2 years ago
  • They are encrypted with AWS managed keys. I've added the specific KMS permissions to the access role I'm using, but still ends up denying the access. I've tried adding full KMS access, still nothing.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions