I can't connect to an external s3 bucket as a data source for a glue crawler

Question

I am trying to connect to an external s3 bucket as a data source for a glue crawler. The bucket has the necessary arn permissions as i can connect to it via s3 browser as an external bucket. The crawler has the trust relationship:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CID",
            "Effect": "Allow",
            "Principal": {
                "Service": "glue.amazonaws.com",
                "AWS": "arn:aws:iam:::role/"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Running the crawler, i get the error User does not have access to target s3://.

Are there any reasons why this isn't working?

Thanks

rePost-User-6739329 · Answer

Hello,

Please note that assume role may not work for glue crawler case. In order to ensure you are able to crawler external s3 bucket as a data source you need to ensure the below:
1. Crawler role in Account A should have access to Account B s3 bucket(Get*, List*)
2. Account B s3 bucket must allow required permissions(Get, List etc) to account A crawler role in it's bucket policy.
3. Account B s3 bucket must not be using SSE-KMS(aws/s3) key, if bucket is encrypted with aws/s3 AWS Managed KMS key then cross account s3 access won't work
4. If Account B s3 bucket is SSE-KMS CMK(custom key) encrypted then, KMS key policy in Account B must allow Account A glue crawler role.

Please check on all the above permissions in order to resolve the issue. You can also refer to below repost link as well:
https://repost.aws/questions/QU_8lhusbHSLOg9CE7U30W7w/cross-account-s3-bucket-access-in-aws-glue-crawler

I can't connect to an external s3 bucket as a data source for a glue crawler

Relevant content