Crawler Error: User does not have access to target s3://test-bucket-mis6389/evdata/

Question

When other AWS users are trying to access my bucket with their crawler, they are getting the below error.
"Crawler Error:
User does not have access to target s3://test-bucket-mis6389/evdata/"

My s3 bucket contains a data file. So, other AWS users while running the crawler, the get the above error.
This is my bucket policy.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAccessFromMultipleAccounts",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::test-bucket-mis6389/*"
        }
    ]
}

The crawler is attached the role containing with below custom policy and aws managed policy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "kms:Decrypt"
            ],
            "Resource": [
                "arn:aws:s3:::test-bucket-mis6389/evdata/*"
            ]
        }
    ]
}
and aws managed policy:
AWSGlueServiceRole

Answer

Hi,
Assuming your crawler and s3 bucket are in the same region!

Your crawler attached role has some issues i believe.

The crawler attached role should contain with below custom policy and aws managed policy: 
```
{ "Version": "2012-10-17", "Statement": [ { "Effect": "AllowS3Actions", "Action": [ "s3:GetObject", "s3:PutObject" ], "Resource": [ "arn:aws:s3:::test-bucket-mis6389/evdata/*" ] } ,  { "Effect": "AllowKMSAction", "Action": [ "kms:Decrypt" ], "Resource": [ "" ] ] } 
```
and aws managed policy: AWSGlueServiceRole

You have given s3 actions and KMS action in single statement but in resources section you have only mentioned s3 bucket ARN. You need to also specify KMS Decrypt action on particular KMS key. As a security best practise, i have segregated them into 2 separate statement blocks.

Also, make sure that if users belonging to other account, the KMS key trust policy should trust the other account.

You may also refer this for additional checks https://repost.aws/knowledge-center/s3-cross-account-access-denied

Answer

Hello,

Modify your bucket policy to explicitly grant access also to the evdata folder and its contents.
arn:aws:s3:::test-bucket-mis6389/evdata/*

Answer

Would you able to check the CloudTrail logs? You can filter them by the resource and the name of the S3 bucket. This would provide us with detailed information about the API calls made to S3 and any associated error messages

Answer

Hi,

Are the AWS users belong to the same account or different account? If the bucket is owned by a different AWS account than the one where the crawler is running, you may need to update the bucket policy to grant cross-account access. Check also if there are any other policies attached to the IAM role or the AWS account that might be denying access to the S3 bucket.

Answer

Thank you for the response Paul. But, It's not working.

Crawler Error: User does not have access to target s3://test-bucket-mis6389/evdata/

Relevant content