403 Access denied error from S3 in Glue

Question

I have some data that I've uploaded to an S3 bucket in CSV format. I decided to try crawling it with Glue. I added a crawler, pointed it to the bucket, and gave it an IAM role that has the AWSGLueServiceRole managed policy.

I ran the crawler and it didn't identify any tables, even though I know I have several CSV files in the bucket. I checked CloudWatch logs and the crawler is getting a 403 access denied error from S3:

>  ERROR : Error Access Denied
> (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;
> Request ID: 4A6CA0B1723E03DC) retrieving file at
> s3://data-lake-us-east-1-930604210392/rJEzO-5Jb/1493993509425/Unemployment_Insurance_Recipients_and_UI_Benefit_Payments_by_County__Monthly_.csv.
> Tables created did not infer schemas from this file.

It looks to me like the AWSGlueServiceRole managed policy should give it plenty of access to S3. What else can be going on?

rridgley · Accepted Answer

The IAM Role will need access to the S3 bucket it cannot access. The AWSGlueServiceRole policy only gives the ability to GetObject from resources names aws-glue-*. You will need to attach another policy or create your own policy to run the glue jobs on that S3 bucket.

403 Access denied error from S3 in Glue

Relevant content