Error Running Glue Crawler

Question

Hi,

I am trying to catalog some data from S3 as part of Lake Formation setup. I am using a crawler to read a CSV file from an S3 bucket. The crawler is set up to use the default AWSGlueServiceRoleDefault IAM role, which has full access to S3. However, the job keeps failing with the following error:

ERROR : Insufficient Lake Formation permission(s) on s3://<<bucket-name>>/<<file-name>>

(Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 6ca688e0-1b6d-11ea-9499-cb57d11186a3). For more information, see Setting up IAM Permissions in the Developer Guide (http://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html)

As per the documentation, using AWSGlueServiceRoleDefault should be sufficient to execute the crawler. Any ideas why this is failing?

Gareth_E · Accepted Answer

If you have registered the S3 path in Lake Formation, then Lake Formation will be trying to vend temporary credentials to Glue to use to crawl that location, based on the permissions defined in Lake Formation (so not using S3 IAM permissions at all). So you need to GRANT permissions on the database and table (at least select permissions) for AWSGlueServiceRoleDefault ... you do this from the Lake Formation console.

Check this guide to Migrate Permissions to Lake Formation.

Check [this guide](https://docs.aws.amazon.com/lake-formation/latest/dg/upgrade-glue-lake-formation.html) to Migrate Permissions to Lake Formation.

Error Running Glue Crawler

Relevant content