Error Running Glue Crawler
Hi,
I am trying to catalog some data from S3 as part of Lake Formation setup. I am using a crawler to read a CSV file from an S3 bucket. The crawler is set up to use the default AWSGlueServiceRoleDefault IAM role, which has full access to S3. However, the job keeps failing with the following error:
ERROR : Insufficient Lake Formation permission(s) on s3://<<bucket-name>>/<<file-name>>
(Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 6ca688e0-1b6d-11ea-9499-cb57d11186a3). For more information, see Setting up IAM Permissions in the Developer Guide (http://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html)
As per the documentation, using AWSGlueServiceRoleDefault should be sufficient to execute the crawler. Any ideas why this is failing?
If you have registered the S3 path in Lake Formation, then Lake Formation will be trying to vend temporary credentials to Glue to use to crawl that location, based on the permissions defined in Lake Formation (so not using S3 IAM permissions at all). So you need to GRANT permissions on the database and table (at least select permissions) for AWSGlueServiceRoleDefault ... you do this from the Lake Formation console.
Check this guide to Migrate Permissions to Lake Formation.
Relevant questions
403 Access denied error from S3 in Glue
Accepted Answerasked 5 years agoAWS Glue crawler detecting a .(dot) before header of a csv file
asked 3 years agoAWS Glue crawler
asked a month agoGlue Crawler getting 403 from S3 because "ciphertext refers to a CMK that doesn't exist." (using SSE-S3, not KMS)
Accepted Answerasked 3 months agoHow could we have Glue to get data from csv as String?
Accepted Answerasked 2 months agoError Running Glue Crawler
Accepted Answerasked 3 years agoCan an Glue Crawler use a S3 Lambda Access Point as a data store?
asked 8 days agoAWS Crawler to directly read Delta lake files from S3
asked 5 days agoI need to read S3 data, transform and put into Data Catalog. Should I be using a Crawler?
Accepted Answerasked 4 months agoDelete partitions in Glue Data Catalog using crawler not working.
asked 2 months ago