- Newest
- Most votes
- Most comments
Hi ajt,
It sounds like there might be an issue with the permissions for accessing the data sources after adding the IAM pass role permission. Double-check that the IAM role assigned to the Glue job has the necessary permissions to access the S3 buckets where the CSV files are located. Also, ensure that the IAM policies attached to the role grant appropriate permissions for Glue to read from S3.
You can see the instructions Here to set up AWS Identity and Access Management (IAM) permissions for AWS Glue.
I hope it helps.
Notice the list operation is done in the bucket, not the prefix, so you need to add the resource with and without prefix wildcard: : ["arn:aws:s3:::MY-BUCKET-NAME", "arn:aws:s3:::MY-BUCKET-NAME/*"]
Yes, that was an example/template. I added my resource/bucket name. But yeah, regardless, the inline policy shouldn't be required because it's superseded by "AmazonS3FullAccess".
It's a very basic project, so this is extremely perplexing.
UPDATE: Actually, I'm not entirely sure what you were saying. I went by this documentation > https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_resource.html.
It states the following below.
The following example refers to all items within a specific Amazon S3 bucket. "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
So I'm not sure why I would need to add "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/" AND "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET"? I would have access to all the items within that one bucket with "/"?
But again, not sure why any of this would matter if I am using "AmazonS3FullAccess" to begin with?
I should now rephrase the question. Would anyone know why I'm having issues with the join data population on joins (no keys or data preview) when I have "AmazonS3FullAccess"?
The inline policy (allowing bucket access) seems to be redundant for no reason.
The policies used are below.
USER POLICIES:
- AmazonAthenaFullAccess
- AmazonS3FullAccess
- AWSGlueConsoleFullAccess
- AWSQuicksightAthenaAccess
- AWSQuickSightDescribeRDS
- IAMFullAccess
- Inline policy for access to specific buckets (shouldn't be necessary w/ "AmazonS3FullAccess" above)
ROLE POLICIES:
- AmazonS3FullAccess
Relevant content
- Accepted Answerasked a year ago
- Accepted Answerasked 2 years ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated a year ago
Thanks for that. Just a quick question though. For this particular project, others were only adding "AmazonS3FullAccess" and they did not encounter these issues.
But along with "AmazonS3FullAccess", I added "IAMFullAccess". Of course, I already have "AWSGlueConsoleFullAccess" to begin with, as well.
For all intents and purposes for my project, shouldn't the "AmazonS3FullAccess" permissions be fine since it's wide-open access to all S3 buckets?
I'm a bit confused.
Update: So even though the "AmazonS3FullAccess" should suffice, I did add the inline policy (below) to the user, but no luck. I'm really not sure what's going on because it seems pretty straight-forward.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:Get*", "s3:List*", "s3:Object" ], "Resource": [ "arn:aws:s3:::MY-BUCKET-NAME/*" ] } ] }