I see the exception 403 forbidden error while reading from s3 raw bucket . My bucket will have nested folders , to access that files my spark reader will call recursively that folder files


My exception **** Exception occured while attempting to list the files in the directory and the error message is com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.Amazons3Exception:AccessDenied(Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1A9HE35MJBQGWN; S3 Extended Request ID: y/2NMrtO7HWH+FGOp/wudjoxnJ4SnU+0g6QJINjjMmuCmswaqwKeq/amKjM64au61DWDdm3Rue=;Proxy: null), S3 Extended Request ID: y/2NMrtO7HWH+FGOp/WUdjoxnJ4SnU+0g6QJINjjMmuCmswaqwkeq/amKjM64au61DWDdm3Ruo=

My bucket policy :

	Version": "2012-10-17",
    "Id": "SecureBucketPolicy",
    "Statement": [
						"Sid": "Allow external accounts access and defer to deny statements",
						"Effect": "Allow",
						"Principal": {"AWS". "arn:aws:iam::accountid:root" }
						"Action": {
						"Resource": [
						"Sid". "Lock Down Read to Apps Roles",
					"Effect": "Deny",
					"Principal'. '*'
					'Action": {
					"Resource": "arn.aws:3:.raw/*",
					"Condition": {
					"ForAny Value: Arn NotLike": {
						"Sid". "Lock Down List to Apps Roles and ESF"
						"Effect": "Deny',
						"Principal": "*"
						"Action": [
						"Resource": "arn:aws:s3.:rawbucket'
						"Condition": {
						"ForAnyValue:Arn NotEquals": {
						"aws:PrincipalArn": [
						"ForAnyValue:StringNotLike": {
						"aws:PrincipalArn": [
						"arn:aws:iam:: accountid:role/globalaccess/*!
						"arn:aws.iam: accountid::user/globalaccess/*"

  • Hi, your bucket policy is a mess and really hard to interpret. Can you copy and paste your current bucket policy from S3 bucket permissions page?

  • I have updated the bucket policy , let me know or any guidance how to overcome the exception

  • Did you really copy & paste this from your actual bucket policy? The bucket policy is not even a valid JSON...

    First of all you need to learn the correct policy syntax in IAM User Guide and S3 User Guide.

asked 10 months ago440 views
1 Answer

Still the s3 bucket policy is difficult to interpret.. Json format is not in correct format and s3 arn under the resources do not looks good: such as "Resource": "arn.aws:3:.raw/*" (which seems incomplete and incorrect)

Generally 403 forbidden error in Amazon s3 occurs due to various reason:

-> If your bucket is encrypted and IAM users/roles do not have permission to use KMS (AWS Key management service) key.

-> If there is explicit deny in the bucket policy to specific IAM users or roles.

-> If certain permission are missing that users needs to perform for example, the bucket policy doesn’t include permission to the s3:PutObject action. If the IAM user tries to upload objects, then the user gets an Access Denied error.

-> AWS organizations service control policy doesn't allow access to Amazon s3..

-> Other policy error :

        - Check that there aren’t any extra spaces/special character or incorrect ARNs in the bucket policy or IAM user policies.
                 For example, if an IAM policy has an extra space in the Amazon Resource Name (ARN) as follows: arn:aws:s3::: DOC-EXAMPLE-BUCKET/*. In this case, the ARN is then incorrectly evaluated as arn:aws:s3:::%20DOC-EXAMPLE-BUCKET/ and gives the IAM user an access denied error.

There are several AWS public document on this. You can refers to below document and see if this helps..




answered 10 months ago
  • I’m trying to fetch the files from nested folders how the policy should be ?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions