I see the exception 403 forbidden error while reading from s3 raw bucket . My bucket will have nested folders , to access that files my spark reader will call recursively that folder files

Question

My exception ****
Exception occured while attempting to list the files in the directory and the error message is com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.Amazons3Exception:AccessDenied(Service:
Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1A9HE35MJBQGWN; S3 Extended Request ID: y/2NMrtO7HWH+FGOp/wudjoxnJ4SnU+0g6QJINjjMmuCmswaqwKeq/amKjM64au61DWDdm3Rue=;Proxy: null), S3 Extended Request ID:
y/2NMrtO7HWH+FGOp/WUdjoxnJ4SnU+0g6QJINjjMmuCmswaqwkeq/amKjM64au61DWDdm3Ruo=

```
My bucket policy :

{
	Version": "2012-10-17",
    "Id": "SecureBucketPolicy",
    "Statement": [
	                {
						"Sid": "Allow external accounts access and defer to deny statements",
						"Effect": "Allow",
						"Principal": {"AWS". "arn:aws:iam::accountid:root" }
						"Action": {
						"53:ListBucket",
						"s3:GetObject",
						"s3:PutObject"
						}
						"Resource": [
						"arn:aws:s3.::-raw",
						"arn:aws:$3:.:vgi-ics-raw/*"]
					},
					{
						"Sid". "Lock Down Read to Apps Roles",
					"Effect": "Deny",
					"Principal'. '*'
					'Action": {
					"S3:GetObject",
					"S3:GetObiectVersion"
					"S3:PutObiect"
					}
					"Resource": "arn.aws:3:.raw/*",
					"Condition": {
					"ForAny Value: Arn NotLike": {
					"awsPrincipalArn":"arnawsiam::accountid:role/*"
					}
					}
					}
					{
						"Sid". "Lock Down List to Apps Roles and ESF"
						"Effect": "Deny',
						"Principal": "*"
						"Action": [
						"s3:ListBucket",
						"s3:ListBucketVersions"
						],
						"Resource": "arn:aws:s3.:rawbucket'
						"Condition": {
						"ForAnyValue:Arn NotEquals": {
						"aws:PrincipalArn": [
						"arn:aws:iam::acountid:role/Cloud",
						"arn:aws:iam::acoountid:role/lambda"
						"ForAnyValue:StringNotLike": {
						"aws:PrincipalArn": [
						"arn:aws:iam:: accountid:role/globalaccess/*!
						"arn:aws.iam: accountid::user/globalaccess/*"
						]}]}}

```

Answer

Still the s3 bucket policy is difficult to interpret.. Json format is not in correct format and s3 arn under the resources do not  looks good: such as "Resource": "arn.aws:3:.raw/*" (which seems incomplete and incorrect)

Generally 403 forbidden error in Amazon s3 occurs due to various reason:

-> If your bucket is encrypted and IAM users/roles do not have permission to use KMS (AWS Key management service) key.

-> If there is explicit deny in the bucket policy to specific IAM users or roles.

-> If certain permission are missing that users needs to perform for example, the bucket policy doesn’t include permission to the s3:PutObject action. If the IAM user tries to upload objects, then the user gets an Access Denied error.

-> AWS organizations service control policy doesn't allow access to Amazon s3..

-> Other policy error :

- Check that there aren’t any extra spaces/special character or incorrect ARNs in the bucket policy or IAM user policies.
                     For example, if an IAM policy has an extra space in the Amazon Resource Name (ARN) as follows: arn:aws:s3::: DOC-EXAMPLE-BUCKET/*. In this case, the ARN is then incorrectly evaluated as arn:aws:s3:::%20DOC-EXAMPLE-BUCKET/ and gives the IAM user an access denied error.

There are several AWS public document on this. You can refers to below document and see if this helps..

https://docs.aws.amazon.com/AmazonS3/latest/userguide/troubleshoot-403-errors.html

https://repost.aws/knowledge-center/s3-troubleshoot-403

https://repost.aws/knowledge-center/s3-403-forbidden-error

I see the exception 403 forbidden error while reading from s3 raw bucket . My bucket will have nested folders , to access that files my spark reader will call recursively that folder files

Relevant content