Why does my AWS Glue job return the “403 Access Denied” error?

Short description

The following examples are common reasons that you might get an "Access Denied" error:

• The AWS Identity and Access Management (IAM) role doesn't have the required permissions to access the bucket.
• The Amazon S3 bucket policies don't allow the required permissions to the IAM role.
• The S3 bucket owner is different from the object owner.
• The Amazon Virtual Private Cloud (Amazon VPC) endpoint policy doesn't include the required permissions to access the S3 bucket.
• The object is encrypted by AWS Key Management Service (AWS KMS). The AWS KMS policy doesn't grant the minimum required permissions to the IAM role to use the key.
• The S3 bucket has the Requester Pays option turned on.
• The AWS Organizations service control policies restrict access to the S3 bucket.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Update the IAM role permissions to access the S3 bucket

The IAM role that runs the AWS Glue job needs access to the S3 bucket. To grant the required permissions to the IAM role, attach an IAM policy to the IAM role. It's also a best practice to attach the AWSGlueServiceRole Managed Policy to the IAM role to confirm that the basic AWS Glue job permissions are provided. Additionally, create and attach a Customer Managed policy for permissions to put S3 objects while writing.

To update the IAM role's permissions to access the bucket, complete the following steps:

1. Open the IAM console.
2. Open the IAM role that's associated to the AWS Glue job and requires access to the bucket.
3. In the Permissions tab of the IAM user/role, expand each policy to view its JSON policy document.
4. In the JSON policy documents, look for policies with the bucket's name. Then, confirm that these policies allow the correct S3 actions on the bucket. If the IAM role doesn't grant the required access to the bucket, then add a policy that grants the correct permissions. For example, the following IAM policy grants the IAM role access to put objects (s3:PutObject) into the S3 bucket DOC-EXAMPLE-BUCKET:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "ExampleStmt",
         "Action": "s3:PutObject",
         "Effect": "Allow",
         "Resource": [
           "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
         ]
       }
     ]
   }

Note: Replace DOC-EXAMPLE-BUCKET in the policy with the name of your S3 bucket.

Update the bucket policy to grant the required permissions to the IAM role

Review the bucket policy for the following configurations:

• Any statements that explicitly deny the IAM role's access to the bucket
• Any missing permissions and conditions that might restrict the IAM role's access

To review and modify the bucket policy to grant the required access to the IAM role, complete the following steps:

1. Open the Amazon S3 console.
2. In the navigation pane, choose Buckets.
3. Select the S3 bucket.
4. On the Permissions page, under Bucket policy, review the bucket policy for any statements that deny the role's access to the bucket.
5. Modify the bucket policy to edit or remove any statements that deny the IAM role's access to the bucket.

For sample bucket policies, see Examples of Amazon S3 bucket policies.

Update the owner of an object

By default, the AWS Account that uploads an object owns the S3 object. This setting is true even when a different account owns the bucket. If other accounts can upload objects to your bucket, then check which account owns the objects that your IAM identity can't access. To check what account owns the object, run the GetObjectAcl command.

If IAM identities from other accounts upload objects to your S3 bucket, then configure S3 Object Ownership. Then, add a bucket policy that requires objects to be uploaded with the bucket-owner-full-control access control list (ACL). When you add the bucket policy, the policy automatically changes the object's owner to the bucket owner when the object is uploaded with bucket-owner-full-control ACL. For more information, see When other AWS accounts upload objects to my Amazon S3 bucket, how can I require that they grant me full control of the objects?

Update the Amazon VPC endpoint policy

Update the VPC endpoint policy to include the required permissions to access the S3 buckets and objects when both the following conditions are true:

• Your AWS Glue job reads or writes objects into Amazon S3.
• The connection uses a VPC endpoint to route to Amazon S3.

For example, the following VPC endpoint policy allows access to only the bucket DOC-EXAMPLE-BUCKET. If your bucket isn't listed as an allowed resource in the policy, then the users or roles can't use the VPC endpoint to access your bucket:

{
  "Statement": [
    {
      "Sid": "Access-to-specific-bucket-only",
      "Principal": "*",
      "Action": [
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
    }
  ]
}

Note: Replace DOC-EXAMPLE-BUCKET in the policy with the name of your S3 bucket.

If users or roles upload objects with an ACL, then you must update the VPC endpoint policy to grant access to the PutObjectAcl action. For example:

{
  "Statement": [
    {
      "Sid": "Access-to-specific-bucket-only",
      "Principal": "*",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
    }
  ]
}

Update the AWS KMS key policy to allow access to the IAM role

If your extract, transform, and load (ETL) job reads or writes encrypted data into Amazon S3, then your job must have the following configurations:

• The IAM role's policy includes the permissions required for the AWS KMS actions.
• The AWS KMS key's policy includes the required permissions for the IAM role.

To allow the necessary AWS KMS actions, include the following permissions in the IAM role's policy:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": [
      "kms:Decrypt",
      "kms:Encrypt",
      "kms:GenerateDataKey"
    ],
    "Resource": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
  }
}

Note: Replace the Amazon Resource Name (ARN) in the policy with the ARN of your choice.

For more information, see Setting up encryption in AWS Glue.

Review the KMS key policy to verify if the policy allows access to the AWS Glue job's role. For more information about key policies, see Key policies in AWS KMS.

Review the Requester Pays option in your S3 bucket

If the S3 bucket has the Requester Pays option turned on, then all requests to the bucket from the AWS Glue job must include the Requester Pays header. By default, AWS Glue requests to Amazon S3 don't include the Requester Pays header. Without this header, an API call to a Requester Pays bucket fails with an "Access Denied" exception. To add the Requester Pays header to an ETL script, use hadoopConfiguration().set() to include fs.s3.useRequesterPaysHeader on the GlueContext variable or the Apache Spark session variable. For more information, see How do I access Amazon S3 Requester Pays buckets from AWS Glue, Amazon EMR, or Athena?

Review your AWS Organizations service control policies

If you use AWS Organizations, then review the service control policies to make sure that the policy allows access to Amazon S3. For example, the following policy explicitly denies access to Amazon S3 and results in an "Access Denied" error:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Action": "s3:*",
      "Resource": "*"
    }
  ]
}

For more information, see Turning on all features for an organization with AWS Organizations.

Related information

How do I troubleshoot 403 Access Denied errors from Amazon S3?