Redshift Data Sharing - running into decryption permission issues cross accounts

Question

Hi, I have a full data sharing scenario setup between two same-region but different AWS accounts. On the producer side, I am using external schema pointing to a Glue catalog with Iceberg tables. On the consumer side, I am simply trying to mount and query the data shares. On both sides, I am using Serverless workgroups.

When I query from the consumer side, I am getting this error:
RROR: ----------------------------------------------- error: Error getting Iceberg table metadata from S3. What: S3ServiceException:User: arn:aws:sts:::assumed-role/SND8_Redshift_Serverless_Role/i-0d6474336788e2c28@99392845 is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:2107 code: 15003 context: query: -1[child_sequence:1] location: iceberg.hpp:1048 process: padbmaster [pid=1073873231] ----------------------------------------------- [ErrorId: 1-694964e4-107c4fb72300edf21c7eb10e]

We have updated KMS policies to the nth degree and nothing seems to be working. There is one and only one indication after all of my research with this single sentence: "Data sharing of data lake tables does not support customer managed AWS KMS keys for Amazon S3 bucket encryption. You can use AWS managed keys for encryption" in the considerations article for data lake tables: https://docs.aws.amazon.com/redshift/latest/dg/considerations-datashare-datalake.html

Is this really the problem? Do I really need to give up our CMK on S3? Isn't that a compliance issue?

Anshita Kaushik · Answer

=====================

ERROR: ----------------------------------------------- error: Error getting Iceberg table metadata from S3. What: S3ServiceException:User: arn:aws:sts:::assumed-role/SND8_Redshift_Serverless_Role/i-0d6474336788e2c28@99392845 is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:2107 code: 15003 context: query: -1[child_sequence:1] location: iceberg.hpp:1048 process: padbmaster [pid=1073873231] ----------------------------------------------- [ErrorId: 1-694964e4-107c4fb72300edf21c7eb10e]

=====================

yes your understanding is correct about the error ,this is because data sharing of data lake tables does not support customer managed AWS KMS keys for Amazon S3 bucket encryption. You can use AWS managed keys for encryption. More information on this is highlighted here: https://docs.aws.amazon.com/redshift/latest/dg/considerations-datashare-datalake.html

Additionally for an encrypted data catalog you need to ensure that the Redshift execution role has permissions to assume a role that will decrypt the catalog.

NOTE: Ensure that your S3 bucket uses server-side encryption with Amazon S3 managed keys (SSE-S3) then follow the below steps to encrypt your Glue data catalog:

```
Step 1. Create a Glue role that will be used for decryption of the catalog:

Glue role permission:
a). AmazonS3FullAccess

b). Custom-1

{
      "Effect": "Allow",
      "Action": [
        "glue:CreateDatabase",
        "glue:GetDatabase",
        "glue:GetDatabases",
        "glue:UpdateDatabase",
        "glue:CreateTable",
        "glue:DeleteTable",
        "glue:UpdateTable",
        "glue:GetTable",
        "glue:BatchCreatePartition",
        "glue:CreatePartition",
        "glue:DeletePartition",
        "glue:BatchDeletePartition",
        "glue:UpdatePartition",
        "glue:GetPartition",
        "glue:GetPartitions",
        "glue:BatchGetPartition",
        "glue:GetTables"
      ],
      "Resource": [
        "arn:aws:glue:::database/*",
        "arn:aws:glue:::table/*",
        "arn:aws:glue:::catalog"
      ]
    }
  ]
}

c). Custom-2
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:Encrypt",
        "kms:GenerateDataKey"
      ],
      "Resource": "arn:aws:kms:::key/"
    }
  ]
}

d). Glue role Trust Policy:
{
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam:::role/RedshiftRole"
  },
  "Action": "sts:AssumeRole"
}

Step 2.  Add assumed role policy to Redshift execution role
{
  "Effect": "Allow",
  "Action": "sts:AssumeRole",
  "Resource": "arn:aws:iam:::role/GlueRole"
}

3. Go to Glue "catalog setting" and under the encryption options, check the "metadata encryption" box and Choose a Customer Managed Key (CMK)  "arn:aws:kms:::key/"

4. Check the "Delegate KMS operations to an IAM role" box and select the glue role created earlier. Save the changes.

More details can be found here: https://docs.aws.amazon.com/glue/latest/dg/encrypt-glue-data-catalog.html

Redshift Data Sharing - running into decryption permission issues cross accounts

Add your answer

Relevant content