- Newest
- Most votes
- Most comments
The answer to the above issue is you need to attach IAM policy with sts:AssumeRole access for the dynamodb IAM role in account A to redshift cluster role in account B. Also make sure the DynamoDB table and Redshift Cluster in both accounts exist in the same region. Apart from that hashkey or rangekey in the dynamodb table should have corresponding columns in the redshift table. Make sure they exactly match in name and have suitable types else you will get error something like this:
Error: invalid end key specified detail: ----------------------------------------------- error: invalid end key specified code: 9005 context: table name = sales query: 202639 location: copy_dynamodb_scanner.cpp:203 process: query0_126_202639 [pid=19265] -----------------------------------------------
for more info refer this stackoverflow link: https://superuser.com/questions/590632/what-does-it-mean-when-redshift-gives-you-invalid-end-key-specified-on-a-dynam
The updated redshift cluster account template:
# version: 1.0
AWSTemplateFormatVersion: "2010-09-09"
Resources:
RootRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- redshift.amazonaws.com
- redshift-serverless.amazonaws.com
- scheduler.redshift.amazonaws.com
- dynamodb.amazonaws.com
AWS:
- arn:aws:iam::<redshift_account>:root
Action:
- "sts:AssumeRole"
Path: "/"
RoleName: "terraform_iam_role"
IAMPolicy:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: drdc_iam_policy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "ec2:*"
- "ecs:*"
- "redshift-serverless:*"
- "redshift:*"
- "iam:*"
- "ec2:*"
- "cloudwatch:*"
- "s3:*"
- "logs:*"
- "cloudtrail:*"
- "sns:*"
- "lambda:*"
- "kms:*"
- "route53:*"
Resource: "*"
- Effect: Allow
Action:
- iam:PassRole
Resource:
- !Sub arn:aws:iam::${AWS::AccountId}:role/aws-service-role/redshift.amazonaws.com/AWSServiceRoleForRedshift
- !Sub arn:aws:iam::${AWS::AccountId}:role/drdc_lambda_execution_redshift_role
- !Sub arn:aws:iam::${AWS::AccountId}:role/terraform_iam_role
CrossAccountAssumeRolePolicy:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: drdc_cross_account_assume_role_policy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- sts:AssumeRole
Resource: arn:aws:iam::<dynamodbtableaccount>:role/terraform_iam_role
Roles:
- Ref: RootRole
Cheers!!
Apparently my Redshift IAM role was missing a policy to STS:AssumeRole on the source IAM role.
Relevant content
- asked 5 years ago
- AWS OFFICIALUpdated 9 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated 5 months ago