How do I resolve "The IAM role must delegate access to an Amazon Redshift account" error in Amazon Redshift when using AWS CloudFormation?

Short description

With AWS CloudFormation, you can create a template in JSON or YAML that describes all the AWS resources that you need. AWS CloudFormation then provisions and configures the AWS resources for you. You can even use the AWS CloudFormation template to create an Amazon Redshift cluster or a scheduled action.

However, you must correctly reference the IAM role that authorizes the Amazon Redshift cluster to access the other AWS services. Otherwise, you receive the following error:

"The IAM role <role> is not valid. The IAM role must delegate access to an Amazon Redshift account."

To resolve this issue, make sure to properly create and attach the AWS IAM role using CloudFormation. After your CloudFormation template file is created, your Amazon Redshift cluster and any specified AWS resources (such as a stack) are then also automatically created. No additional (manual) updates to your stacks are required.

Resolution

Update your AWS CloudFormation template parameters (in YAML)

To update the AWS CloudFormation template parameters in YAML, perform the following steps:

1.    Define your parameters:

AWSTemplateFormatVersion: 2010-09-09
Description: Create Redshift Stack. 
Parameters:
  Environment:
    Description: Environment of the resources.
    Type: String
    Default: staging
    AllowedValues:
      - production
      - staging
      - testing
  Name:
    Description: Cluster name.
    Type: String
    Default: 'mycluster'
  Service:
    Description: Service name.
    Type: String
    Default: redshift
    AllowedValues:
      - redshift 
  DatabaseName:
    Description:  Database name.
    Type: String
    Default: dev
    AllowedPattern: "([a-z]|[0-9])+"
  ClusterType:
    Description: The type of cluster
    Type: String
    Default: multi-node
    AllowedValues:
    - single-node
    - multi-node
  NumberOfNodes:
    Description: Compute nodes count. For multi-node clusters,
      the NumberOfNodes parameter must be greater than 1
    Type: Number
    Default: '2'
  NodeType:
    Description: The type of node to be provisioned
    Type: String
    Default: dc2.large
    AllowedValues: 
    - dc2.large
    - dc2.8xlarge
    - ra3.4xlarge
    - ra3.16xlarge
  MasterUsername:
    Description: Master user name.
    Type: String
    Default: awsuser
    AllowedPattern: "([a-z])([a-z]|[0-9])*"
  MasterUserPassword:
    Description: Master user password. Must have a length of 8-64 characters, contain one uppercase letter, one lowercase letter, and one number. Also only contain printable ASCII characters except for '/', '@', '"', ' ', '\' and '\'.
    Type: String
    NoEcho: 'true'
  PortNumber:
    Description: The port number on which the cluster accepts incoming connections.
    Type: Number
    Default: '5439'
Conditions:
  IsMultiNodeCluster:
    Fn::Equals:
    - Ref: ClusterType
    - multi-node

Note: Rather than embedding sensitive information in your AWS CloudFormation templates, it is best practice to use dynamic references in your stack template. For more information on best practices, see Security best practices for AWS CloudFormation.

2.    Under Resources, create the IAM role that will be assumed by the RedShift service to access other AWS services: Resources: RedshiftRole: Type: AWS::IAM::Role Properties:

Resources:
  RedshiftRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${Environment}-${Name}-${Service}
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
              - redshift.amazonaws.com
          Action:
            - sts:AssumeRole
          Condition:
            StringEquals:
              sts:ExternalId: !Sub 'arn:aws:redshift:${AWS::Region}:${AWS::AccountId}:dbuser:${Environment}-${Name}-${Service}/awsuser'
      Path: "/"

3.    Under Policies, specify the IAM policies to be attached to the IAM role:

Policies:  
  - PolicyName: policy-s3
    PolicyDocument:
    Version: 2012-10-17
    Statement:
    - Effect: Allow
    Action:
      - 's3:AbortMultipartUpload'
      - 's3:GetBucketLocation'
      - 's3:ListBucket'
      - 's3:ListBucketMultipartUploads'
      - 's3:GetObject'
      - 's3:PutObject'
    Resource:
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1"
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2"
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2/*"
    - Effect: Allow 
    Action:
      - 's3:DeleteObject'
    Resource:
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
      - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2/*"
  - PolicyName: policy-cloudwatch
    PolicyDocument:
    Version: 2012-10-17
    Statement:
    - Effect: Allow
    Action:
      - 'logs:CreateLogGroup'
      - 'logs:CreateLogStream'
      - 'logs:PutLogEvents'
    Resource: "*"

4.    Use the Function Fn::GetAtt function to create the Amazon Redshift cluster, and then attach the IAM role (RedshiftRole):

RedshiftCluster:
  Type: AWS::Redshift::Cluster
  Properties: 

    IamRoles:
    - Fn::GetAtt: [ RedshiftRole, Arn ]

    AllowVersionUpgrade: true
    AutomatedSnapshotRetentionPeriod: 7 
    ClusterIdentifier: !Sub ${Environment}-${Name}-${Service} 
    ClusterVersion: 1.0
    ClusterType:
    Ref: ClusterType
    NumberOfNodes:
    Fn::If:
    - IsMultiNodeCluster
    - Ref: NumberOfNodes
    - Ref: AWS::NoValue
    NodeType:
    Ref: NodeType
    DBName:
    Ref: DatabaseName
    MasterUsername:
    Ref: MasterUsername
    MasterUserPassword:
    Ref: MasterUserPassword
    Port:
    Ref: PortNumber
    PreferredMaintenanceWindow: Sun:18:30-Sun:19:30
    PubliclyAccessible: yes 
    AvailabilityZone: !Select [0, !GetAZs ""]

The Fn::GetAtt function returns the value for a specified attribute.

Note: If you use an incorrect form of reference to attach to your IAM role (such as the Ref function), you receive an IAM role error.

Update your AWS CloudFormation template parameters (in JSON)

To update the AWS CloudFormation template parameters in JSON, perform the following steps:

1.    Define your parameters:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "Create Redshift Stack.",
  "Parameters": {
    "Environment": {
      "Description": "Environment of the resources.",
      "Type": "String",
      "Default": "staging",
      "AllowedValues": [
        "production",
        "staging",
        "testing"
      ]
    },
    "Name": {
      "Description": "Cluster name.",
      "Type": "String",
      "Default": "mycluster"
    },
    "Service": {
      "Description": "Service name.",
      "Type": "String",
      "Default": "redshift",
      "AllowedValues": [
        "redshift"
      ]
    },
    "DatabaseName": {
      "Description": "Database name.",
      "Type": "String",
      "Default": "dev",
      "AllowedPattern": "([a-z]|[0-9])+"
    },
    "ClusterType": {
      "Description": "The type of cluster",
      "Type": "String",
      "Default": "multi-node",
      "AllowedValues": [
        "single-node",
        "multi-node"
      ]
    },
    "NumberOfNodes": {
      "Description": "Compute nodes count. For multi-node clusters, the NumberOfNodes parameter must be greater than 1",
      "Type": "Number",
      "Default": "2"
    },
    "NodeType": {
      "Description": "The type of node to be provisioned",
      "Type": "String",
      "Default": "dc2.large",
      "AllowedValues": [ 
        "dc2.large",
        "dc2.8xlarge",
        "ra3.4xlarge",
        "ra3.16xlarge"
      ]
    },
    "MasterUsername": {
      "Description": "Master user name.",
      "Type": "String",
      "Default": "awsuser",
      "AllowedPattern": "([a-z])([a-z]|[0-9])*"
    },
    "MasterUserPassword": {
      "Description": "Master user password. Must have a length of 8-64 characters, contain one uppercase letter, one lowercase letter, and one number. Also only contain printable ASCII characters except for '/', '@', '\"', ' ', '\\' and '\\'.",
      "Type": "String",
      "NoEcho": "true"
    },
    "PortNumber": {
      "Description": "The port number on which the cluster accepts incoming connections.",
      "Type": "Number",
      "Default": "5439"
    }
  },
  "Conditions": {
    "IsMultiNodeCluster": {
      "Fn::Equals": [
        {
          "Ref": "ClusterType"
        },
        "multi-node"
      ]
    }
  },

Note: Rather than embedding sensitive information in your AWS CloudFormation templates, it is best practice to use dynamic references in your stack template. For more information on best practices, see Security best practices for AWS CloudFormation.

2.    Under Resources, create the IAM role to be used to access your Amazon Redshift cluster:

"Resources": {
    "RedshiftRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "RoleName": {
          "Fn::Sub": "${Environment}-${Name}-${Service}"
        },
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "redshift.amazonaws.com"
                ]
              },
              "Action": [
                "sts:AssumeRole"
              ],
              "Condition": {
                "StringEquals": {
                  "sts:ExternalId": {
                    "Fn::Sub": "arn:aws:redshift:${AWS::Region}:${AWS::AccountId}:dbuser:${Environment}-${Name}-${Service}/awsuser"
                  }
                }
              }
            }
          ]
        },
        "Path": "/",

3.    Under Policies, specify the IAM policies to be attached to the IAM role:

"Policies": [
      {
      "PolicyName": "policy-s3",
      "PolicyDocument": {
        "Version": "2012-10-17",
        "Statement": [
        {
          "Effect": "Allow",
          "Action": [
          "s3:AbortMultipartUpload",
          "s3:GetBucketLocation",
          "s3:ListBucket",
          "s3:ListBucketMultipartUploads",
          "s3:GetObject",
          "s3:PutObject"
          ],
          "Resource": [
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1"
          },
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
          },
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2"
          },
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2/*"
          }
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
          "s3:DeleteObject"
          ],
          "Resource": [
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
          },
          {
            "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2/*"
          }
          ]
        }
        ]
      }
      },
      {
      "PolicyName": "policy-cloudwatch",
      "PolicyDocument": {
        "Version": "2012-10-17",
        "Statement": [
        {
          "Effect": "Allow",
          "Action": [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:PutLogEvents"
          ],
          "Resource": "*"
        }
        ]
      }
      }
    ]
    }
  },

4.    Use the Function Fn::GetAtt function to create the Amazon Redshift cluster, and then attach the IAM role (RedshiftRole):

"RedshiftCluster": {
      "Type": "AWS::Redshift::Cluster",
      "Properties": {
        "IamRoles": [
          {
            "Fn::GetAtt": [
              "RedshiftRole",
              "Arn"
            ]
          }
        ],
        "AllowVersionUpgrade": true,
        "AutomatedSnapshotRetentionPeriod": 7,
        "ClusterIdentifier": {
          "Fn::Sub": "${Environment}-${Name}-${Service}"
        },
        "ClusterVersion": 1,
        "ClusterType": {
          "Ref": "ClusterType"
        },
        "NumberOfNodes": {
          "Fn::If": [
            "IsMultiNodeCluster",
            {
              "Ref": "NumberOfNodes"
            },
            {
              "Ref": "AWS::NoValue"
            }
          ]
        },
        "NodeType": {
          "Ref": "NodeType"
        },
        "DBName": {
          "Ref": "DatabaseName"
        },
        "MasterUsername": {
          "Ref": "MasterUsername"
        },
        "MasterUserPassword": {
          "Ref": "MasterUserPassword"
        },
        "Port": {
          "Ref": "PortNumber"
        },
        "PreferredMaintenanceWindow": "Sun:18:30-Sun:19:30",
        "PubliclyAccessible": "true",
        "AvailabilityZone": {
          "Fn::Select": [
            0,
            {
              "Fn::GetAZs": ""
            }
          ]
        }
      }
    }
  }
}

The Fn::GetAtt function returns the value for a specified attribute.

Note: If you use an incorrect form of reference to attach to your IAM role (such as the Ref function), you receive an IAM role error.

Create a new stack in AWS CloudFormation

To create a stack, using your JSON or YAML template, perform the following steps:

1.    Open the AWS CloudFormation console.

2.    Choose Create stack to create a new stack.

3.    Under Prerequisite - Prepare template in Step 1, choose Template is ready.

4.    Under Specify template in Step 1, select your template source.

5.    (Optional) If your template file must be uploaded, upload your template file.

6.    Choose Next.

7.    Under Specify stack details in Step 2, specify your Stack name and Parameters.

8.    Choose Next.

9.    Review all the stack details under Configure stack options in Step 3.

10.    (Optional) To modify your stack options, choose Previous until you reach the page you want update. Update the stack options as needed, and then choose Next until you return to the Configure stack options page.

11.    Choose Create stack.

Related information

Amazon Redshift resource type reference

Amazon Redshift template snippets

Automate Amazon Redshift cluster creation using AWS CloudFormation