How do I resolve the IAM role error that I receive in Amazon Redshift when I use CloudFormation?

6 minute read
0

I want to create an Amazon Redshift cluster or scheduled action in AWS CloudFormation. However, I receive the "The IAM role must delegate access to an Amazon Redshift account" AWS Identity and Access Management (IAM) role error.

Short description

When you try to create an Amazon Redshift cluster or scheduled action in CloudFormation, you receive the following error message:

"The IAM role is not valid. The IAM role must delegate access to an Amazon Redshift account."

To resolve this issue, you must define the IAM role in your CloudFormation template parameters so that the cluster can access other AWS services.

Resolution

Update your CloudFormation template parameters in YAML

To update the CloudFormation template parameters in YAML, complete the following steps:

  1. Define your parameters to create a stack:

    AWSTemplateFormatVersion: 2010-09-09
    Description: Create Redshift Stack. 
    Parameters:
      Environment:
        Description: Environment of the resources.
        Type: String
        Default: staging
        AllowedValues:
          - production
          - staging
          - testing
      Name:
        Description: Cluster name.
        Type: String
        Default: 'mycluster'
      Service:
        Description: Service name.
        Type: String
        Default: redshift
        AllowedValues:
          - redshift 
      DatabaseName:
        Description:  Database name.
        Type: String
        Default: dev
        AllowedPattern: "([a-z]|[0-9])+"
      ClusterType:
        Description: The type of cluster
        Type: String
        Default: multi-node
        AllowedValues:
        - single-node
        - multi-node
      NumberOfNodes:
        Description: Compute nodes count. For multi-node clusters,
          the NumberOfNodes parameter must be greater than 1
        Type: Number
        Default: '2'
      NodeType:
        Description: The type of node to be provisioned
        Type: String
        Default: dc2.large
        AllowedValues: 
        - dc2.large
        - dc2.8xlarge
        - ra3.4xlarge
        - ra3.16xlarge
      MasterUsername:
        Description: Master user name.
        Type: String
        Default: awsuser
        AllowedPattern: "([a-z])([a-z]|[0-9])*"
      MasterUserPassword:
        Description: Master user password. Must have a length of 8-64 characters, contain one uppercase letter, one lowercase letter, and one number. Also only contain printable ASCII characters except for '/', '@', '"', ' ', '\' and '\'.
        Type: String
        NoEcho: 'true'
      PortNumber:
        Description: The port number on which the cluster accepts incoming connections.
        Type: Number
        Default: '5439'
    Conditions:
      IsMultiNodeCluster:
        Fn::Equals:
        - Ref: ClusterType
        - multi-node

    Note: It's a best practice to use dynamic references in your stack template. For more information about best practices, see Security best practices for CloudFormation.

  2. Use the following template to create an IAM role for Amazon RedShift to assume when it accesses other AWS services:

    Resources:
      RedshiftRole:
        Type: AWS::IAM::Role
        Properties:
          RoleName: !Sub ${Environment}-${Name}-${Service}
          AssumeRolePolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Principal:
                Service:
                  - redshift.amazonaws.com
              Action:
                - sts:AssumeRole
              Condition:
                StringEquals:
                  sts:ExternalId: !Sub 'arn:aws:redshift:${AWS::Region}:${AWS::AccountId}:dbuser:${Environment}-${Name}-${Service}/awsuser'
          Path: "/"
  3. Under Policies, specify the IAM policies to attach to the IAM role:

    Policies:  
      - PolicyName: policy-s3
        PolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
        Action:
          - 's3:AbortMultipartUpload'
          - 's3:GetBucketLocation'
          - 's3:ListBucket'
          - 's3:ListBucketMultipartUploads'
          - 's3:GetObject'
          - 's3:PutObject'
        Resource:
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1"
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2"
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2/*"
        - Effect: Allow 
        Action:
          - 's3:DeleteObject'
        Resource:
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
          - !Sub "arn:aws:s3:::${Environment}-${Name}-tier2/*"
      - PolicyName: policy-cloudwatch
        PolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
        Action:
          - 'logs:CreateLogGroup'
          - 'logs:CreateLogStream'
          - 'logs:PutLogEvents'
        Resource: "*"
  4. Use the following template with the Fn::GetAtt function to create the Amazon Redshift cluster, and include the RedshiftRole:

    RedshiftCluster:
      Type: AWS::Redshift::Cluster
      Properties: 
    
        IamRoles:
        - Fn::GetAtt: [ RedshiftRole, Arn ]
    
        AllowVersionUpgrade: true
        AutomatedSnapshotRetentionPeriod: 7 
        ClusterIdentifier: !Sub ${Environment}-${Name}-${Service} 
        ClusterVersion: 1.0
        ClusterType:
        Ref: ClusterType
        NumberOfNodes:
        Fn::If:
        - IsMultiNodeCluster
        - Ref: NumberOfNodes
        - Ref: AWS::NoValue
        NodeType:
        Ref: NodeType
        DBName:
        Ref: DatabaseName
        MasterUsername:
        Ref: MasterUsername
        MasterUserPassword:
        Ref: MasterUserPassword
        Port:
        Ref: PortNumber
        PreferredMaintenanceWindow: Sun:18:30-Sun:19:30
        PubliclyAccessible: yes 
        AvailabilityZone: !Select [0, !GetAZs ""]

    Note: An incorrect IAM role reference, such as Ref, causes an error.

Update your CloudFormation template parameters in JSON

To update the CloudFormation template parameters in JSON, complete the following steps:

  1. Define your parameters to create a stack:

    {
      "AWSTemplateFormatVersion": "2010-09-09",
      "Description": "Create Redshift Stack.",
      "Parameters": {
        "Environment": {
          "Description": "Environment of the resources.",
          "Type": "String",
          "Default": "staging",
          "AllowedValues": [
            "production",
            "staging",
            "testing"
          ]
        },
        "Name": {
          "Description": "Cluster name.",
          "Type": "String",
          "Default": "mycluster"
        },
        "Service": {
          "Description": "Service name.",
          "Type": "String",
          "Default": "redshift",
          "AllowedValues": [
            "redshift"
          ]
        },
        "DatabaseName": {
          "Description": "Database name.",
          "Type": "String",
          "Default": "dev",
          "AllowedPattern": "([a-z]|[0-9])+"
        },
        "ClusterType": {
          "Description": "The type of cluster",
          "Type": "String",
          "Default": "multi-node",
          "AllowedValues": [
            "single-node",
            "multi-node"
          ]
        },
        "NumberOfNodes": {
          "Description": "Compute nodes count. For multi-node clusters, the NumberOfNodes parameter must be greater than 1",
          "Type": "Number",
          "Default": "2"
        },
        "NodeType": {
          "Description": "The type of node to be provisioned",
          "Type": "String",
          "Default": "dc2.large",
          "AllowedValues": [ 
            "dc2.large",
            "dc2.8xlarge",
            "ra3.4xlarge",
            "ra3.16xlarge"
          ]
        },
        "MasterUsername": {
          "Description": "Master user name.",
          "Type": "String",
          "Default": "awsuser",
          "AllowedPattern": "([a-z])([a-z]|[0-9])*"
        },
        "MasterUserPassword": {
          "Description": "Master user password. Must have a length of 8-64 characters, contain one uppercase letter, one lowercase letter, and one number. Also only contain printable ASCII characters except for '/', '@', '\"', ' ', '\\' and '\\'.",
          "Type": "String",
          "NoEcho": "true"
        },
        "PortNumber": {
          "Description": "The port number on which the cluster accepts incoming connections.",
          "Type": "Number",
          "Default": "5439"
        }
      },
      "Conditions": {
        "IsMultiNodeCluster": {
          "Fn::Equals": [
            {
              "Ref": "ClusterType"
            },
            "multi-node"
          ]
        }
      },
  2. Use the following template to create the IAM role to access your Amazon Redshift cluster:

    "Resources": {
        "RedshiftRole": {
          "Type": "AWS::IAM::Role",
          "Properties": {
            "RoleName": {
              "Fn::Sub": "${Environment}-${Name}-${Service}"
            },
            "AssumeRolePolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Principal": {
                    "Service": [
                      "redshift.amazonaws.com"
                    ]
                  },
                  "Action": [
                    "sts:AssumeRole"
                  ],
                  "Condition": {
                    "StringEquals": {
                      "sts:ExternalId": {
                        "Fn::Sub": "arn:aws:redshift:${AWS::Region}:${AWS::AccountId}:dbuser:${Environment}-${Name}-${Service}/awsuser"
                      }
                    }
                  }
                }
              ]
            },
            "Path": "/",
  3. Under Policies, specify the IAM policies to attach to the IAM role:

    "Policies": [
          {
          "PolicyName": "policy-s3",
          "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
            {
              "Effect": "Allow",
              "Action": [
              "s3:AbortMultipartUpload",
              "s3:GetBucketLocation",
              "s3:ListBucket",
              "s3:ListBucketMultipartUploads",
              "s3:GetObject",
              "s3:PutObject"
              ],
              "Resource": [
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1"
              },
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
              },
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2"
              },
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2/*"
              }
              ]
            },
            {
              "Effect": "Allow",
              "Action": [
              "s3:DeleteObject"
              ],
              "Resource": [
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier1/log-internal-${Service}/*"
              },
              {
                "Fn::Sub": "arn:aws:s3:::${Environment}-${Name}-tier2/*"
              }
              ]
            }
            ]
          }
          },
          {
          "PolicyName": "policy-cloudwatch",
          "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
            {
              "Effect": "Allow",
              "Action": [
              "logs:CreateLogGroup",
              "logs:CreateLogStream",
              "logs:PutLogEvents"
              ],
              "Resource": "*"
            }
            ]
          }
          }
        ]
        }
      },
  4. Use the following template with the Fn::GetAtt function to create the Amazon Redshift cluster, and include the RedshiftRole:

    "RedshiftCluster": {
          "Type": "AWS::Redshift::Cluster",
          "Properties": {
            "IamRoles": [
              {
                "Fn::GetAtt": [
                  "RedshiftRole",
                  "Arn"
                ]
              }
            ],
            "AllowVersionUpgrade": true,
            "AutomatedSnapshotRetentionPeriod": 7,
            "ClusterIdentifier": {
              "Fn::Sub": "${Environment}-${Name}-${Service}"
            },
            "ClusterVersion": 1,
            "ClusterType": {
              "Ref": "ClusterType"
            },
            "NumberOfNodes": {
              "Fn::If": [
                "IsMultiNodeCluster",
                {
                  "Ref": "NumberOfNodes"
                },
                {
                  "Ref": "AWS::NoValue"
                }
              ]
            },
            "NodeType": {
              "Ref": "NodeType"
            },
            "DBName": {
              "Ref": "DatabaseName"
            },
            "MasterUsername": {
              "Ref": "MasterUsername"
            },
            "MasterUserPassword": {
              "Ref": "MasterUserPassword"
            },
            "Port": {
              "Ref": "PortNumber"
            },
            "PreferredMaintenanceWindow": "Sun:18:30-Sun:19:30",
            "PubliclyAccessible": "true",
            "AvailabilityZone": {
              "Fn::Select": [
                0,
                {
                  "Fn::GetAZs": ""
                }
              ]
            }
          }
        }
      }
    }

Create a new stack in CloudFormation

After you update the template parameters, use your JSON or YAML template to create a new CloudFormation stack.

Related information

Amazon Redshift resource type reference

Amazon Redshift template snippets

Automate Amazon Redshift cluster creation using AWS CloudFormation

AWS OFFICIAL
AWS OFFICIALUpdated 2 months ago