CloudFormation Template failing while setting up Athena CUR

0

Hi AWS, I was querying Cost and Usage Reports (CURs) using Athena. By default a CloudFormation template gets created and it gets stored inside the bucket where the report is getting stored, however when I was trying to deploy the CloudFormation stack using the S3 Object URL CloudFormation template, it is failing with the error provided in the screenshot below.

CUR Athena CloudFormation Error

The IAM user is having AdministratorAccess and AWSGlueConsoleFullAccess AWS managed IAM policies attached to it.

Here is the code for CloudFormation Template:

AWSTemplateFormatVersion: 2010-09-09
Resources:

  AWSCURDatabase:
    Type: 'AWS::Glue::Database'
    Properties:
      DatabaseInput:
        Name: 'athenacurcfn_ec2_costing_report_cur'
      CatalogId: !Ref AWS::AccountId

         

  AWSCURCrawlerComponentFunction:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - glue.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      ManagedPolicyArns:
        - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSGlueServiceRole'
      Policies:
        - PolicyName: AWSCURCrawlerComponentFunction
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 'glue:UpdateDatabase'
                  - 'glue:UpdatePartition'
                  - 'glue:CreateTable'
                  - 'glue:UpdateTable'
                  - 'glue:ImportCatalogToGlue'
                Resource: '*'
              - Effect: Allow
                Action:
                  - 's3:GetObject'
                  - 's3:PutObject'
                Resource: !Sub 'arn:${AWS::Partition}:s3:::ec2-cur-reporting/ec2-cur/ec2-costing-report-cur/ec2-costing-report-cur*'
        - PolicyName: AWSCURKMSDecryption
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'kms:Decrypt'
                Resource: '*'
       

  AWSCURCrawlerLambdaExecutor:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: AWSCURCrawlerLambdaExecutor
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 'glue:StartCrawler'
                Resource: '*'
       

  AWSCURCrawler:
    Type: 'AWS::Glue::Crawler'
    DependsOn:
      - AWSCURDatabase
      - AWSCURCrawlerComponentFunction
    Properties:
      Name: AWSCURCrawler-ec2-costing-report-cur
      Description: A recurring crawler that keeps your CUR table in Athena up-to-date.
      Role: !GetAtt AWSCURCrawlerComponentFunction.Arn
      DatabaseName: !Ref AWSCURDatabase
      Targets:
        S3Targets:
          - Path: 's3://ec2-cur-reporting/ec2-cur/ec2-costing-report-cur/ec2-costing-report-cur'
            Exclusions:
              - '**.json'
              - '**.yml'
              - '**.sql'
              - '**.csv'
              - '**.gz'
              - '**.zip'
      SchemaChangePolicy:
        UpdateBehavior: UPDATE_IN_DATABASE
        DeleteBehavior: DELETE_FROM_DATABASE
       

  AWSCURInitializer:
    Type: 'AWS::Lambda::Function'
    DependsOn: AWSCURCrawler
    Properties:
      Code:
        ZipFile: >
          const { GlueClient, StartCrawlerCommand } = require('@aws-sdk/client-glue');
          const response = require('./cfn-response');
          exports.handler = function (event, context, callback) {
            if (event.RequestType === 'Delete') {
              response.send(event, context, response.SUCCESS);
            } else {
              const glue = new GlueClient();
              const input = {
                Name: 'AWSCURCrawler-ec2-costing-report-cur',
              };
              const command = new StartCrawlerCommand(input);
              glue.send(command, function (err, data) {
                if (err) {
                  const responseData = JSON.parse(this.httpResponse.body);
                  if (responseData['__type'] == 'CrawlerRunningException') {
                    callback(null, responseData.Message);
                  } else {
                    const responseString = JSON.stringify(responseData);
                    if (event.ResponseURL) {
                      response.send(event, context, response.FAILED, {
                        msg: responseString,
                      });
                    } else {
                      callback(responseString);
                    }
                  }
                } else {
                  if (event.ResponseURL) {
                    response.send(event, context, response.SUCCESS);
                  } else {
                    callback(null, response.SUCCESS);
                  }
                }
              });
            }
          };

      Handler: 'index.handler'
      Timeout: 30
      Runtime: nodejs18.x
      ReservedConcurrentExecutions: 1
      Role: !GetAtt AWSCURCrawlerLambdaExecutor.Arn
     

  AWSStartCURCrawler:
    Type: 'Custom::AWSStartCURCrawler'
    Properties:
      ServiceToken: !GetAtt AWSCURInitializer.Arn
     

  AWSS3CUREventLambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: !GetAtt AWSCURInitializer.Arn
      Principal: 's3.amazonaws.com'
      SourceAccount: !Ref AWS::AccountId
      SourceArn: !Sub 'arn:${AWS::Partition}:s3:::ec2-cur-reporting'
     

  AWSS3CURLambdaExecutor:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: AWSS3CURLambdaExecutor
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub 'arn:${AWS::Partition}:logs:*:*:*'
              - Effect: Allow
                Action:
                  - 's3:PutBucketNotification'
                Resource: !Sub 'arn:${AWS::Partition}:s3:::ec2-cur-reporting'
       

  AWSS3CURNotification:
    Type: 'AWS::Lambda::Function'
    DependsOn:
    - AWSCURInitializer
    - AWSS3CUREventLambdaPermission
    - AWSS3CURLambdaExecutor
    Properties:
      Code:
        ZipFile: >
          const { S3Client, PutBucketNotificationConfigurationCommand } = require('@aws-sdk/client-s3');
          const response = require('./cfn-response');
          exports.handler = function (event, context, callback) {
            const s3 = new S3Client();
            const putConfigRequest = function (notificationConfiguration) {
              return new Promise(function (resolve, reject) {
                const input = {
                  Bucket: event.ResourceProperties.BucketName,
                  NotificationConfiguration: notificationConfiguration,
                };
                const command = new PutBucketNotificationConfigurationCommand(input);
                s3.send(command, function (err, data) {
                  if (err)
                    reject({
                      msg: this.httpResponse.body.toString(),
                      error: err,
                      data: data,
                    });
                  else resolve(data);
                });
              });
            };
            const newNotificationConfig = {};
            if (event.RequestType !== 'Delete') {
              newNotificationConfig.LambdaFunctionConfigurations = [
                {
                  Events: ['s3:ObjectCreated:*'],
                  LambdaFunctionArn:
                    event.ResourceProperties.TargetLambdaArn || 'missing arn',
                  Filter: {
                    Key: {
                      FilterRules: [
                        { Name: 'prefix', Value: event.ResourceProperties.ReportKey },
                      ],
                    },
                  },
                },
              ];
            }
            putConfigRequest(newNotificationConfig)
              .then(function (result) {
                response.send(event, context, response.SUCCESS, result);
                callback(null, result);
              })
              .catch(function (error) {
                response.send(event, context, response.FAILED, error);
                console.log(error);
                callback(error);
              });
          };
      Handler: 'index.handler'
      Timeout: 30
      Runtime: nodejs18.x
      ReservedConcurrentExecutions: 1
      Role: !GetAtt AWSS3CURLambdaExecutor.Arn
     

  AWSPutS3CURNotification:
    Type: 'Custom::AWSPutS3CURNotification'
    Properties:
      ServiceToken: !GetAtt AWSS3CURNotification.Arn
      TargetLambdaArn: !GetAtt AWSCURInitializer.Arn
      BucketName: 'ec2-cur-reporting'
      ReportKey: 'ec2-cur/ec2-costing-report-cur/ec2-costing-report-cur'
     

  AWSCURReportStatusTable:
    Type: 'AWS::Glue::Table'
    DependsOn: AWSCURDatabase
    Properties:
      DatabaseName: athenacurcfn_ec2_costing_report_cur
      CatalogId: !Ref AWS::AccountId
      TableInput:
        Name: 'cost_and_usage_data_status'
        TableType: 'EXTERNAL_TABLE'
        StorageDescriptor:
          Columns:
            - Name: status
              Type: 'string'
          InputFormat: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
          OutputFormat: 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
          SerdeInfo:
            SerializationLibrary: 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
          Location: 's3://ec2-cur-reporting/ec2-cur/ec2-costing-report-cur/cost_and_usage_data_status/'

      

Neither CloudFormation Linter nor ChatGPT detected any error in the infrastructure code. Please help as I have used the default generated template provided by AWS.

1 Answer
0

Even if your IAM principal (user or role) is using the AdministratorAccess policy you may lack the necessary permissions to run the CFN template, if your account is part of an AWS Organization and the Organization has a Service Control Policies (SCP) defined that denies the action.

One way to troubleshoot is to use the Policy Simulator to see if your principal has the permissions to execute the CreateCrawler action for AWS Glue, since that's the step where your template is failing. If you find that your principal is not authorized to perform this action, and that your principal has the AdministratorAccess policy assigned, it is very likely that you have a SCP within your Organization that does not authorize this action. If that's the case, you'll need to work with your Organization administrators to modify the SCP to allow this action.

AWS
answered a month ago
profile picture
EXPERT
reviewed a month ago
  • Hi andrewsjw, it is a standalone account with no SCP applied. The weird part is the CloudFormation template is auto-generated by AWS, still it is erroring out. However, I have checked the user permission for the Glue action CreateCrawler and it is showing that it has the permission however I tried to deploy the template a couple of times but still that issue persists. Please help.

  • I have attached an inline policy having glue:CreateCrawler access but I am getting the same error while deploying the CloudFormation template. Here is the error:

    Account XXXXXXXXXXXX is denied access. (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: f52b53db-4d10-4704-8e98-70ee46eb8f52; Proxy: null).

    This is a little confusing because I have tried deploying the template using the root user as well but the same issue persists. These are the policies attached to the IAM user:

    • AdministratorAccess
    • AWSGlueConsoleFullAccess
    • glue_create_crawler_policy

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions