CloudFormation: CUR ValidationException - what is wrong?

Question

The S3 bucket `client-${AccountId}-cost-export` exists and I can create a Cost and Usage Export (CUR) targeting that bucket manually using the web interface without issues, but trying to create a CUR using the following cloudformation template fails with:

```
Event name: PutReportDefinition
Error code: ValidationException
```

That error message does not help at all and I have no clue what is wrong :(

```
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  AccountId:
    Description: AccountId
    Type: String
Resources:
  CostAndUsageReport:
    Type: AWS::CUR::ReportDefinition
    Properties:
      AdditionalArtifacts:
        - REDSHIFT
        - ATHENA
      AdditionalSchemaElements: 
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: true
      ReportName: cost-usage-export
      ReportVersioning: CREATE_NEW_REPORT
      S3Bucket: !Sub client-${AccountId}-cost-export
      S3Prefix: cur
      S3Region: us-east-1
      TimeUnit: DAILY
```

Leo K · Accepted Answer

There seem to be a bunch of rules that aren't explained in the API documentation.

When you specify `AdditionalArtifacts` as ATHENA, it seems you should specify the format and compression as Parquet and `ReportVersioning` as `OVERWRITE_REPORT`. It will fail with `CREATE_NEW_REPORT`.

For Redshift, the format appears to have to be `textORcsv` and compression `ZIP`, but `ReportVersioning` accepts either `OVERWRITE_REPORT` or `CREATE_NEW_REPORT`.

So in practice, both ATHENA and REDSHIFT don't appear to be possible to define for one CUR, but you'll need two separate CUR exports. The manual configuration wizard in the console also enforces this.

NARRAVULA MUNISAI TEJA · Answer

Hello,

The **ValidationException** error in your CloudFormation template for creating a Cost and Usage Report (CUR) can be frustrating.

**1. IAM Permissions:**
Ensure that the IAM role or user running the CloudFormation stack has the necessary permissions to create a CUR and write to the specified S3 bucket. The required permissions typically include:

```
*cur:PutReportDefinition
* s3:PutObject
* s3:PutObjectAcl
* s3:GetBucketAcl
* s3:GetBucketPolicy
* s3:ListBucket

```
**2.Verify S3 Bucket Permissions:**

The S3 bucket **'client-${AccountId}-cost-export'** must have the correct bucket policy to allow the CUR service to write reports to it.

```

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "billingreports.amazonaws.com"
      },
      "Action": [
        "s3:PutObject",
        "s3:GetBucketAcl"
      ],
      "Resource": "arn:aws:s3:::client-${AccountId}-cost-export/*"
    }
  ]
}
```

**CloudFormation Template:**

Ensure that the **'AccountId'** parameter is being substituted correctly.

```
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  AccountId:
    Description: AccountId
    Type: String

Resources:
  CostAndUsageReport:
    Type: AWS::CUR::ReportDefinition
    Properties:
      AdditionalArtifacts:
        - REDSHIFT
        - ATHENA
      AdditionalSchemaElements: 
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: true
      ReportName: cost-usage-export
      ReportVersioning: CREATE_NEW_REPORT
      S3Bucket: !Sub client-${AccountId}-cost-export
      S3Prefix: cur
      S3Region: us-east-1
      TimeUnit: DAILY
```
**A similar issue has been reported on GitHub regarding the AWS::CUR::ReportDefinition resource:**

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cur-reportdefinition.html

Adeleke Adebowale Julius · Answer

The error you're encountering when trying to create a Cost and Usage Report (CUR) using AWS CloudFormation is often related to permissions or incorrect properties in the CloudFormation template. Here are a few steps to troubleshoot and resolve the issue:

1. Validate Permissions
Ensure that the IAM role or user executing the CloudFormation stack has the necessary permissions to create the Cost and Usage Report. Specifically, the following permissions are required:

```
cur:PutReportDefinition
cur:DeleteReportDefinition
s3:PutObject
s3:GetBucketAcl
s3:GetBucketLocation
```
2. Verify S3 Bucket Permissions
The S3 bucket client-${AccountId}-cost-export must have the correct bucket policy to allow the CUR service to write reports to it. Below is an example of a bucket policy that grants the necessary permissions:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "billingreports.amazonaws.com"
      },
      "Action": [
        "s3:PutObject",
        "s3:GetBucketAcl"
      ],
      "Resource": "arn:aws:s3:::client-${AccountId}-cost-export/*"
    }
  ]
}

```

3. Correct CloudFormation Template
The CloudFormation template appears to be mostly correct. However, ensure that the AccountId parameter is being substituted correctly. Here’s a slightly modified version of your template:

```

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  AccountId:
    Description: AccountId
    Type: String

Resources:
  CostAndUsageReport:
    Type: AWS::CUR::ReportDefinition
    Properties:
      AdditionalArtifacts:
        - REDSHIFT
        - ATHENA
      AdditionalSchemaElements: 
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: true
      ReportName: cost-usage-export
      ReportVersioning: CREATE_NEW_REPORT
      S3Bucket: !Sub client-${AccountId}-cost-export
      S3Prefix: cur
      S3Region: us-east-1
      TimeUnit: DAILY
```
4. Region Specific Considerations
Ensure that the S3Region specified in the CloudFormation template (us-east-1 in this case) matches the region where the S3 bucket is created. If the bucket is in a different region, you need to update this parameter accordingly.

5. Debugging Further
If the above steps do not resolve the issue, you can enable detailed logging for AWS CloudFormation to get more insights into what might be going wrong. You can check the CloudFormation stack events and logs in the AWS CloudFormation console to get more details about the error.

Also reviewing  detailed CloudFormation stack events and logs will help pinpoint the exact issue.

CloudFormation: CUR ValidationException - what is wrong?

Relevant content