Creating a Data Protection policy for a CloudWatch log group using a custom Data Identifier

Question

I have a json field with sensitive information getting logged to a cloudwatch group. I would like to censor this field using the Data Protection policies on cloudwatch, however this it doesn't match any existing data identifier.

I created a custom data identifier in AWS Macie, but It wasnt listed on the UI's drop down of available data identifiers.  I tried the syntax identifier editor and added the new custom identifier's arn, but got an error saying

```
An error occurred saving the data protection policy.
arn:aws:macie2:us-east-1:xxxxxxxxxxx:custom-data-identifier/41f47215-e263-45c3-8ca0-d0e7bab7788f is not a valid Data Identifier
```

Am I assuming these services are connected when they really arent?

These logs are created from a POST request to a rest api built in Api Gateway set to log errors only, and is in us-east-1.

Any guidance or alternatives would be appreciated.

Answer

Hi,

I understand you would like to know how the data protection policies used by cloudwatch logs does not match any existing custom data identifier that you have created.

First, confirm that the JSON properties for the data protection policy is in this format:
{
  "Name": "CloudWatchLogs-PersonalInformation-Protection",
  "Description": "Protect basic types of sensitive data",
  "Version": "2021-06-01",
  "Statement": [
        ...
  ]
}

Alternatively, to select the sensitive data of interest in the json field, trying using managed data identifiers. According to the documentation, it mentioned specifically about CloudWatch Logs managed data identifiers for sensitive data types. Also, you can use the supported identifiers listed in the "CloudWatch Logs managed data identifiers for sensitive data types" as keywords as considerations for the custom data identifiers.

I hope this helps. Let me know if I answered your question by up voting my response or if you have any follow-up.

Kind regards, 
Ahmed

References: 
[1]  https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch-logs-data-protection-policies.html
[2] https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/protect-sensitive-log-data-types.html#CWL-managed-data-identifiers

Creating a Data Protection policy for a CloudWatch log group using a custom Data Identifier

Relevant content