Unable to create new OpsItems from EventBridge when using Input Transformer for deduplication and adding category and severity values

0

Apologize to all for the duplicate post. I created my login under the wrong account when I initially posted this question.

I’m able to generate a new OpsItem for any EC2, SecurityGroup, or VPC configuration change using an EventBridge rule with the following event pattern.

{ "source": "aws.config", "detail-type": "Config Configuration Item Change", "detail": { "messageType": "ConfigurationItemChangeNotification", "configurationItem": { "resourceType": "AWS::EC2::Instance", "AWS::EC2::SecurityGroup", "AWS::EC2::VPC" } } }

The rule and target work great when using Matched event for the Input but I noticed that launching one EC2 using the AWS wizard creates at least three OpsItems, one for each resourceType. Therefore I’d like to implement a deduplication string to cut down on the number of OpsItems generated to one if possible and I’d also like to attach a category and severity to the new OpsItem. I’m trying to use an Input Transformer as recommended by the AWS documentation but even the most simplest of Input Transformers when applied prevent any new OpsItems from being generated. When I've tested, I've also ensured that all previous OpsItems were resolved. Can anyone tell me what might be blocking the creation of any new OpsItems when using this Input Transformer configuration?

Here’s what I have configured now.

Input path { "awsAccountId": "$.detail.configurationItem.awsAccountId", "awsRegion": "$.detail.configurationItem.awsRegion", "configurationItemCaptureTime": "$.detail.configurationItem.configurationItemCaptureTime", "detail-type": "$.detail-type", "messageType": "$.detail.messageType", "notificationCreationTime": "$.detail.notificationCreationTime", "region": "$.region", "resourceId": "$.detail.configurationItem.resourceId", "resourceType": "$.detail.configurationItem.resourceType", "resources": "$.resources", "source": "$.source", "time": "$.time" }

Input template { "awsAccountId": "<awsAccountId>", "awsRegion": "<awsRegion>", "configurationItemCaptureTime": "<configurationItemCaptureTime>", "resourceId": "<resourceId>", "resourceType": "<resourceType>",

"title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "<detail-type>", "source": "<source>", "time": "<time>", "region": "<region>", "resources": "<resources>", "messageType": "<messageType>", "notificationCreationTime": "<notificationCreationTime>", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{"dedupString":"ConfigurationItemChangeNotification"}" } } }

Output when using the AWS supplied Sample event called “Config Configuration Item Change” { "awsAccountId": "123456789012", "awsRegion": "us-east-1", "configurationItemCaptureTime": "2022-03-16T01:10:50.837Z", "resourceId": "fs-01f0d526165b57f95", "resourceType": "AWS::EFS::FileSystem",

"title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "Config Configuration Item Change", "source": "aws.config", "time": "2022-03-16T01:10:51Z", "region": "us-east-1", "resources": "arn:aws:elasticfilesystem:us-east-1:123456789012:file-system/fs-01f0d526165b57f95", "messageType": "ConfigurationItemChangeNotification", "notificationCreationTime": "2022-03-16T01:10:51.976Z", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{"dedupString":"ConfigurationItemChangeNotification"}" } } }

1 Answer
0
Accepted Answer

Hi, and thanks for reaching out!

To look in to this question, I set up a similar workflow as yours: 1. EventBridge rule to detect "Config Configuration Item Change" events from the Config service, using your same event pattern. 2. Target is a Systems Manager Ops Item 3. Target configured with Input Transformer to send data formatted as you have specified.

I was able to replicate the same behavior you described: in my first few tests without utilizing the input transformer, the Ops items were created correctly (I triggered the rule by just modifying a Security Group). But when implementing the Input Transformer configuration you supplied, I ran in to the same blocker where the Ops Item was not created, and the rule reported a Failed Invocation.

I set up a DLQ on the EventBridge rule target, and found that the Failed Invocation was due to the following error being thrown:

'ERROR_MESSAGE': {'stringValue': 'Invalid input for target.'}

Checking the required parameters for the CreateOpsItem API, it turns out the Ops Items are failing to create due to the parameters being passed by the event rule. Creating Ops Items requires a specific set of parameters, some of which were being supplied by your input transformer, but the rest of the fields being passed are not supported as part of a CreateOpsItem call.

Without the Input Transformer, it looks like the raw event data just gets passed as the "operationalData" field. But when using the Input Transformer, it appears to be necessary to specify the correct expected fields.

To test this, I stripped the Input Paths down to just fields expected by the CreateOpsItem call's top level fields. As well, to be sure you can pass other custom event data as part of the Ops Item, I used other variable event data as Operational Data keys. Note: Operational data keys can't begin with the following: amazon , aws , amzn , ssm , /amazon , /aws , /amzn , /ssm, so I changed them to simply "account" and "region"

Input Path:

{
  "awsAccountId": "$.detail.configurationItem.awsAccountId",
  "awsRegion": "$.detail.configurationItem.awsRegion",
  "configurationItemCaptureTime": "$.detail.configurationItem.configurationItemCaptureTime",
  "detail-type": "$.detail-type",
  "messageType": "$.detail.messageType",
  "notificationCreationTime": "$.detail.notificationCreationTime",
  "region": "$.region",
  "resourceId": "$.detail.configurationItem.resourceId",
  "resourceType": "$.detail.configurationItem.resourceType",
  "resources": "$.resources",
  "source": "$.source",
  "time": "$.time"
}

Input Template:

{
	"title": "Template under ConfigDrift-EC2-Dedup4",
	"description": "Configuration Drift Detected.",
	"category": "Security",
	"source": "EC2",
	"severity": "3",
	"operationalData": {
		"/aws/dedup": {
			"type": "SearchableString",
			"value": "{\"dedupString\":\"ConfigurationItemChangeNotification\"}"
		},
		"account": {
			"value": "<awsAccountId>"
		},
		"region": {
			"value": "<awsRegion>"
		}
	}
}

Invoking the rule again with this Input Transformer configuration, the Ops Item was successfully created, including the dedupString as an OpsItem detail, and the account and region variables passed in the Operational Data fields.

If you wish to pass more of your original custom fields that are not expected top-level fields of a CreateOpsItem call, you can pass them as custom Operational Data key-value pairs as seen above.

Hope this helps!

AWS
SUPPORT ENGINEER
answered 2 years ago
  • The deduplication worked just as you advised in my environment but I ended up redesigning the EventBridge rules entirely and no longer needed the deduplication. However, the insights I gained from your extremely helpful response helped me write and troubleshoot the Input Transformers I needed for my new EventBridge rules. Thank you!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions