Skip to content

High AWS GuardDuty cost due to Step Functions logging to CloudTrail

0

I'm investigating a high cost for AWS GuardDuty. The GuardDuty "Usage" page shows me that 90% of the GuardDuty cost comes from CloudTrail.

I've turned off all Trails in AWS CloudTrail. The AWS CloudTrail cost is zero.

If I look at the CloudTrail -> Event History page, I see the vast majority of events there are from AWS Step Functions, which we use extensively. Lots of SendTaskHeartbeat, SendTaskSuccess, SendTaskFailure. These events are pretty useless for the purposes of Intrusion detection.

The GuardDuty cost is roughly 44% of the Step Function cost, and is effectively an added "tax" on the service without adding any benefit. I would like to prevent GuardDuty from examining the AWS CloudTrail events from State Machines, but I don't see any way to do that in either CloudTrail or GuardDuty. I could disable GuardDuty completely, but I would like to keep it for the other monitoring it does.

3 Answers
0

Hi,

Given your high use of Step Functions and Lambdas, a large chunk of Guard Duty costs may come from the Lambda protection feature of GD: https://docs.aws.amazon.com/guardduty/latest/ug/lambda-protection.html

Did you try to deactivate it specifically to see if it solves your cost issue?

Best,

Didier

EXPERT

answered 2 years ago

EXPERT

reviewed 2 years ago

  • Didier, thanks for your taking a look at this. However we do not use Lambda for hardly anything, so I'm confident that isn't contributing to the cost. All of the work is done on an EC2 instance.

0

Hi bchecketts,
As you've already determined, Task Token events fall under AWS CloudTrail management events. CloudTrail management events are a foundational data source for GuardDuty, and currently, there are no granular controls to filter out specific CloudTrail management events. If you have Enterprise Support, then it's worth reaching out to your AWS account team to check if this feature is on the roadmap for GuardDuty.

In the meantime, it's recommended to dive into your CloudTrail logs to determine the dominating event sources, and review whether their components need to call at this frequency.

Best regards,
Randy

AWS

answered 2 years ago

0

Thanks Randy, You are correct that the Task Token events are counted as Management Events. This seems like a misclassification to me as things like SendTaskSuccess, SendTaskHeartbeat, and SendTaskFailure occur during normal use of the service and are not used in "Managing" the service.

If anybody from the @AWSStepFunctions team is paying attention I would also argue that CreateActivity, GetActivityTask and DeleteActivity are similar in that they occur during normal use of the services.

These Step Function activities are all more akin to the DynamoDB operations like GetItem, PutItem, Query, and Scan where you are working with the service rather than organizing or managing it. Or the S3 GetObject and PutObject operations, again used during normal, everyday use of the service.

For DynamoDB and S3, these are classified by CloudTrail as Data Events rather than Management events. It does looks like the Step Function InvokeHTTPEndpoint event is classified as a Data Event

There really is no way to lessen the number of these API calls being made, as they are part of the operation and use of the service.

Unfortunately, we're fall too small for Enterprise support, so this likely won't get any attention. We'll probably just pay the extra "AWS Tax" since it is in the hundreds of dollars a month range.

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.