AWS Config & AWS Fargate's Dynamic Nature

Question

**Context**
Recently I came across a scenario where, due to rolling out Fargate infrastructure and [ Fargates dynamic task networking nature ](https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-networking.html) it resulted in a significant increase in cost due to a near-continuous stream on Configuration Item updates for the `AWS::EC2::Subnet` Config Resource.

The nitty gritty seems to be attributed to the following:
* As seen in the above resource Fargate will assign an ENI to each task (1:1 relationship) with a primary Private IP address.
* The impact on the `AWS::EC2::Subnet` resource Configuration Item is that: (1) the `Configuration.AvailableIpAddressCount` will change as a result and (2) the `Relationships` metadata will change to be updated with the most current ENI information, which is again 1:1 depending on the number of tasks you have running at any point in time.

When you have a high volume task operating context this results in a significant increase in cost (see the [Config Pricing calculator](https://aws.amazon.com/config/pricing/) for Configuration Item pricing) if you don't keep your eye on things.

**Response**
In response to this, instead of tracking all resource types by default in AWS Config, I explicitly specified which resource types to include which did not include the `AWS::EC2::Subnet` resource type. This seems to have mitigated the issue from the standpoint of "stopping recording" but it is not ideal because I'd like to be able to track resource changes to Subnets and action them as needed based on findings.

**Questions**
* Has anyone else come across this issue? I'm wondering if maybe I''m overlooking a setting somewhere to tune this such that it reasonably controls cost.
* Is this a bug or are feature requests open somewhere with AWS to find a solution to this? I haven't found anything yet but very well could have missed something.

Michael_R · Answer

Hi Matt -

I can confirm that this is an expected behavior for both Fargate and Config.

As you note, Fargate creates Task ENIs in the subnets, triggering Configuration Items for the ENI to Subnet association and Available IP addresses. [1]
AWS Config pricing is currently $0.003 per config item recorded. [2]

Depending on the volume of tasks you are executing, this can drive a significant number of config items & corresponding AWS Config cost.

Your workaround of disabling AWS Config is appropriate, though as you note is not ideal as you lose change tracking for subnets. AWS Config allows for selecting resource types, however at this point in time it's not possible to get more granular than resource type.

To compensate for the reduced visibility, you could implement additional controls, for example by ensuring you have CloudTrail logs enabled & query capability against those logs, via [Athena ](https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html) [3] or [CloudTrail Lake](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-lake.html) [4].

Given that you have a significant volume of tasks running, it might be worthwhile to evaluate alternate architectures.

The first possibility would be to switch to EC2 tasks rather than Fargate tasks. EC2 based tasks have multiple task [networking options](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-networking.html) [5], including `bridge` and `host` modes. You'll need to consider the security implications of switching these modes and if that's acceptable for your use cases.

Another thought would be to adjust the task architecture to pull tasks from a queue or other input and process multiple tasks inside a single execution. This would reduce the task ENI churn and corresponding Config records.

Alternatively, if your task executes fairly quickly (<15 minutes), you could consider executing the container in Lambda instead of Fargate. Lambda uses [Hyperplane ENIs](https://docs.aws.amazon.com/lambda/latest/dg/foundation-networking.html) [6], which can share network interfaces
across functions, reducing the churn and corresponding Config records.

---
1. https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-networking.html
2. https://aws.amazon.com/config/pricing/
3. https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html
4. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-lake.html
5. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-networking.html
6. https://docs.aws.amazon.com/lambda/latest/dg/foundation-networking.html

AWS Config & AWS Fargate's Dynamic Nature

Relevant content