Unable to delete CloudFormation added security group because of added elastic network interfaces not deleting

0

I have a CloudFormation template that installs a number of lambda functions as well as a security group that they use. I am to install the template but when I come to delete it, the process fails. It is unable to delete the security group because there are two ENIs still attached.

I do not explicitly add the two ENIs in my template. They get added as part of the process. I know this because their description is generated and includes part of the lambda name.

Strangely, there are no ENIs associated with the other installed lambdas. (There are other ENIs installed but they get removed). The only difference between this lambda and the others as far as I can see is that this one is connected to an event bridge schedule (which is also created and removed by the template)

Has anybody encountered this before?

As a temporary workaround I remove the security group from the ENI and am able to delete the CloudFormation stack. I then have to delete those ENIs separately. This is not ideal as this is a solution that we are delivering to our customers.

2 Answers
0

Hello.

Does this mean that the ENI used by Lambda does not disappear when Lambda is connected to VPC?
I haven't looked at the CloudFormation template, so I don't know the exact details, but is it possible that the Lambda IAM role disappears first, making it impossible to delete the ENI?
I have experienced this in the past, but I failed to delete the ENI because the Lambda IAM role disappeared first and I did not have the authority to delete the ENI.

By the way, normally the ENI disappears within 20 minutes.
https://repost.aws/knowledge-center/lambda-eni-find-delete

If the output lists no other functions or function versions, then Lambda automatically deletes the network interface within 20 minutes.

profile picture
EXPERT
answered 3 months ago
  • The problem is that CF does not delete because it cannot delete the security group. The lambda is deleted. I suppose it may be a case of the lambda being deleted so that the ENIs cannot be deleted (because they are still pointing to the lambda) and because the ENIs are not deleted the security group cannot be deleted.

    I don't really know why the ENIs are created in the first place to be honest. I can only think that they are needed because of the link the the Event Bridge scheduler.

  • No ENI will be created unless you connect Lambda to your VPC. Do you have a requirement to connect AWS resources (such as EC2 or RDS) in a VPC from Lambda? If you don't have these requirements, you don't need to connect Lambda to your VPC. If you delete the "VpcConfig" part, Lambda will no longer connect to the VPC. https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html#cfn-lambda-function-vpcconfig

0

It's indeed not optimal interaction between Lambda and CloudFormation for the Lambda function's hyperplane ENIs in your VPC not to disappear when the function has finished getting deleted.

However, it is a general recommendation to consider connecting Lambda functions to VPCs, because in addition to giving access to resources within the VPC or connected networks, it makes it possible to connect to AWS APIs and the outside world from dedicated source IP addresses (such as the elastic IPs of NAT gateways or the VPC's IPv6 addresses) and via VPC endpoints. Doing so allows the Lambda function's credentials to be limited to being used from those public IPs or VPC IDs (when connecting through endpoints), making it mostly infeasible or impractical for potentially exfiltrated credentials to be exploited by outsiders.

You can work around your issue in a couple of ways. One option is to separate the security group into a separate stack, which the customer could delete separately after deleting the Lambda stack. You can declare the security group's ID as an export from the stack containing it, so that it can be cleanly referenced from the Lambda stack.

The other option would be to parameterise the single template so that creating the Lambda would be controlled by a parameter. By default, it would be enabled, causing the security group and Lambda to be created in the same way as before, but by disabling the selection parameter, the stack could be updated to delete the Lambda function while the security group remains. The whole stack would only be deleted after Lambda had enough time to delete its hyperplane ENIs.

EXPERT
Leo K
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions