Error connecting to Secrets Manager from lambdas in VPC, connect ETIMEDOUT

0

We have a number of lambdas within a VPC that have recently started experiencing errors attempting to connect to and read SecretsManager secrets.

2024-01-22T23:46:24.786Z	6b8bbf99-4aa1-47f9-9e67-af91c0c227a0	ERROR	Could not lookup for secretName=xx/xx/xx on SecretManager.
2024-01-22T23:46:24.786Z	6b8bbf99-4aa1-47f9-9e67-af91c0c227a0	ERROR	11:46:24 PM data-ingestion-service::connect ETIMEDOUT 52.8.30.108:443

When attempting to read a secret that lambda hangs for about 6 minutes before eventually timing out. It was working fine up until late December, and I know of no reason why it started experiencing this. We have 2 other VPCs, built in the same manner, with essentially the same code that are not having this issue.

I tried adding a SecretsManager VPC Endpoint to see if that would help, but now I'm getting another error:

getaddrinfo ENOTFOUND secretsmanager.us-west-1.amazonaws.com

In this case, the lambda immediately returns the error.

Right now, I'm considering completely rebuilding the VPC and all resources to see if that helps. I've already rebuild everything, with the exception of the VPC, which i've been trying to avoid as it includes some manual steps, e.g., peering connections.

Our application is written in Node, running a number of lambda functions.

2 Answers
1

Hello.

I think the VPC endpoint will have the following format:
I think you will probably be able to communicate if you can successfully set up a VPC endpoint in your VPC.
Alternatively, I think you can set up a NAT Gateway.
https://docs.aws.amazon.com/secretsmanager/latest/userguide/vpc-endpoint-overview.html

com.amazonaws.us-west-1.secretsmanager
profile picture
EXPERT
answered 3 months ago
  • Thanks, I was using the VPC's NAT Gateway, but that seems to have stopped working, thus the attempt to use a VPC Endpoint, which, after configuring to use public subnets, now works.

0
Accepted Answer

I was able to get the application working by configuring the SecretsManager Endpoint to use the VPC's public subnets, previously I has used private.

It's still a mystery why the application now needs this, but at least it works.

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions