Is there an account-global connection limits in lambda that could cause timeouts?

Question

Hello,

I'm trying to debug a rather bizarre issue we've been dealing with that started about 5 days ago. We integrate with several google services, and we have lambdas both inside and outside of VPC. Since the 23rd we've been getting connection timeout across google services GCP, non-GCP, simple things like auth, and events collection. This is also using both direct HTTP calls, and using google's own SDKs with their transport layer.

Because it's affecting simple things on google's end like OAuth (across several GCP projects) and events collection (which doesn't use auth), I don't believe it's something happening on their end. *It's *almost* like AWS is not opening a connection for ~2 minutes at times across all of our lambda functions to Google's network (hosts are different, but IPs are Google's). Non-google calls do not seem to have been impacted, which are sometimes happening on the same lambda.

Any advice would be helpful.

Answer

Hi,

Lambda service has various quotas: https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

You should check the current values for your account via AWS Console and ask for raises when needed.

The one that comes first in mind is concurrent executions: default is at 1000 but it can be raised much higher on request

Best.

Didier

Answer

I would run some test from a different machine (not Lambda) the same time of the failures to check if the issue is on the GCP side or not. I would run something from your Laptop and also maybe from some EC2 instance.

Is there an account-global connection limits in lambda that could cause timeouts?

Relevant content