How can we create thousands of Lambdas, beyond quota limits?

0

Hi everyone,

For our requirement we need to create thousands of Lambdas for our customers. Currently AWS Lambda quota limits allow only 300 (max 600) Routes and 600 API's per region. I wonder how Vercel (which is deploying millions of Lambdas for their customers) is doing it on AWS ?

Any suggestions or information would be highly appreciated. Thank you

2 Answers
0

I can't speak for the example you give but in general the answer is: Use multiple accounts.

While we can increase the limits for some services well beyond what the defaults are, there will be side-effects. For example, having so many Lambda functions in a single account means that the concurrency limits will affect how many of those can be run at once and that (in turn) affects how you scale. Note that this is true of any account running many workloads - they can potentially get in each other's way as the limits are reached.

Again, limits can be increased but at some point it's better to be running those workloads in separate accounts so that they can scale independently of each other.

Account separation also means that you can delegate administration duties much easier, rather than trying to scope IAM policies to individual Lambda functions.

Is there overhead in running multiple accounts? Absolutely. But at the scale you're talking (thousands of functions) I would assume that you have a level of automation for those functions; and therefore you can automate the cross-account activities as well.

profile pictureAWS
EXPERT
answered 2 years ago
  • Hi, Thank you for such detailed comment. First thing I already spoke and requested for quota increase, but customer support said 600 is the max.

    The main issue with multilple accounts is automation and managment. I want to make it fully automatic. And it is very difficult to have hundreds of accounts(assuming separate account for each user). That will difficultly in resource managment and billing aswell.

  • I agree: there is a level of complexity when running multiple accounts. In the long term that complexity is far less than the challenges faced by running everything in one account. It's worth going down that path - a little hard work now means a lot less pain later. In my opinion, billing becomes easier: If you split the accounts across various subsystems it's now simple to allocate costs based on how much each account is consuming.

0

There is facility to request for increase in quotas under account in the console. See if you can request for what you want. If not, reach out to support and get their guidance.

answered 2 years ago
  • Hi, I requested for quota increase and spoke with customer support aswell, they said they cant increase beyond 600. Thank you.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions