AWS Lambda does not choose warm environments

Question

I have a script to keep two AWS Lambda functions warm:

```py
PING_DELAY = 300
last_ping = -PING_DELAY
while True:
    try:
        timer = Timer()
        timestamp_str = \
            datetime.now().strftime("%d.%m.%Y %H:%M:%S.%f")[:-3]
        print(f"Ping at {timestamp_str} ", end='', flush=True)
        response = lambda_client.invoke(
            FunctionName='myfunc',
            InvocationType = 'RequestResponse',
            Payload=json.dumps({'command': 'ping'}))
        payload = json.load(response['Payload'])
        if 'errorMessage' in payload:
            raise Exception(payload['errorMessage'])
        else:
            my_time = timer.stop()
            stats = payload['stats']
            print(f"took {my_time}ms.   n_cold: {stats['n_cold']}   total_init: {stats['total_init']}ms", flush=True)
    except Exception as e:
        print(f"AWS Lambda submit failed: {e}", flush=True)
    
    time.sleep(PING_DELAY)
```

Every five minutes, it invokes the `myfunc` AWS Lambda function, which in turn invokes in parallel 109 instances of another Lambda function. Based on the responses, `myfunc` reports in its response how many instances experienced a cold start (`n_cold`) among other measurements. Here is part of the output of this script:

```none
Ping at 28.11.2023 08:58:06.605 took 586ms.   n_cold: 0   total_init: 0ms
Ping at 28.11.2023 09:03:07.291 took 7133ms.   n_cold: 4   total_init: 24215ms
Ping at 28.11.2023 09:08:14.524 took 54809ms.   n_cold: 0   total_init: 0ms
Ping at 28.11.2023 09:14:09.433 took 10211ms.   n_cold: 13   total_init: 77080ms
Ping at 28.11.2023 09:19:19.744 took 694ms.   n_cold: 19   total_init: 118006ms
Ping at 28.11.2023 09:24:20.538 took 4617ms.   n_cold: 0   total_init: 0ms
Ping at 28.11.2023 09:29:25.185 took 2042ms.   n_cold: 0   total_init: 0ms
Ping at 28.11.2023 09:34:27.327 took 566ms.   n_cold: 4   total_init: 24450ms
Ping at 28.11.2023 09:39:27.993 took 602ms.   n_cold: 8   total_init: 47270ms
Ping at 28.11.2023 09:44:28.695 took 658ms.   n_cold: 11   total_init: 66350ms
```

As the output shows, at least 90 of the 109 environments remain warm at all times. For the actual computation that I am interested in, `myfunc` invokes in parallel 50 instances of that same Lambda function. Given the availability of at least 90 warm environments, I would expect that all the 50 instances would be run in warm environments. However, the logs show that some of those 50 invocations result in a cold start. I checked that the computation did not coincide in time with the invocations for keeping the environments warm.

Why does AWS Lambda not choose a warm environment when an abundant number of such environments is available?

P.S. The [question in SO](https://stackoverflow.com/q/77563018/2725810)

Answer

Lambda keeps environments warm for a few minutes after the invocation, so that if another request comes in, it will be a warm invocation. It will also recycle environments after some time, even if they are in the few minutes grace. This is done for various reasons. So it may well be that you warmed 50 instances, and some of them got to their recycle time and this is why you get cold invocations.

This is exactly the reason that we created Provisioned Concurrency. It makes sure that you have warm environments when you needs them. Using these warmers to keep environments is more, is more cost effective compared to Provisioned Concurrency, but it has limitations, as you can see for yourself. Another limitation is that when you try to warm them, you don't really know how many active environments are there, so you may be creating new ones. Same can happen when you get actual requests, and they can't find a warm environment, because they are currently being invoked by the warmer.

AWS Lambda does not choose warm environments

Relevant content