Why does the CLI return a "Rate Exceeded" error when reserved concurrency is "1"?
We have a Lambda function that is limited to Reserved Concurrency = 1.
When we invoke it via the AWS CLI, we receive the below error indicating the Invocation Rate has been exceeded.
aws lambda invoke --function-name func-name --invocation-type RequestResponse response.json
An error occurred (TooManyRequestsException) when calling the Invoke operation (reached max retries: 4): Rate Exceeded.
When I check the CloudWatch logs for the function, I see that only one invocation happened, but the CLI response doesn't indicate that success.
Is there any way to guarantee that the successful invocation is what is returned by the CLI invoke request?
- Newest
- Most votes
- Most comments
I'd suggest you to check error logs for this function and see what's causing this lambda function failure. From the metrics, it's evident that lambda function was invoked first time and errored out but that has nothing to with TooManyRequestsException.
For the error you mentioned, please find below the explanation
Since you have set reserved concurrency as 1, so only one invocation can take place for it's running duration and if you attempt to invoke it again while previous invocation is already running, you may see this message.
See Reserved concurrency, where it says,
-
The reserved concurrency is deducted from the overall capacity for the AWS account in a given Region. The Lambda function always has the reserved concurrency available exclusively for its own invocations.
-
The reserved concurrency restricts the maximum number of concurrent invocations for that function. Synchronous requests arriving in excess of the reserved concurrency limit will fail with a throttling error.
Please refer concurrency discussion in this Lambda FAQs.
Lastly, The Lambda service scales up to consume the unreserved concurrency of 100, and then throttling occurs. Please check account level lambda metrics and see if that's causing the problem. Please refer Identifying and managing throttling.
Hope you find this information helpful.
Comment here if you have additional questions, happy to help.
Abhishek
Relevant content
- asked a year ago
- Accepted Answerasked 2 years ago
- asked 2 years ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 months ago
Hi Abhishek - The Lambda function itself didn't fail, the log stream shows a success! Our concern is the result of the CLI invocation returning an error code, as we don't want to have to log in to the console and query the logs to determine if the invocation was successful or not.
The single synchronous CLI invocation is the only trigger for the function (no other integrations), so there are no other running invocations to interfere.
I understand that you want this to be restricted to only one invocation, did you try increasing the reserved concurrency to higher value. Also as I mentioned, I understand that there any not be any invocations for this functions but what about other lambda functions in the same account. Did you check the account lambda metrics.