- Newest
- Most votes
- Most comments
The error message "An error occurred and the request could not be processed" with type "Service" is likely related to capacity issues when your Lambda function is experiencing high concurrency. Since this happens when you exceed approximately 200 concurrent executions, there are several potential causes and solutions to consider:
-
Insufficient capacity error: AWS might not have enough available compute capacity to execute your function at that moment, especially at higher concurrency levels. This can happen even if your function worked fine previously at higher concurrency.
-
EFS connection issues: Since your Lambda is connected to EFS, there could be mounting or connection issues when many concurrent executions try to access the file system simultaneously. Even if the EFS itself isn't saturated, the connection process might be experiencing issues.
-
Concurrency throttling: While you mentioned there's no restriction on concurrency, Lambda might still be throttling requests due to account-level concurrency limits or service quotas.
To address this issue, you could try:
- Increase the function's memory allocation, which might provide access to different hardware resources
- Implement retry logic in your client code to handle these temporary capacity issues
- Set up reserved concurrency for your function to ensure a specific amount is always available
- Consider distributing your workload across multiple regions
- Check if your function is timing out during initialization or execution
- Review CloudWatch metrics for throttling, errors, and invocation metrics
- Ensure your function code gracefully handles duplicate events and has proper error handling
If the problem persists after trying these solutions, it might be worth contacting AWS Support to investigate if there are any underlying issues specific to your account or the regions you're using.
Sources
Insufficient capacity error when triggering a Lambda function | AWS re:Post
Troubleshoot invocation issues in Lambda - AWS Lambda
How Lambda handles errors and retries with asynchronous invocation - AWS Lambda
Based on your description, I suspect that is is tied to a concurrency setting/limit in your account, especially when occurring regularly at a very round number like 200. Look deep at those settings and the settings for concurrency of the other Lambda functions in the same account.
Relevant content
asked 3 years ago

I wanted to follow up on this. The only way I found to solve this issue was by throttling my calls to the lambda function. I found that a delay of 4ms between calls to the lambda function seemed to work well and I was able to test it up to 500 concurrent calls.
I am only using 200 in production so this is good.
I'm not really happy with this solution, especially as the routine can often require 100k calls to be completed so obviously time added by this throttling is not ideal.
I have been a bit surprised by this problem as I had never seen it happening before ( ie. a few months back at least when we last altered the number of concurrent calls) and I've not been able to find any info on AWS about max calls/sec etc to the lambda functions.
It is working for now but I will probably need to come back and figure out a more intelligent way to throttle ( potentially in batches or growing delay times etc). I'm not really sure.
If anyone has any ideas on this or can point me to some AWS info regarding this issue then it would be appreciated.