Why isn't my Lambda function with an Amazon SQS event source scaling optimally?

Resolution

Note: When you configure an Amazon SQS queue as an event source, Lambda functions can optimally scale up to 60 more instances per minute. The maximum number of concurrent invocations is 1,000. If you use FIFO event source mapping, then functions can scale in concurrency to the number of active message groups. For more information see, Scaling and processing.

Identify and resolve any Lambda function invocation errors

To prevent errors at scale, Lambda throttles function scaling when invocation errors occur. When the errors are resolved, Lambda continues to scale the function. For more information, see Backoff strategy for failed invocations.

For best practices on how to resolve Lambda function invocation errors, see Troubleshooting issues in Lambda and How do I troubleshoot Lambda function failures?

Configure your Lambda function with optimal concurrency for your use case

Reserved concurrency

If you configured reserved concurrency on your function, then Lambda throttles your function when it reaches the reserved value. Make sure that the amount of concurrency that's reserved for your function has at least the following values:

• For standard Amazon SQS queues: 1,000
• For FIFO queues: The number of active message groups

Unreserved concurrency

If you don't configure reserved concurrency on your function, then your function has a default unreserved concurrency quota of 1,000. This default quota applies to other functions in the same AWS account and AWS Region. If you have at least 1,000 unreserved concurrency available in your function's Region, then the function scales until it reaches the maximum available concurrency. When all of your account's concurrency is in use, Lambda throttles invocations.

Note: Lambda functions initially scale as per burst capacity.

If your expected traffic arrives quicker than the default burst capacity, then you can use Provisioned Concurrency to make sure that your function is available. With Provisioned Concurrency, the function continues to scale to a predefined value. After the provisioned concurrency is fully used, the function uses the unreserved concurrency pool of the account.

Important: To scale up additional concurrent invocations, your account must not be near the service quota for scaling or burst concurrency in the Region. If you need a higher concurrency for a Region, then request a service quota increase in the Service Quotas console.

Confirm that there are enough messages in your Amazon SQS queue to allow your Lambda function to scale

If an Amazon SQS queue is configured to invoke a Lambda function, then Lambda will scale invocations only if there are messages in the queue.

To check how many messages in your Amazon SQS queue still need to be processed, review your ApproximateNumberOfMessagesVisible metric.

If the metric is low or at 0, then your function can't scale.

If the metric is high and there are no invocation errors, then try increasing the batch size on your event notification. Increase the batch size until the duration metric increases faster than the batch size metric. For more information, see Monitoring functions on the Lambda console.

Note: The maximum batch size for a standard Amazon SQS queue is 10,000 records. For FIFO queues, the maximum batch size is 10 records. For more information, see ReceiveMessage in the Amazon SQS API Reference.

Related information

Using Lambda with Amazon SQS

Managing AWS Lambda function concurrency

Configuring maximum concurrency for Amazon SQS event sources