Why isn't my Lambda function with an Amazon SQS event source scaling optimally?

Resolution

Note: When running optimally, Lambda functions with an Amazon SQS queue configured as an event source can scale up to 60 more instances per minute. The maximum number of concurrent invocations is 1,000. If you're using FIFO event source mapping, then functions can scale in concurrency to the number of active message groups. For more information see, Scaling and processing.

Identify and resolve any Lambda function invocation errors

To prevent errors happening at scale, Lambda throttles function scaling when invocation errors occur. When the errors are resolved, Lambda continues to scale the function. For more information, see Backoff strategy for failed invocations.

For best practices on how to identify and resolve Lambda function invocation errors, see the following:

• Troubleshooting issues in Lambda
• How do I troubleshoot Lambda function failures?

Confirm that your Lambda function is configured with optimal concurrency for your use case

Reserved concurrency

If you configured reserved concurrency on your function, then your function is throttled when it reaches the reserved value. It's a best practice to make sure that the amount of concurrency reserved for your function is at least the following:

• For standard Amazon SQS queues: 1,000
• For FIFO queues: At least the number of active message groups or higher

Provisioned concurrency

If you haven't configured reserved concurrency on your function, then your function has a default unreserved concurrency (provisioned concurrency) quota of 1,000. This default quota is shared with other functions in the same AWS account and Region. If there's at least 1,000 provisioned concurrency available in your function's Region, then the function continues to scale until it reaches the maximum concurrency. Invocations are throttled when all of your provisioned concurrency is in use.

For more information, see Configuring reserved concurrency.

Important: To scale up additional concurrent invocations, your account must not be near the service quota for scaling or burst concurrency in the Region. If you need a higher concurrency for a Region, request a service quota increase in the Service Quotas console.

Confirm that there are enough messages in your Amazon SQS queue to allow your Lambda function to scale

If an Amazon SQS queue is configured to invoke a Lambda function, then Lambda will scale invocations only if there are messages in the queue.

To check how many messages in your Amazon SQS queue still need to be processed, review your ApproximateNumberOfMessagesVisible metric.

If the metric is low or at 0, then your function can't scale.

If the metric is high and there are no invocation errors, then try increasing the batch size on your event notification. Increase the batch size until the duration metric increases faster than the batch size metric. For more information, see Monitoring functions on the Lambda console.

Note: The maximum batch size for a standard Amazon SQS queue is 10,000 records. For FIFO queues, the maximum batch size is 10 records. For more information, see ReceiveMessage in the Amazon SQS API Reference.

Related information

Using Lambda with Amazon SQS

Managing AWS Lambda function concurrency

Configuring maximum concurrency for Amazon SQS event sources