Config for lambda internal queue batch size
have a query around lambda trigger notifications: sources i am referring to:
- https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
- https://docs.aws.amazon.com/lambda/latest/dg/with-sqs-example.html
Observations: though trigger from S3 has field records a list in the payload but it has only one record while sqs payloads have a no of events as per the batch size
Query: Couldn’t find if any aggregation happens in lambda’s internal queue, and in case there is any operation happening in the lambda’s internal queue, how can we config the batch size for the internal queue? Add is the batchSize always 1 in case of S3?
Although I can't find any document stating that the batchSize will always be 1 I think it will be.
Nevertheless usually when I implemented these lambda's I used one of the following patterns:
- just create a loop over the records. just in case it changes in future...
- refuse events that have > 1 record count, generate an error -> cloudwatch logs -> alarm
Only Event source mapping (SQS, Kinesis, MQ, DDB Streams) invocation allow for batching. All other invocation, synchronous (e.g., API Gateway) or asynchronous (e.g., S3, even though there is an internal queue), deliver the events to Lambda with a single event at a time.
If you want to handle S3 events in a batch you can use S3 -> SQS -> Lambda.
Relevant questions
Config for lambda internal queue batch size
asked a month agoEnable Lambda trigger
Accepted Answerasked 3 months agoRecommended Lambda function size
Accepted Answerasked 3 years agoHow do I query X-ray using a http URL, where that URL is not the first part of the request?
asked 2 months agoTrigger AWS DeepLens Lambda
asked 3 years agoPolling using Lambda
Accepted Answerasked 7 months agoWorkaround for Lambda 15 minute timeout
Accepted Answerasked 2 years agoRunning Ansible in aws lambda
asked 18 days agoUsing MSK as trigger to a Lambda with SASL/SCRAM Authentication
asked a month agoWhat is the maximum size for a lambda docker image ?
Accepted Answerasked a year ago