After further investigation, it is indeed related with my filter but not the one in my example. I also have another lambda function consuming messages from the same queue with a different filter. I believe this is what was happening:
Lambda Event Filter deletes messages from the Queue when the message do not match the filter criteria. If a queue has multiple lambda consumers with different event filters, there is a data loss when a filter which would match only one consumer was deleted by the other one because of a filter mismatch. It was hard to debug as there is no clear indication of who removed a message and why.
How did you determine that Lambda has fewer invocations than SQS messages?
If you look at the Lambda CloudWatch logs, are there any logs that were processed for SQS messages?
The most obvious suspect is the filter that you use. Could it be that not all message match the filter? If this is the case, messages that do not fit the filter are discarded and the function is not invoked.
Another, not so likely, suspect is the Delay param. When you send a message it will not be available for consumption for 10 seconds. Could it be that you did not wait enough time to see if the function is invoked?
- Accepted Answerasked 10 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated 5 months ago
- Why is my Lambda function retrying valid Amazon SQS messages and placing them in my dead-letter queue?AWS OFFICIALUpdated a year ago
- EXPERTpublished 9 months ago
- EXPERTpublished a year ago