Messages did not get processed by the lambda when the sqs acts as event source

Question

* I have two lambdas and one sqs fifo queue. 
* First lambda was used to add all the messages to the queue. 
* And when the message meets certain pattern in the sqs queue, the second lambda will be trigged to process all the filtered data in the queue.

BUT, if I send multiple messages into the queue, say 4 messages in 3 second, then very likely only 2 messages will get processed by the second lambda, although all the 4 messages meet my filter pattern.

Currently I changed the batch size to 1, just in case, but nothing changed.
Don't know why it happened, it looks like the lambda is very unstable. Any help will be appreciated.

Answer

Ok so there might be possibility of have two consumers over here like two functions added to the queue to consume the message. when you have multiple consumers added to the SQS then message gets polled from the queue. Now once a message is processed by first lambda function then lambda service will delete the message from the queue which is why the other function is not getting the other message.

Suggestion would be to use sqs sns fanout option where you will have two queues for each lambda functions and queues will be added as destination/subscriber to an SNS topic. So you will have to send message to the topic which will get forwarded to both the queues and then to the lambda functions.

Messages did not get processed by the lambda when the sqs acts as event source

관련 콘텐츠