Eventbridge with Lambda target, not pushing to DLQ when lambda is throttled

Question

Hello,

I might be misunderstanding how this is supposed to work, but I have an eventbridge rule with a lambda target configured and everything works perfectly. I also set a DLQ on the eventbridge trigger.

My understanding of how that works is when an event fails to send, after the event bridge has exhausted its own retry policy (configured to 1 hour and 0 retry attempts) it will push the event to the DLQ I've configured.

However when turning off the lambda which is configured as the target (Setting the throttle on the lambda to 0) I'm not seeing any events coming into my DLQ even after 24 hours.

Does a throttled lambda not count as a failed event send request? How can I further debug the issue here?

Kind regards

Answer

To understand this better, I would encourage you to read this document - https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

Eventbridge is one of the lambda event sources that invokes lambda asynchronously, which implies that unless the lambda service itself is down, eventbridge will deliver the event in the lambda service's event queue.

If the lambda service cannot start a lambda instance because of throttling or any other reason, those events will stay in the event queue of the lambda service. As far as Eventbridge is concerned, it has already delivered the event to the lambda service.

What you need to do to handle your scenario is to also configure an "On Failure" lambda error destination DLQ, as explained in the same document.

Eventbridge with Lambda target, not pushing to DLQ when lambda is throttled

Contenus pertinents