My event bridge rule creates two duplicate messages
We have a lambda function that is triggered by EventBridge.
This EventBridge is used for the salesforce event - the event bus
So when a user creates an order, SF will generate an event for the order and send it through the event bus. Then AWS event bridge - will get it and invoke a lambda function.
All of a sudden, Event Bridge has been sending the same orders twice consecutively to Lamdda. Therefore, our orders have always been duplicated.
We can check this happen in our cloud watch logs and order logs as well. they have a different requestID but inside, they are the same order.
- Newest
- Most votes
- Most comments
It's strange, but all of a sudden, all processes back to normal after 2 days of duplication. now, I am starting idempotency on Eventbridge.
EventBridge is a distributed service intended to ensure delivery. It makes the choice of potentially delivering more than once versus losing an event: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-troubleshooting.html#eb-rule-triggered-more-than-once
A best practice is to ensure the events are idempotent.
If it is happening repeatedly, then the likely source is the invocation of the event. You can look in CloudTrail and confirm that it is being invoked more than once.
Hello,
I understand that the service is working normally now.
If the problem arises again or to identify the root cause of the issue , I would recommend you to reach out to AWS Premium Support by raising a Support Case using the same account with all the information [+]. Kindly note that to identify root cause of the issue we require more information from your end which cannot be shared over public platform.
[+] https://console.aws.amazon.com/support/home#/case/create
Thank you.
Relevant content
- Accepted Answerasked 10 months ago
- asked 2 months ago
- Why can't I view the EventBridge trigger in my Lambda function that my CloudFormation stack created?
AWS OFFICIALUpdated a month ago - AWS OFFICIALUpdated 9 months ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated 2 years ago
Thanks for the answer, my case is more complex - I could not define where this came from, but I will explain a bit more:
While processing #2, even if we got one order event message, in the lambda log, we could see it runs twice, creating two same orders in ERP. Also, sometimes, when we check the log from the event bridge, it appears to have two messages at the same time as the attached pics, but it doesn't matter how many messages we find in the log; invoking the same event is always more than twice.
In case we create a new rule and disable the current rule, it will always create a duplicate message per rule, no matter if it's been disabled or enabled. For instance, three rules (two disabled) create 3 same messages.