How to fix duplicate transactional email issue caused by multiple EC2 instances?

0

A business guy's description: customers get multiple email notifications about the same transaction.

Developer's description: We have a problem with the load balancer and scheduled cron jobs in the Elastic Beanstalk ".ebextensions"-folder. When the load balancer scales in, there are multiple EC2 - instances and every instance executes cron jobs defined in the .ebextensions - folder of the project. That causes problems with the notification email delivery and our clients can receive multiple notifications -> sent email count is the same as the number of active instances. We've already tried scale in protection and "leader_only" = true definition for commands, but those are not working.

What should we do to fix this problem correctly?

2개 답변
2

The "cloud native" way of doing this is to use a queue.

It's not detailed in the question but my guess here is that there is a single submission to the application which is intended to cause a single email email notification. But the submission is (probably) stored in a database somewhere and when all of the instances run their cron jobs they all pick up the job out of the database and all send emails.

Instead: Have the submission of the job create a message in SQS and have the email notification system retrieve it from SQS. Even if there are multiple instances running the notification code only one of them will ever receive the SQS message - that's how the service is designed.

profile pictureAWS
전문가
답변함 2년 전
profile pictureAWS
전문가
Chris_G
검토됨 2년 전
1

Hello,

Brettskis SQS Solution is probably the correct and more robust solution at scale. However, it could be a bit lite over-engineer for such an easy task.

Just create an endpoint in your service for "notifications" (the same function that your cron job triggered), then have a cron job in cloud watch event bridge trigger it using SNS, or go via a private lambda if it's a private endpoint. T

This way, only one request will be sent even if you have multiple EC2s, and it's easy to test and trigger add hoc "notifications". And you won't have to spend money on SQS pooling (keep in mind that SQS pooling blooks one thread in your machine).

Hope this helps!

//Carl

profile picture
답변함 2년 전
  • Given that SQS is free for the first million requests per month if this is a small-scale application then costs isn't going to be an issue. I run this exact solution for a personal website because it is inexpensive and it's really easy to engineer. Many AWS services are there to solve problems small and large. And SQS polling only blocks for the long polling period specified by the caller.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠