Long running event processing

0

I am planning to build a long-running event processing system. I already have EC2 instances for my application and want to utilize the same compute to process long-running events. I am planning to use SQS with higher visibility timeouts and if the processing goes beyond that limit, I will change the message visibility time while processing. Can this pattern be used for scheduled events from the event bridge (do we have a concept of visibility timeout for event bridge events or do we have to send those to the SQS to get the same pattern)? Also, curious to know about ECS as a long-running compute for such events in the future (if I want to separate the compute from my existing ec2 compute), do we have to follow the same change in visibility timeouts for ECS as well if so?

1 個回答
1
已接受的答案

I think you've answered all of your questions yourself (nice!) but to confirm:

Your thoughts around SQS and visibility timeouts are correct. What compute platform you use there (EC2 or ECS) is not crucial - it's the same logic working the same way.

EventBridge doesn't work the same way as SQS - there is no concept of event visibility in the sense that SQS has message visibility. But you could use EventBridge to schedule a message to be sent to SQS.

profile pictureAWS
專家
已回答 2 年前
profile picture
專家
A_J
已審閱 15 天前
  • I also stumbled upon AWS Batch. I think I can use my existing compute with AWS Batch. Do you think AWS Batch would be a better fit both in terms of better dev experience (not using the SQS pattern with custom de-dup for effective one-time processing, etc) and cost?

  • Perhaps. You might also consider Step Functions.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南