Data loss or duplication on Cloudtrail LookupEvents API StartTime parameter

0

I need to use LookupEvents API provied by Cloudtrial to periodically fetch events. Initially I set a **StartTime **parameter to something initially, and fetched all events. The after some time when I fetch, I specify the **StartTime **as recent event's **EventTime **from previous fetch + 1 second.

Can I use this? Or will I miss any events? If so, can I get any suggestions while also avoiding duplicate events.

已提問 1 年前檢視次數 245 次
1 個回答
3
已接受的答案

Your approach of using the most recent event's EventTime + 1 second as the StartTime for the next lookup can generally work, but there is a small chance of missing events or having duplicate events if events have the same timestamp or if there is a delay in the processing and delivery of events.

To minimize the risk of missing or duplicating events, you can slightly modify your approach:

  1. Use the EndTime parameter along with StartTime. Set the EndTime to the current time when you make the API call. This ensures you're only fetching events within a specific time window.
  2. When setting the StartTime for the next API call, subtract a small buffer (e.g., 5 seconds) from the most recent event's EventTime. This will help account for events with the same timestamp and potential delays.

By implementing these changes, you will reduce the chance of missing or duplicating events. However, you may still need to handle duplicates on your side by storing and comparing event IDs or implementing deduplication logic.

profile picture
專家
已回答 1 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南