Regaring scheduling an event bridge trigger

0

Hi All, I am having a requirement, which should to the following things:

  1. Call an external cliient api from AWS to fetch a file daily once.

  2. While fetching a file, the below are the scenarios:

    -- If fetching of the file from the external api is failed in the first attempt, then check again in a interval of 15 minutes. Do this check like this for every 15 minutes upto 2 hours. If after 2 hours also, the file fetching is failed from the external api, then log the failure.

    -- If fetching of the file is sucussful, then stop calling the external api to fetch the file again, for that      day and push that file to store into s3.
    

The Problem:

As per inital design, I am using an aws event bridge scheduler to schedule for every 15 minutes to trigger the lambda upto 2 hours

This lambda will call the external api and keep the file into s3.

But one, thing which I am finding a bit hard to figure it out is that, in case,

 -- If i  am successful, in retrieving the file from the external api, then how can I ask the aws event      scheduler to stop re-triggering for that day.

  ---  If even after 2 hours, I am not able to fetch the file, then how could eventbridge or the lambda can log (because, 

  ---- 1) event bridge will not know anything about success of failure and it only knows that it. should trigger for every 15 minutes upto 2 hours

        2) lambda is not aware, if the window scheduled for 2 hours is completed)

Is there any better approach if any one can suggest for the above.

asked 2 years ago384 views
2 Answers
2
Accepted Answer

I would use a different approach. Use EventBridge to schedule a daily Step Functions process. The Step Functions workflow will handle the retrieving of the file using a Lambda function. Step Functions has built in Retries. You will set the Interval to 900 seconds (15 minutes) maximum retries to 8 (first try + 8 retries = 2 hours). So a single step in Step Functions will retrieve the file as designed.

profile pictureAWS
EXPERT
Uri
answered 2 years ago
profile pictureAWS
EXPERT
reviewed 10 months ago
profile picture
EXPERT
reviewed 10 months ago
0

Maybe you could trigger the lambda when the file is uploaded to S3. Then you could use a DynamoDB table to store how long since the last trigger if you want to write some logic around that.

https://i.imgur.com/DsRsejU.png

answered 2 years ago
  • My question, is about the lambda hitting the external api with the event bridge scheduler for every 15 mins, to pull a file from an external api. If the lambda is successful, in retrieving the file, then how can I stop the event bridge scheduler to stop from further asking the Lambda to trigger the external api for that day. Only if the external api call is failed, then only the even bridge scheduler should re-trigger in the next 15 minutes.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions