Public Rest API to S3

Question

I am new to AWS , I would like to know the easiest method to move data from a public APi such as Data.gov to an S3 bucket for further analysis . I am familiar with Aws services such as Glue and AppFlows but they don’t seem suited to pull data from a custom rest api endpoint .

Accepted Answer

If you want to simply fetch the data from the API and write it to S3, this can be done with Lambda Functions. You could then set up EventBridge to trigger your Lambda Function at a predefined schedule, allowing you to poll the API periodically.

However if your workflow has more complex steps or involves long-running processes, you might want to look into other services like AWS Step Functions or AWS Batch.

Public Rest API to S3

Contenido relevante