Is there an AWS service that can codelessly, serverlessly download an external file via HTTPS to an S3 bucket?

0

Is there an AWS service that can codelessly, serverlessly download an external file via HTTPS to an S3 bucket?

Is there a way I can serverlessly, codelessly download a file from a given external HTTPS URL and save the downloaded file to a given S3 bucket?

Is there a way I can do this with Step Functions?

Is my only option to drop the codeless requirement and use Lambda?

1 個回答
0

Step Functions supports the ability to call HTTP endpoints through API Gateway, but does not currently support the ability to call generic HTTP endpoints. Here is a link to the documentation that talks about this. https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html

Here is a youtube video on how to upload files to S3 using AWS Lambda. If you are now leaning toward this approach. https://www.youtube.com/watch?v=mw_-0iCVpUc

Other options is to use Amazon Firehose. This is great for analytics use-cases and can handle larger volumes of data. It can also incorporate compression and Parquet conversions to make storing big data more cost-effective. https://aws.amazon.com/kinesis/data-firehose/features

AWS
已回答 2 年前
  • I have been combing through the Amazon Data Firehouse documentation and don't see where how to use Amazon Data Firehouse to do an HTTP GET to a generic HTTP endpoint. Where is that documented? Thank you.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南