Is there an AWS service that can codelessly, serverlessly download an external file via HTTPS to an S3 bucket?

Question

Is there a way I can serverlessly, codelessly download a file from a given external HTTPS URL and save the downloaded file to a given S3 bucket?

Is there a way I can do this with Step Functions?

Is my only option to drop the codeless requirement and use Lambda?

Answer

Step Functions supports the ability to call HTTP endpoints through API Gateway, but does not currently support the ability to call generic HTTP endpoints. Here is a link to the documentation that talks about this. https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html

Here is a youtube video on how to upload files to S3 using AWS Lambda. If you are now leaning toward this approach.
https://www.youtube.com/watch?v=mw_-0iCVpUc

Other options is to use Amazon Firehose. This is great for analytics use-cases and can handle larger volumes of data. It can also incorporate compression and Parquet conversions to make storing big data more cost-effective. https://aws.amazon.com/kinesis/data-firehose/features

Is there an AWS service that can codelessly, serverlessly download an external file via HTTPS to an S3 bucket?

Relevanter Inhalt