Transfer files (1GB to 2G) from web url to S3 on a schedule
0
My customer has a use case to access file from a web url periodically and upload to S3. This is currently implemented using lambda with schedule trigger and is working well for smaller files. But larger files need higher memory settings in lambda. Is there a way to directly upload to S3 without storing in memory or are there better serverless implementation options for this use case?
demandé il y a 4 ans300 vues
1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
I've previously used a very small container running in ECS for this. ECS has scheduling in-built, so you simply set it the schedule appropriately, and it runs regularly thereafter.
The container I used was built on Alpine, and used wget to pull the files I needed.
Contenus pertinents
- demandé il y a un an
- demandé il y a 6 mois
- demandé il y a 3 mois
AWS OFFICIELA mis à jour il y a 2 ans- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a un an