By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Use API Gateway to run a Fargate task and then terminate that pod upon completion

0

I need to run a container/code that will zip files from S3. I am zipping a lot of files so it could take up to 1 hour to zip all the files which means Lambda is not an option.

This leaves me to use Fargate. However, based on my understanding, Fargate will have a pod running at all times.

I want to use API Gateway that will "invoke" a Fargate pod, run the code, then terminate the pod when the files are done being zipped. That way each zip function will have its own isolated environment and I will only be charged for the time that I need to zip files and not all the time the pod is running.

Is this possible and if so how would one go about it? Or is there another AWS service that would be better suited?

Thank you

Topics
ServerlessFront-End Web & MobileNetworking & Content DeliveryContainersMicroservices
Tags
Amazon API GatewayAWS FargateAmazon Elastic Container ServiceMicroservices
Language
English
AWS-User-5311484
asked 2 years ago1030 views
1 Answer
1
  1. Have you considered AWS Batch?
  2. Unfortunately out of the box, generally Fargate does not support scaling to absolutely Zero as of now, unless you are using Kubernetes/EKS and Karpenter; generally there might be some overhead cost, but in terms of ECS for example, it would be a small amount. that said, you can also visit AWS App Runner and see if its pricing are acceptable.
Jason_S
answered 2 years ago
  • AWS-User-5311484
    2 years ago

    @Jason_S Thank you for that info. How would AWS Batch help? My understanding is that AWS Batch just does scheduling for task on Fargate or EC2. I am looking to use API Gateway to trigger the zipping of files dynamically at non regular interval times. It will be based upon when users on the web app want the files zipped.

    If that is the case then wouldn't Batch just be triggering my Fargate task to run but still not able to scale to zero? Or could API Gateway trigger a Batch job 1 second in the future and then destroy the pod after the job completes?

  • Jason_S
    2 years ago

    My assumption from your question is that while the task(s) are initiated by an API call / user, it's not something that is urgent / synchronous because the zipping takes up to one hour. One option is to use AWS Batch, where like you said API gateway can trigger a Lambda execution which creates a job in AWS Batch, which in turn will schedule it for execution in Fargate. However, you don't have much control as to how fast the jobs will be scheduled, as it is handled by AWS Batch scheduler which you does not directly control.

  • Jason_S
    2 years ago

    Additionally, if shortest processing time is so desired, you have the option to spend a little bit more money and let's say have one Fargate instance running. While not optimal, it typically would not create significant costs for a production application.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content