Charging operate Task for execution time

0

Hello. I have a PDF service (Task definition + Service + Task), which runs in the EC2 cluster. This service runs less than 100 times per month and duration is approx 3-5 minutes, but it is a high-performance task, and I enforced each time increase the power of the instance. Also, this instance operates 13 additional tasks with low performance. I start this task on Linux t3.large Instance.

So I would like to separate this PDF Task from the EC2 instance to save monthly charging by paying just for operating PDF tasks. I've read about Fargate, but as I understood if I create a container, task definition I will pay all the same for running the Fargate cluster, but not for time running task!

Question: How to run Task and pay only for it operating?

asked 2 years ago294 views
2 Answers
1
Accepted Answer

Fargate is your best answer here: You do not pay per Task Definition or Service. Only when the service creates tasks (containers) does AWS bill you. Fargate also only bills you based on the amount of CPU/RAM defined in your Task definition so you can be very precise about this. Each task (set of containers) deployed is on its own host so there is no eating out the compute resources of other services / tasks running. If you want to use the same ECS cluster as you do now, you can set a capacity provider to use FARGATE (and/or FARGATE_SPOT) for that one service only and leave the others as they are.

profile picture
answered 2 years ago
  • So, as I understood, I will create the Fargate cluster, Task Definition, and Task. Then I set up a trigger on Lambda to run this Task. And in this case, I will pay just for the duration of the task operation? Is it correct?

  • If you create an ECS Service (as opposed to a scheduled task) you need to have something change the desired capacity for that service, both to add and remove containers, within the min/max you would define. That is totally possible to do, very easy to implement, but you need to know how.

    If you create a scheduled task (so, that uses an ECS Task Definition but does not create an ECS Service) then all you need to "stop paying" is make sure your tasks exits(). The task will be created and run for as long at it takes to complete (no timeout).

    If your workflow is files -> S3 to trigger the build, I would recommend you just get S3 to notify SQS. In SQS you will get a very simple message with the bucket name / file name etc. Based on SQS queue depth, start containers, get them to do their job, and when messages in queue = 0, autoscaling will take the number of containers for that service down to your set minimum (here, 0). Having lambda as the source trigger for your ECS services is not the most straightforward way and it does not give any context to your containers as to why they got started.

  • I see second your sample, but not file on S3, it is a DB record from RDS. If the table gets a new row (record) task should be started. In this case, RDS should notify SQS and based on it starts the container and do Task?

  • In this case I've created:

    • Fargate container (active)
    • Task Definition (4 vCPU and 8Gb memory) with ECR container of PDF application - (active)
    • Task to operate PDF, created but stopped (not running)

    So, as I understood I will pay only for the running task? or in this case, I will pay for the active time of task definition, because it uses a container?

0

Another good option to consider is using a Lambda function if your PDF generation doesn't take longer than 15 mins.

AWS
Aneel_M
answered 2 years ago
  • Unfortunately, I tried to do it with developers, something went wrong due to the python environment...

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions