Maximum concurrent request in ECS service


I have an ECS (fargate) service with a number of tasks. Desired number of tasks is just 1. I have set up auto scaling with a load balancer. Autoscaling seems to work as I see that the number of tasks is being increased as more requests come in. However, it appears that busy targets (tasks) are handling requests before the scale out event is being triggered. I want a behaviour whereby, if there are no idle tasks, a new task should be started to handle the request. I basically want a task to handle one and only one request at a time. How will I go about this. Is this possible.

2 Respuestas

In order to have each ECS Fargate task serving one request at a time and right after one request starts another ECS Fargate task you can implement an SQS Queue. I think this would be a good use case for the problem you are trying to solve.

I have provided a few steps that you could take.

  1. Change the Service Configuration to a Queue-Based Architecture. Instead of directly using a load balancer to trigger task creation, treat it as an indirection level where you can put an Amazon SQS queue that mediates access to the service and triggers task creation. Here's what that might look like:

    • Create an SQS Queue: Create a new SQS queue where all of these incoming requests will be sent.
    • Lambda Trigger for SQS: The Lambda function is triggered by some period while poll slashing the Queue SQS, and whenever there is a particular message, it creates a new service to implement the ECS task.
    • Configure ECS Task Definition: Make sure your ECS task was configured to process one request from the queue.
  2. Set Up the ECS Task Execution Logic: Update the task definition so that it can run only to process a single request and then shut down.

  • Poll the SQS Queue: Configure the ECS task to poll the SQS queue, pick up a message, process it, and then shut down the execution process. By this, the task will cater to the message one after another.
  • Step-Based Auto-Stop: Define the ECS task so that it stops after processing a single step. This can be done by ensuring the application running inside the task stops after processing one step.
  1. Lambda Function Implementation: The following AWS Lambda function will relaunch a new ECS Task for each of the messages in the SQS queue:

  2. Auto-Scaling Configuration:

    • Configure your ECS service to use Fargate Spot Instances to manage costs if appropriate.
    • Ensure that, at the end of processing, the task stops to avoid carrying on consuming the cost. Monitoring and Alerts: Set up monitoring and alerting for creating and terminating tasks using Amazon CloudWatch. Set up the notification to alert on the failure of a task or a delay in processing.

An SQS combined with the Lambda function to trigger the ECS task completion means that a single task will process one request. This will give you the added potential to be in control over how the tasks are executed and for the functions to scale respectively for only one request at a time.

respondido hace un mes

Configure your ECS service with a target tracking scaling policy based on CPU or memory utilization, and adjust load balancer settings for quick response. Ensure your application handles one request per task and monitor with CloudWatch to fine-tune scaling. This setup ensures each task processes only one request at a time and scales out immediately for new requests.

This workshop could help you:

profile picture
respondido hace un mes

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas