Maximum concurrent request in ECS service

In order to have each ECS Fargate task serving one request at a time and right after one request starts another ECS Fargate task you can implement an SQS Queue. I think this would be a good use case for the problem you are trying to solve.

I have provided a few steps that you could take.

1. Change the Service Configuration to a Queue-Based Architecture. Instead of directly using a load balancer to trigger task creation, treat it as an indirection level where you can put an Amazon SQS queue that mediates access to the service and triggers task creation. Here's what that might look like:

   • Create an SQS Queue: Create a new SQS queue where all of these incoming requests will be sent.
   • Lambda Trigger for SQS: The Lambda function is triggered by some period while poll slashing the Queue SQS, and whenever there is a particular message, it creates a new service to implement the ECS task.
   • Configure ECS Task Definition: Make sure your ECS task was configured to process one request from the queue.

2. Set Up the ECS Task Execution Logic: Update the task definition so that it can run only to process a single request and then shut down.

• Poll the SQS Queue: Configure the ECS task to poll the SQS queue, pick up a message, process it, and then shut down the execution process. By this, the task will cater to the message one after another.
• Step-Based Auto-Stop: Define the ECS task so that it stops after processing a single step. This can be done by ensuring the application running inside the task stops after processing one step.

3. Lambda Function Implementation: The following AWS Lambda function will relaunch a new ECS Task for each of the messages in the SQS queue:

4. Auto-Scaling Configuration:

   • Configure your ECS service to use Fargate Spot Instances to manage costs if appropriate.
   • Ensure that, at the end of processing, the task stops to avoid carrying on consuming the cost. Monitoring and Alerts: Set up monitoring and alerting for creating and terminating tasks using Amazon CloudWatch. Set up the notification to alert on the failure of a task or a delay in processing.

An SQS combined with the Lambda function to trigger the ECS task completion means that a single task will process one request. This will give you the added potential to be in control over how the tasks are executed and for the functions to scale respectively for only one request at a time.

https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/run-message-driven-workloads-at-scale-by-using-aws-fargate.html

Configure your ECS service with a target tracking scaling policy based on CPU or memory utilization, and adjust load balancer settings for quick response. Ensure your application handles one request per task and monitor with CloudWatch to fine-tune scaling. This setup ensures each task processes only one request at a time and scales out immediately for new requests.

This workshop could help you:

• DEPLOY ECS CLUSTER AUTO SCALING
• Target tracking scaling policies for Amazon EC2 Auto Scaling