ECS task trigger from EventBridge, but ECS capacity unavailable

When your ECS Fargate task fails due to capacity unavailability, Amazon ECS emits a service action event that you can capture with EventBridge. This event has a specific format that you can use to create an EventBridge rule.

For capacity-related failures, ECS generates a SERVICE_TASK_PLACEMENT_FAILURE event with an ERROR event type. Here's how you can set up an EventBridge rule to detect these failures:

{
"source": ["aws.ecs"],
"detail-type": ["ECS Service Action"],
"detail": {
"eventType": ["ERROR"],
"eventName": ["SERVICE_TASK_PLACEMENT_FAILURE"],
"reason": ["RESOURCE:FARGATE"]
}
}

This pattern will match events where ECS was unable to acquire Fargate capacity. When this event is triggered, you can configure EventBridge to invoke a target (like a Lambda function or Step Functions state machine) that will retry the RunTask operation.

For standalone tasks that use asynchronous operations like RunTask, you can use AWS Step Functions with EventBridge integration to implement retry logic for throttled or failed operations. This approach allows you to handle the capacity unavailability by waiting and retrying the task launch, potentially with different subnet configurations to target different availability zones.

If you need to retry in different availability zones, your retry logic should modify the subnet parameters in the RunTask call to target subnets in other AZs.

Remember that you'll need to ensure your EventBridge target has the appropriate IAM permissions (using the ecsEventsRole or similar) to run ECS tasks on your behalf.
Sources
Amazon ECS service action events - Amazon Elastic Container Service
Amazon ECS EventBridge IAM Role - Amazon Elastic Container Service
Handle Amazon ECS throttling issues - Amazon Elastic Container Service
Amazon ECS service event messages - Amazon Elastic Container Service