Resource handler returned message: ECS Deployment Circuit Breaker was triggered. (HandlerErrorCode: GeneralServiceException)

0

Goal: I am attempting to run a simple NodeJS application with two routes: '/' which displays "Hello, World!" and '/health' which returns a status code of 200 along with a message.

Problem: I keep getting this error message: Resource handler returned message: "Error occurred during operation 'ECS Deployment Circuit Breaker was triggered'." (RequestToken: 9d7b28d1-2a1f-ed3f-3764-3da55e89b691, HandlerErrorCode: GeneralServiceException).

Tried: I attempted to resolve the issue by removing the existing cluster and service. I then proceeded to create a new cluster and service, starting from scratch, utilizing the provided task definition (shown below).

Certainly: Could someone please assist me in identifying the cause based on the clues I provided here? I need guidance on where to look and what to fix. Your assistance in troubleshooting is greatly appreciated. Thank you for taking the time to help with my issue.

CloudFormation events: Enter image description here

Task definition: nodejs-app-task-revision1.json

{
    "taskDefinitionArn": "arn:aws:ecs:ap-south-1:<ACCOUNT_ID>:task-definition/nodejs-app-task:1",
    "containerDefinitions": [
        {
            "name": "nodejs-app-container",
            "image": "<ACCOUNT_ID>.dkr.ecr.ap-south-1.amazonaws.com/nodejs-server",
            "cpu": 0,
            "portMappings": [
                {
                    "name": "nodejs-app-container-8000-tcp",
                    "containerPort": 8000,
                    "hostPort": 8000,
                    "protocol": "tcp",
                    "appProtocol": "http"
                }
            ],
            "essential": true,
            "environment": [],
            "environmentFiles": [],
            "mountPoints": [],
            "volumesFrom": [],
            "ulimits": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/nodejs-app-task",
                    "awslogs-region": "ap-south-1",
                    "awslogs-stream-prefix": "ecs"
                },
                "secretOptions": []
            },
            "healthCheck": {
                "command": [
                    "CMD-SHELL",
                    "curl -f http://localhost:8000/health || exit 1"
                ],
                "interval": 30,
                "timeout": 5,
                "retries": 3
            }
        }
    ],
    "family": "nodejs-app-task",
    "executionRoleArn": "arn:aws:iam::<ACCOUNT_ID>:role/ecsTaskExecutionRole",
    "networkMode": "awsvpc",
    "revision": 1,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.24"
        },
        {
            "name": "ecs.capability.execution-role-awslogs"
        },
        {
            "name": "com.amazonaws.ecs.capability.ecr-auth"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
        },
        {
            "name": "ecs.capability.container-health-check"
        },
        {
            "name": "ecs.capability.execution-role-ecr-pull"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        },
        {
            "name": "ecs.capability.task-eni"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.29"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EC2",
        "FARGATE"
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "1024",
    "memory": "3072",
    "runtimePlatform": {
        "cpuArchitecture": "ARM64",
        "operatingSystemFamily": "LINUX"
    },
    "registeredAt": "2024-02-08T11:24:00.551Z",
    "registeredBy": "arn:aws:iam::<ACCOUNT_ID>:root",
    "tags": []
}
2 回答
0

Hi, Johnson

I would check to make sure you're not missing any dependencies (for example, environment variables) as it appears the application isn't bootstrapping correctly, as is deemed unhealthy.

Please take a look at this https://repost.aws/knowledge-center/ecs-task-container-health-check-failures

If this helps solve your problem, please accept this as the Accepted Answer, so that others on re:Post may benefit - thank you.

profile pictureAWS
已回答 3 个月前
profile picture
专家
已审核 1 个月前
  • Hi, Eric I have tested running the Dockerized application locally and have successfully passed the health checks. Additionally, I have cross-checked the commands specified in the container definition. However, the issue persists.

0
"healthCheck": {
                "command": [
                    "CMD-SHELL",
                    "curl -f http://localhost:8000/health || exit 1"
                ],

The health check command seems to be using localhost to access the container's health endpoint. However, in ECS, the health check is executed externally, not within the container itself. Therefore, using localhost won't work. You should replace localhost with the IP address or domain name of your container.

athar
已回答 2 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则