ECS container application fails and is restarted but restart is unsuccessful

0

I am deploying a container application using copilot with a manifest file that specifies a health check and service count of 2. The initial deployment is successful with 2 services running and load balancing is working. When one of the two services fail, a restart of the service occurs as expected, but it is immediately declared unhealthy and is terminated and the restart procedure is repeated with the same result. The second copy of the service is never successfully restarted. I did not experience this problem 6 months ago. What suggestion do you to trouble shoot this problem? David W.

DavidW
asked 6 months ago262 views
3 Answers
0

Hello.

If you check the container logs, you might be able to find out why it doesn't start properly.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_awslogs.html

You can also check the reason for the stopped task by following the steps in the document below.
https://docs.aws.amazon.com/AmazonECS/latest/userguide/stopped-task-errors.html

profile picture
EXPERT
answered 6 months ago
0

Hello, Firstly need to verify logs and events inside services.inside services event we can find each event when anything happens. Also verify cloudwatch logs if any thing bad happening regarding application.

Also please verify target group configuration regarding health check monitoring settings.

profile picture
answered 6 months ago
0

The application is a Java Spring application. The cloudwatch logs show the Spring server starting with no errors, and then there is a shutdown message. There is no console log message that indicates an error or why the server is shutdown. This is likely because the process was terminated by an external interrupt.

Some other information, if the ECS service continues for some 5-10 minutes, after 5 attempts at restarting the failed container only for the restarted container to be marked UNHEALTHY and thereby terminated, the container does finally do a restart and is reported as HEALTHY.

Satish's answer says to verify the target group configuration for health check monitoring settings. I am not sure exactly what this refers to or how to do this. In the manifest.yml file used for deployment of the container into ECS, the settings are

------------------------------------------------- manifest.yml for service cst438service -------------------------------------------------- name: cst438service

type: Load Balanced Web Service

http:

path: '/'

healthcheck: '/check'

interval: 300

image:

build: cst438container/Dockerfile

port: 80

cpu: 256 # Number of CPU units for the task.

memory: 512 # Amount of memory in MiB used by the task.

count: 2 # Number of tasks that should be running in your service.

exec: true # Enable running commands in your container.


David

DavidW
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions