Hello there,
There are few ECS fargate services in the cluster of my AWS account. Each service has only one task running (it is desired number of tasks) with essential container. Deployment type: Blue green deployment (With CodeDeploy). Service Discovery is enabled with TTL 60sec.
I face the issue of auto-deploying the second task for some of these services. When I check ECS service's events, I see event "STABILIZING" and after this event the second replica of the task starts to deploy. Time when STABILIZING event occurs is always the same: working days, 10:27 UTC+2. It can be one service or more, but the time of event is always the same. New tasks are trying to reach healthy status, some of them deploying successfully, but some of them restarts in the loop and can't reach healthy status.
My observations:
- There is no difference what application is running inside container (I have NextJS and Java)
- STABILIZING event occurs in the services which has task's uptime more then 15 days
I fix this issue with clicking on Update Service button and updating service without any changes, but with enabled Force Deployment radio button. After Force Deployment, issue is eliminated and ECS service reaches steady state.
I have seen this issue not for the first time, I thought that it was just some temporary bug of ECS. But this issue appears again and again and makes using ECS unreliable. Moreover, this ECS cluster uses for production of the project. So, I get some inappropriate behavior of the production application from time to time due to this issue.
The main question is: Why ECS service is trying "STABILIZING" something even though the original essential task is healthy and there is no errors from its side.
