How to I troubleshoot a deploy that is forever in "IN_PROGRESS"?

0

I'm using ECS with Fargate, I have a service running and it is working OK. But after I update the task definition (new deploy) and the console (ECS -> Clusters -> Tasks tab) shows that my current task is INACTIVE, which is normal, but it doesn't show any new task being created, nor any stopped task, even after an hour. It is as if ECS is not trying to run my new definition.

If I use the awscli to find information about my service:

aws ecs describe-services --cluster cluster-xxxxxxx --services service-svc-xxxxxxx --region us-east-1
It has two deployments. The first is alright, it is the running deployment. The most recent one, it shows:

          "desiredCount": 1,  
          "pendingCount": 0,  
          "runningCount": 0,  
          "failedTasks": 7,  
          ...  
          "rolloutState": "IN_PROGRESS",  
          "rolloutStateReason": "ECS deployment ecs-svc/XXXXXXXXXXXXXXXXXX in progress."  

Again, there is nothing on the ECS console that points to failed tasks. It is as if the task is failing on a so premature state, its not even logging anything. On the "Task" tab, when I point "Desired task status:" to "Stopped" it shows an empty list with "No Results".

I tried looking at CloudTrail event, but there is nothing there about failed tasks. On CloudWatch, the logs for container insights (/aws/ecs/containerinsights/cluster-xxxxxxx/performance) also don't mention failing tasks.

How can I troubleshoot this situation?

diogofc
gefragt vor 3 Jahren1926 Aufrufe
1 Antwort
0

Turns out AWS is applying quotas without specifying it anywhere on the console or documentation. And there seems to be a "trick", to overcome this, which is to create an EC2 instance on the same region you are trying to use ECS.

https://forums.aws.amazon.com/thread.jspa?threadID=340770

diogofc
beantwortet vor 3 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen