By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Why doesn't the stack rollback option NOT to rollback not work?

0

This should be reported as a bug in the ASW Console (I have many) but there is no way to do that. Trying to find why a service creation with an https load balancer fails. The error messages displayed after 40 (!) minutes of spinning are useless. That's a LONG time to waste only to get a message that doesn't help.

So I configured the stack option on the cluster not to rollback the created items and tried again. Another 40 waste of time. But that setting was ignored. So all my created items were killed.

Is it true that the Console is so buggy that one is forced to use the CLI?

Here is the CloudFormation log: One sees that the created items were deleted anyways.

2023-06-12 17:29:14 UTC-0400 ECS-Console-V2-Service-exchange2-service-exchange2-cluster-1e97d4ec DELETE_COMPLETE

2023-06-12 17:29:12 UTC-0400 ECS-Console-V2-Service-exchange2-service-exchange2-cluster-1e97d4ec DELETE_IN_PROGRESS User Initiated 2023-06-12 17:28:54 UTC-0400 ECS-Console-V2-Service-exchange2-service-exchange2-cluster-1e97d4ec ROLLBACK_COMPLETE

2023-06-12 17:28:53 UTC-0400 SecurityGroup DELETE_COMPLETE

2023-06-12 17:28:20 UTC-0400 SecurityGroup DELETE_IN_PROGRESS

2023-06-12 17:28:20 UTC-0400 TargetGroup DELETE_COMPLETE

2023-06-12 17:28:19 UTC-0400 LoadBalancer DELETE_COMPLETE

2023-06-12 17:28:19 UTC-0400 TargetGroup DELETE_IN_PROGRESS

2023-06-12 17:28:18 UTC-0400 LoadBalancer DELETE_IN_PROGRESS

2023-06-12 17:28:18 UTC-0400 Listener DELETE_COMPLETE

2023-06-12 17:28:16 UTC-0400 Listener DELETE_IN_PROGRESS

2023-06-12 17:28:15 UTC-0400 ECSService DELETE_COMPLETE

2023-06-12 17:25:30 UTC-0400 ECSService DELETE_IN_PROGRESS

2023-06-12 17:25:28 UTC-0400 ECS-Console-V2-Service-exchange2-service-exchange2-cluster-1e97d4ec ROLLBACK_IN_PROGRESS The following resource(s) failed to create: [ECSService]. Rollback requested by user. 2023-06-12 17:25:27 UTC-0400 ECSService CREATE_FAILED Resource handler returned message: "Error occurred during operation 'ECS Deployment Circuit Breaker was triggered'." (RequestToken: cb4610c3-7ebd-6cbf-9e16-34b9498beabf, HandlerErrorCode: GeneralServiceException) 2023-06-12 16:45:53 UTC-0400 ECSService CREATE_IN_PROGRESS Resource creation Initiated 2023-06-12 16:45:51 UTC-0400 ECSService CREATE_IN_PROGRESS

2023-06-12 16:45:50 UTC-0400 Listener CREATE_COMPLETE

2023-06-12 16:45:49 UTC-0400 Listener CREATE_IN_PROGRESS Resource creation Initiated 2023-06-12 16:45:47 UTC-0400 Listener CREATE_IN_PROGRESS

2023-06-12 16:45:46 UTC-0400 LoadBalancer CREATE_COMPLETE

2023-06-12 16:44:24 UTC-0400 TargetGroup CREATE_COMPLETE

2023-06-12 16:44:15 UTC-0400 LoadBalancer CREATE_IN_PROGRESS Resource creation Initiated 2023-06-12 16:44:14 UTC-0400 LoadBalancer CREATE_IN_PROGRESS

2023-06-12 16:44:13 UTC-0400 SecurityGroup CREATE_COMPLETE

2023-06-12 16:44:12 UTC-0400 SecurityGroup CREATE_IN_PROGRESS Resource creation Initiated 2023-06-12 16:44:08 UTC-0400 TargetGroup CREATE_IN_PROGRESS Resource creation Initiated 2023-06-12 16:44:06 UTC-0400 SecurityGroup CREATE_IN_PROGRESS

2023-06-12 16:44:06 UTC-0400 TargetGroup CREATE_IN_PROGRESS

2023-06-12 16:44:03 UTC-0400 ECS-Console-V2-Service-exchange2-service-exchange2-cluster-1e97d4ec CREATE_IN_PROGRESS User Initiated

Topics
Containers
Tags
Amazon Elastic Container Service
Language
English
brianR
asked a year ago243 views
1 Answer
0

The CloudFormation log shows that the "Deployment circuit breaker" is running.
Are there several causes for this to work?
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-circuit-breaker.html

The deployment circuit breaker monitors tasks that are part of the deployment and checks for tasks that are in the RUNNING state. The scheduler ignores the failure criteria when a task in the current deployment is in the RUNNING state and proceeds to the next stage. When tasks fail to reach in the RUNNING state, the deployment circuit breaker increases the failure count by one. When the failure count equals the threshold, the deployment is marked as FAILED.

This stage is entered when there are one of more tasks in the RUNNING state. The deployment circuit breaker performs health checks on the following resources for the tasks in the current deployment:

Elastic Load Balancing load balancers

AWS Cloud Map service

Amazon ECS container health checks

When a health check fails for the task, the deployment circuit breaker increases the failure count by one. When the failure count equals the threshold, the deployment is marked as FAILED.

profile picture
EXPERT
Riku_Kobayashi
answered a year ago
  • brianR
    a year ago

    I have looked at that and it doesn't help. I am trying to ditch that 'circuit breaker' so I can get some information on why my service is not getting created. So I had hoped that all my successfully created entities (load balancer, targetgroup, listener, etc.) would remain when the service fails to create (I still don't know why the service fails - no help from the console). But after 40 minutes of waiting, the rollback happens anyways deleting everything even though I have turned rollback off.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content