- Newest
- Most votes
- Most comments
I have a parent stack that's "UPDATE_ROLLBACK_FAILED", and the child stack "UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS" for about 11+ days. My parent stack actually has other children stacks that have 'DELETE_COMPLETE', and cannot be "rollback", and thus the parent stack cannot be rollback. I created a support ticket 10+ days ago, and provided the ARN of both parent stack and child stack with the errors mentioned, since earlier comments suggest that creating support case was a way to resolve more complicated situations [1][2]. I just got a response yesterday (9 days after I created the support ticket/case). While the parent/child stack still say UPDATE_ROLLBACK_FAILED, and UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS, I was able to resume deletion much further, while also being able to delete the child stack that was previously UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS. The parent stack has some custom lambda resources (properly constructed with syntax for delete, yet taking a long time to delete), and edge lambda functions on the parent stack that will take some time to delete.
I understand that root stack is stuck with UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS, which is very common behavior in case of nested stack where some conflicts happen and one of the most common reason is, if updates were pushed directly to child stack.
I assume you don't have AWS support, so I'd suggest you to wait for an hour and let cloudformation timeout, once it times out, you should be able to do "Continue Rollback"
Also, you can try this, if it helps(very less likely):
aws cloudformation continue-update-rollback --stack-name <Root_Stack> --resources-to-skip <Child_Stack> --profile <CLI_Profile>
Please refer Cloudformation Update Stacks, specifically "Cancelling a stack update", "Prevent updates to stack resources", "Continue rolling back an update"
Also take a look at this re:Post
PS: AWS support is very much of help in such scenarios as they do take care of things from their end for faster resolution. If you have support plan and can create Technical case, please log a case following the instructions from here.
Hope you find this helpful.
Abhishek
It's been 2 1/2 hours since the nested stacks entered the UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS state. During the attempted update, no actions were performed due to a stack role permission issue. Based on the documentation https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html#troubleshooting-errors-nested-stacks-are-stuck my only course of action is to contact support. However that is an expensive endeavor.
I understand, nested stacks sometime cause trouble if not handled appropriately. My response is fairly delayed here and I hope your stack would have come to UPDATE_ROLLBACK_COMPLETE state. Seems like you don't have support plan through which you can contact support, then only option is to wait and let it clear by itself as I can say that, it'd clear in sometime by itself. And if it doesn't then it's in non-recoverable state and can only be fixed by support. I've been in such situations, where I had support plan and support always helped effectively but there were cases where I was able to resolve it by myself by the mentioned command in the answer.
Relevant content
AWS OFFICIALUpdated 2 years ago
As a follow-up [1] the stack deletion ended up timing 2 hours later, and failed to delete 3 resources on the original parent stack, another single/remaining child stack is untouched (nothing has attempted to delete yet). On the second attempt, I selected to ignore these three resources. The deletion took another 2 hours or so to timeout. When I tried to click delete again, only 2 resources had options to "retain resources" from the parent stack, with hopes of manually deleting them after cloudformatiion finishes DELETE_SUCCESS. I attempted to delete the stack again (again the single remaining child stack has not attempted to delete). It's been running for 10 minutes, and looks like it's going to be another 2 hours before deletion time out. I wonder whether 1 resource will remain (with option to be ignored) on next attempt to "retry" deletion. Crazy I have to keep "retry" deletion and hoping each iteration after 2 hour timeout that it deletes a little more than the previous attempt. I really have to develop stuff and get things done, but I can't because of dependencies of this stack, and need to rebuild it.
[1] https://repost.aws/questions/QUXEHzeTlHRJO5edCIQRULsg/cloudformation-stacks-stuck-in-update-rollback-complete-cleanup-in-progress#AN8kUKIe2gQneXXmPLBIa-2g
Another 2 hours later, the parent stack failed. Clicking "retry delete", I now see only one resource to retain (instead of two). Will try again tomorrow, since waiting roughly 2 hours for each timeout is consuming. Tomorrow will have another days worth of time I can sit waiting for CloudFormation to timeout.
Was able to delete the last remaining parent stack today. Wouldn't have been able to do it myself, and definitely required AWS Support.