- Newest
- Most votes
- Most comments
You can set the deregistration delay on the Application Load Balancer target group. https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#deregistration-delay
The autoscaling group sends a deregistration call to the ALB once a scale-in event happens, and the ALB then waits for the configured deregistration delay. During that period, no new requests will reach the target group. After that period elapsed, the instance(s) get terminated. The downsides here are (1) that the ALB will wait even if there are no more critical processes running on your instance. (2) You would need to make the delay large enough to ensure that the processes have enough time to finish.
If you observe a scale-in event in the console, you can see the activity "WaitingForELBConnectionDraining" before the instance gets terminated.
what is the namespace for it so I can add it into my cloudformation configuration template for elasticbeanstalk ,didn't find it in documentation - Namespace: aws:elbv2:loadbalancer OptionName: DeregistrationDelayTimeout Value: '1200' it is valid?
It's an attribute of the target group: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticloadbalancingv2-targetgroup-targetgroupattribute.html Does that help?
Is it possible to still route requests to an instance even if it is marked for termination? I have some requests that have sticky sessions enabled, so these should be routed to the first instance.. And all the new requests with the new sticky session can be routed to the new instance that is being spun up.. Is there some way to do it?
Hi @rePost-User-5980556 - you'll generally want to create a new question when you have a question, instead of adding it as an answer on an older one. Only people who were following this question will see your post
For your question though, since I did see it: during the Terminating:Wait state in an ASG, the instance won't be terminated. Similarly, it won't be terminated by the ASG while ELB Draining is happening. However, the ELB won't honor the sticky session during draining/deregistration. Currently in-flight requests will be finished, but a new request hitting the ELB on that sticky session will be routed to a new healthy target, not sent to the one currently Draining
Hi,
the instance was terminated because of the defined scaling policy:
alarm threshold: NetworkOut < 2000000 for 1 consecutive periods of 300 seconds
This states that after low network out activity for 5 minutes (300 seconds), an instance is going to be terminated.
To allow the instance to complete the long running process, you can use lifecycle hooks
Amazon EC2 Auto Scaling offers the ability to add lifecycle hooks to your Auto Scaling groups. These hooks let you create solutions that are aware of events in the Auto Scaling instance lifecycle, and then perform a custom action on instances when the corresponding lifecycle event occurs. A lifecycle hook provides a specified amount of time (one hour by default) to wait for the action to complete before the instance transitions to the next state.
if my aim is just to let the process complete, what action are you suggesting me to add in that hook?
Relevant content
- asked 4 years ago
There isn't enough info here. Depending on your usecase, either of the below answers might work. The Lifecycle Hook of an ASG starts after the instance is deregistered from the ELB. If the long running process is async and can finish on its own; a Hook would help. If there's a single long running request from a client waiting for the response, longer Deregistration Delay would help.
If the client sends a new request to get the answer, you'll need to rearchitect so the result of the long running process is stored outside of the instance and any instance in the fleet can provide the answer
yes you are right there is not enough info here, the process can finish on its own and could take around 70 minutes in worst case. the client is not waiting for the response. the client does not sends a new request to get the answer.
A Lifecycle Hook (LCH) should work great here then. Add a Terminating LCH to the ASG, which will keep the instance in Terminating:Wait until the hook is completed. Add a cron to the instance regularly polling IMDS to see if the instance has started its LCH. If it has, wait for the job(s) to finish and then complete the hook. See LCH section: https://docs.aws.amazon.com/autoscaling/ec2/userguide/gracefully-handle-instance-termination.html