Attach existing EBS Volumes (including root /os volume) to a new EC2 Windows instance as part of ASG

0

I have a requirement. I have an Auto Scaling Group which has 2 or 3 EC2 instances. This setup is provisioned using Terraform.

When one or more instances become unhealthy & get terminated, new instances are provisioned in their place. However, these instances get new EBS Volumes. But I want to reuse / attach the EBS volumes attached earlier to the terminated or unhealthy EC2 instances. I tried & searched a lot, but didn't get any good results. Can anyone please help me achieve this? I thought of using UserData, but I believe it runs with EC2 in Started state and hence cannot attach. From Terraform, I am not sure if this is possible. I even checked for Termination LifeCycle Hooks, but no luck. Please help.

I saw a similar question, but the posted answer didn't seem very helpful to me. https://repost.aws/questions/QUfWhvtTJBRmuPdiK2W-FEWQ/how-are-ebs-volumes-in-an-unhealthy-instance-handled-when-a-new-instance-is-created-by-auto-scaling

4 Answers
1

It's best to separate your own data from the OS and instance configuration. For OS & config you can bootstrap new ASG instances via UserData / cfn-init, or ideally as @takakuni said creating a golden (rich) AMI. Your own data could be on a 2nd EBS volume you detach from old instances and attach to new in the same AZ, or the options @takakuni mentioned.

EXPERT
answered a year ago
  • Thank you @takakuni and @skinsman for your prompt response. However, my golden AMI is already there generated by Packer. Now, I need to attach the existing EBS volumes for other drives (say D, E, F etc) from the terminating /unhealthy instance to the new instance which are part of the ASG. I need to know how to identify these EBS volumes and how to attach them to the new instance. Any links / samples with detailed steps will be really helpful. I was thinking of using Termination LifeCycle Hooks, but I don't know how to call custom shell/powershell scripts from Terminating hook. I don't want to use a Lambda.

1

Unfortunately, I think it will be difficult.

I think it is better to make golden AMI for the root /os volume as much as possible, and to save the data to Fsx for Windows or S3 if you want to share it after the instance is stopped.

profile picture
answered a year ago
0

The process for a launch hook is as follows; termination hooks are similar. If a hook is active for instance launch:

  • instance enters Pending: Wait state.
  • AS (AutoScaling) sends info & token to the hook's notification target.
  • AS waits until you tell it to continue or timeout ends (1 hour default).
  • You can now perform custom setup etc. on the instance.
    • If you need more time you can make the default wait time longer or record a heartbeat.
  • AS puts instance in Pending: Proceed then InService states.
    • That assumes hook returned "CONTINUE". If it returned "ABANDON" then the instance will instead be terminated.

If you want to run something on an instance when its termination lifecycle hook is triggered you have to listen on the instance somehow, e.g. poll an SQS queue or wait for SNS to invoke an http endpoint.

However on launch, if you know that a launch hook is in place, you can more simply:

  • make sure the instance is configured with ASG name & hook name
  • run some setup stuff
  • query instance ID from metadata
  • Call complete-lifecycle-action passing the above data in; if you have the instance ID you don't need the hook event token.
EXPERT
answered a year ago
  • Hi @skinsman, Thank you for your detailed response. However, I am looking for implementation details, preferably using Terraform.

0

Is it possible to call a custom script (say, Shell or PowerShell) from a Termination Lifecycle Hook which will tag the volumes and make them available? Then, can I have another Launching Lifecycle Hook which will allow me to execute a custom script (shell / powershell) to pick up these volumes and attach them to the new instance? Is this possible? If yes, please provide some detailed steps (with links). If not, what are the alternatives? Please help

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions