Checkpointing an instance.

Question

Hi,
I am trying to run a scientific calculation job on an instance. It is a large calculation which should take a lot of time to compute. Due to the dependency among the calculations, it cannot be parallelized.My concern is that if the instance fails, I will lose all the progress. What are the best practices where if the instance fails, I can resume the computation without having to restart from the beginning.I will prefer to checkpoint periodically and launch another instance from that checkpoint. Does AWS have any built in mechanisms that I can use to checkpoint?
thanks

Answer

You can snapshot the EBS volumes and then restore that to a new volume and resume. It will take some development on your part, and you'll need to ensure you supply the correct permissions to your instance role to take snapshots and restore them. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSSnapshots.html and https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ebs-creating-snapshot.html and https://docs.aws.amazon.com/prescriptive-guidance/latest/backup-recovery/restore.html

Checkpointing an instance.

Relevant content