Bare metal instance with custom scripts and cuttlefish taking 20 min to stop

When an EC2 instance, especially a bare metal instance like c5n.metal with custom scripts and services, is taking an unusually long time to stop (20 minutes in your case), it's often due to processes running on the instance that are preventing it from shutting down cleanly. Here are some steps you can take to address this issue:

1. First, try to force stop the instance. You can do this through the AWS Management Console or by using the AWS CLI command:

   aws ec2 stop-instances --instance-ids your-instance-id --force

2. If the force stop doesn't work after a few minutes, you may need to investigate what's causing the delay. Your custom scripts, the systemctl service, or the cuttlefish installation might be interfering with the shutdown process. You may need to review and possibly modify these to ensure they're not preventing the instance from stopping properly.

3. Check your instance's system logs for any error messages or warnings that might indicate what's causing the delay during shutdown.

4. If the issue persists, you may need to create an AMI of your instance and launch a new one from this AMI. This can help isolate whether the problem is with the instance itself or with your custom configuration.

5. As a last resort, if you're unable to stop the instance and it's critically affecting your operations, you might consider terminating the instance. However, be aware that this will result in data loss if you haven't backed up your data.

6. If none of these steps resolve the issue, it's advisable to contact AWS Support. They can provide more detailed assistance and potentially identify any underlying issues with the instance or the host system.

Remember, while your instance is in the 'stopping' state, you are not being charged for its usage. However, it's still important to resolve this issue to ensure proper management of your resources and to prevent potential problems in the future.

For the long start-up time (5-7 minutes), this is not uncommon for bare metal instances, especially with custom configurations. However, if you need faster start times, you might consider optimizing your boot scripts or using Amazon EBS-optimized instances for faster volume operations during boot.

Sources
My instance not stopping and it is stuck since more than 20 mins now. URGENT PLEASE HELP | AWS re:Post
Troubleshoot Amazon EC2 instance stop issues - Amazon Elastic Compute Cloud