Any way to view the initial setup of an EC2 instance?

0

Disclaimer: I have near-zero experience with AWS. Forgive any ignorance and please try to avoid replying in a manner that only someone with experience would understand. Thanks!

We had a third-party service set up a little system for us that involved an EC2 instance running Ubuntu. The system suddenly stopped working and when I checked AWS, I saw that the instance was not running. Is there any way for me to see if the instance was initially set up to stop working after a certain period of time?

For more (unimportant) context, the system is no longer working, even after starting the instance up again, and the third-party that set this up for us is trying to charge us just to look into the issue. We did nothing with the system other than the basic use exactly how they instructed us to so we know that we did not break anything and it would go a long way if I could prove that they set up the instance to only last so long, effectively giving us a system that was set up to fail.

asked 6 months ago290 views
2 Answers
1

Hello.

You might want to check the CloudWatch metrics for EC2.
The metric "CPUUtilization" described in the document below is the CPU usage rate of EC2.
If this is high, it may be due to a high load caused by an application.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html

You can also check the EC2 system log by following the steps in the document below.
Please check the logs to see if any errors have occurred.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-console.html#instance-console-console-output

Is there any way for me to see if the instance was initially set up to stop working after a certain period of time?

How about creating a new EC2 by creating a AMI from the currently running EC2?
After creation, you can connect to EC2 and troubleshoot.
You can create an EC2 AMI using the following steps.
https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/tkv-create-ami-from-instance.html

profile picture
EXPERT
answered 6 months ago
profile pictureAWS
EXPERT
reviewed 6 months ago
1

Hello illumine8, sorry to hear about your experience with EC2 not starting properly. Also, don't worry about your level of experience. There's quite a bit of great content on AWS SkillBuilder that can help you ramp up quickly.

To answer your question, there are a couple of options you have here. I've listed the options below, as well as links to documentation, for finding out why the instance isn't starting and troubleshooting the instance startup. I would ask a couple of other questions, is there any kind of error code you're getting when trying to start the instance?

  1. Troubleshooting An Unreachable Instance

  2. Using AWS CloudTrail to identify the root cause of why the EC2 instance won't start.

  • To view recent CloudTrail events in the AWS Management console:

  • Sign in to the AWS Management Console and open the CloudTrail console at https://console.aws.amazon.com/cloudtrail/home/

  • In the navigation pane, choose Event history.

  • A filtered list of events appears in the content pane with the latest event first. Scroll down to see more events.

  • To compare events, select up to five events by filling their check boxes in the left margin of the Event history table. View details for selected events side-by-side in the Compare event details table.

  1. If you don't receive an error code when you try to start the instance, then do the following:

  2. Check for the StartInstances event in AWS CloudTrail.

-or-

Run the AWS Command Line Interface (AWS CLI) describe-instances command and specify the instance ID to see the reason for the problem. In the following example, replace MYINSTANCE with the instance ID that you're trying to start.

aws ec2 describe-instances --instance-id MYINSTANCE --output json

Check the StateReason message in the JSON response that the command returns. In the following example response, the StateReason that's returned is Client.InternalError:

"StateReason": { "Message": "Client.InternalError: Client error on launch", "Code": "Client.InternalError" },

Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent version of the AWS CLI.

  1. Review the following common errors to resolve the issue. InsufficientInstanceCapacity

The following error occurs when launching new or restarting stopped instance and there isn't enough available On-Demand capacity to fulfill your request:

An error occurred (InsufficientInstanceCapacity) when calling the StartInstances operation (reached max retries: 4): Insufficient capacity.)

This error occurs if there isn't enough On-Demand capacity to complete your request.

For resolution information, see How do I troubleshoot InsufficientInstanceCapacity errors when starting or launching an EC2 instance? InstanceLimitExceeded

The following error occurs when you launch more than your maximum allowed instances in an AWS Region:

InstanceLimitExceeded: Your quota allows for 0 more running instance(s).

If you receive this message, then submit a quota increase request to AWS Support. In your request, make sure that you include the instance type that you're trying to launch and the AWS Region that you're launching in.

For information on resolving this error, see How do I troubleshoot InstanceLimitExceeded errors when starting or launching an EC2 instance? UnauthorizedOperation

The UnauthorizedOperation error occurs when the user who's trying to launch the instance doesn't have the required permissions. To resolve this error, see How can I decode an authorization failure message after receiving an "UnauthorizedOperation" error during an EC2 instance launch? The requested configuration is currently not supported

One of the following two issues causes this error:

  1. The specified instance type isn't supported in the requested Availability Zone. To check the availability of the instance types in specific Availability Zones, run the aws ec2 describe-instance-type-offerings command in the AWS CLI. Then, launch the instance in a supported Availability Zone.

For example, use the following command to check availability of the c5.2xlarge instance type across all Availability Zones in us-east-1:

aws ec2 describe-instance-type-offerings --location-type availability-zone --filters Name=instance-type,Values=c5.2xlarge --region us-east-1

  1. The Amazon Machine Image (AMI) that's used to launch the instance doesn't support the selected instance type. Before you select the instance type or family, review the AMI descriptions in the AWS Marketplace. This issue commonly occurs when you use AWS Marketplace AMIs, and try to launch unsupported instances using the AWS CLI. When you launch instances from the Amazon EC2 console, unsupported instances are unavailable. Client.InternalError

The following are common reasons why you might receive this error:

An Amazon Elastic Block Store (Amazon EBS) volume isn't correctly attached to the instance.
An EBS volume that's attached to the instance is in an ERROR state.
An encrypted EBS volume is attached to the instance. However, you don't have permissions to access the AWS Key Management Services (AWS KMS) for decryption.

For information on resolving these issues, see How do I troubleshoot an Amazon EC2 instance that stops or terminates when I try to start it? Disk full errors

When you run out of space on an instance and the audit service stops the machine when it boots, a disk full error occurs.

Note: Before you proceed, it's a best practice to create a snapshot to back up your volumes' data.

Important: These resolution steps require you to start and stop the instance. Be aware of the following:

When the instance is stopped, you lose the data in your instance store volumes. For more information, see Determine the root device type of your instance.
If you stop an instance that's part of an Amazon EC2 Auto Scaling group, then you might terminate the instance. Instances that are launched with Amazon EMR, AWS CloudFormation, and AWS Elastic Beanstalk might be part of an AWS Auto Scaling group. Instance termination in this scenario depends on the instance scale-in protection settings for your Auto Scaling group. If your instance is part of an Auto Scaling group, then temporarily remove it from the Auto Scaling group before you proceed.
Stopping and starting your instance changes instance's the public IP address. When you route external traffic to your instance, it's a best practice to use an Elastic IP address instead of a public IP address.

To resolve disk full errors, do the following:

  1. Open the Amazon EC2 console.

  2. Choose Instances from the navigation pane, and then select the instance.

  3. Stop the instance.

  4. Detach the EBS root volume (/dev/xvda for Linux) from the stopped instance.

  5. Launch a new EC2 instance in the same Availability Zone as the impaired instance. The new instance becomes your rescue instance.

  6. Attach the root volume that you detached in step 4 to the rescue instance as a secondary device.

Note: When you attach secondary volumes, you can use different device names.

  1. Connect to your rescue instance using SSH.

  2. Create a mount point directory for the new volume attached to the rescue instance in step 6. In the following example, the mount point directory is /mnt/rescue.

Note: The following $ sudo commands apply only to Linux environments:

$ sudo mkdir /mnt/rescue

  1. Mount the volume at the directory that you created in step 8:

$ sudo mount /dev/xvdf /mnt/rescue

Note: The device (/dev/xvdf, in this example) might have a different device name for the rescue instance that it's attached to. To determine the correct device names, use the lsblk command to view your available disk devices along with their mount points.

  1. Increase the root volume size.

  2. Extend the partition.

  3. Run the umount command to unmount the volume:

$ sudo umount /mnt/rescue

  1. Detach the volume from the temporary instance.

  2. Attach the volume to the original instance with device name (/dev/xvda for Linux), and then start the instance to confirm that it boots successfully.

  3. Review the audit service configuration to make sure that it doesn't have the option to halt (shutdown) the machine if it runs out of space. For Amazon Linux, Amazon Linux 2, and Amazon Linux 2023, make sure that the audit service configuration includes the following options in /etc/audit/auditd.conf:

max_log_file_action = ROTATE admin_space_left_action = SUSPEND disk_full_action = SUSPEND disk_error_action = SUSPEND

Client.InvalidParameterValue

InvalidParameterValue means that a parameter specified in the request isn't valid, is unsupported, or can't be used. The returned message provides an explanation of the error value. For example, if a root volume isn't attached to the instance at /dev/sda1 or /dev/xvda, depending on the AMI, then the instance doesn't start. When this occurs, you see an entry in AWS CloudTrail similar to the following:

An error occurred (InvalidParameterValue) when calling the StartInstances operation: Invalid value 'i-xxxxxxxxxxxxxxxxx' for instanceId. Instance does not have a volume attached at root (/dev/sda1)

To resolve this, use the error message as reference to change the relevant parameter. For example, to resolve the preceding example error, attach the root volume to the instance at /dev/sda1 and then start the instance. For more information, see Error codes for the Amazon EC2 API.

If you're not able to resolve the issues with these steps, I would recommend opening an AWS Support Case with the account that the EC2 instance was created with.

Hope this helps! Brian Anderson - AWS

profile pictureAWS
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions