How to understand the difference between the instances?
Our group is working on a tool to analyze biolgical data. The tools uses dotnet to run on linux machines. As first test I have tried to run it on an Ubuntu AMI (22.04) to be able to compare it our own server. For some reason our tool does not run on Ubuntu AMI. As the tools needs a lot of I/O, I thought at first it has something to do with the storage structure I'm using. But after testing both gp2, gp3 and io1 with the same results. I wasn't sure anymore where I can find the problem.
Just by chance I tested the amazon linux 2023 AMI. Here, I could run the tool all the way through with good results. I also tested it here in io1 and gp3. Both performed well. I don't understand the difference. Why does it work in one instance, but not in the other.
The only difference I can see, when I create the instance is the message attached below. Do I understand it correctly, that the Ubuntu 22.04 AMI doesn't have an instance storage? I tried to view the hardware parameters in both instances via command line but couldn't see any difference. Am I looking at the right place (lscpu, fsidk, etc.) ?
Is it correct to interpret it as such, that having the instance storage in the amazon linux AMI allows for a faster I/O then the EBS storage on the Ubuntu AMI?
How should I interpret this message? Can I add an instance storage to the Ubuntu AMI?
I appreciate the help and would be happy if someone can point me to where I can better understand the difference between the two AMIs.
thanks
- Newest
- Most votes
- Most comments
This link describes the details around Instance Store. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html
Key here is that it is a local physical disk in the server where your EC2 instance is being hosted. The disk is usable while your instance runs but nothing on that disk will be saved when you stop the instance. It is great for cache or working storage to certain I/O intensive workloads.
When choosing instance types look for a d in the instance type name. i.e. r5d.<size> The d indicate that the instance type has Instance Store volumes available.
My understanding is same as yours - By default, the Ubuntu 22.04 AMI is backed by Amazon EBS volumes, not instance store volumes. This means storage is persistent across stops/starts but not tied to the life of the instance.
Ephemeral(Instance) storage can offer faster I/O performance compared to EBS volumes due to its local nature. This might explain why your .NET tool runs better on the Amazon Linux 2023 AMI with ephemeral storage.
The commands lscpu and fdisk are helpful for general hardware information, but they won't directly show the difference between ephemeral and EBS storage. You can use console or the command lsblk to view all available block devices on your instance, which can help identify both ephemeral storage and EBS volumes.
Unfortunately, you cannot directly add ephemeral storage to a running Ubuntu AMI.
Few Cents:
- While EBS might not be as fast as ephemeral storage, you can optimize its performance for your workload. Look into EBS Provisioned IOPS (Input/Output Operations Per Second) volumes, which offer predictable high performance.
2.Test with a Smaller EBS Volume: If your .NET tool only needs a small amount of storage, try launching a new Ubuntu AMI instance with a smaller EBS volume size. Smaller volumes sometimes have better performance compared to larger ones.
also note :Only the following instance types support an instance store volume as the root device: C1, C3, D2, I2, M1, M2, M3, R3, and X1. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-instance-store.html
Thanks for the answer.
A few cents of my own:
- I know about io1 and io2 and have tested it on both ubuntu and amazon linux with the same results. Only on the amazon linux my tool runs.
- I can't use a smaller volume as I have to have art least 50Gb of storage to be able to calculate the results. And this is even still a small data set. real data sets can be over 100GB in total size.
I don't understand the last comment. Does it mean my c7 instance is not using ephermal (instance) storage?
I confirmed this on Canonical, Ubuntu, 22.04 LTS, amd64 jammy image build on 2024-04-11.
I confirmed that multiple instance stores are added when checking with an instance type with non-NVMe instance store volumes, so the configuration may be for non-NVMe instances.
From this point of view, the NVMe driver may not be installed, and I/O performance may not be achieved.
sorry, but I don't understand this answer. What did you confirm?
In an irregular way, I tried to create Ubuntu 22.04 with an instance type e.g. M3 with a non-NVMe instance store volume.
By choosing M3, I could confirm that multiple instance stores would be added, so I assumed the reason for the message when using Ubuntu was configured for non-NVMe instance store volumes.
Relevant content
- asked 10 hours ago
AWS OFFICIALUpdated 14 days ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated 2 years ago
I don't have the for persistent storage, as the idea is to load the data, run the tool and download the results. After that the instance can be terminated. So no problems there.
Do I understand it then correctly, that an instance such as ci7.12xlarge doesn't have ephermal storage, but an instance such as c7gd.12xlarge would have it? and if this is true, it still doesn't explain why I can run my tool on the instance ci7.12xlarge with the amazon linux as OS, but not with Ubuntu.
Reading the information from the link you provided (thx for that), I understand I don't have instance store in the instances I test. So I till don't understand, why the tool runs perfectly, when using the amazon linux AMI but not the ubuntu. Also, what does this message I attached says?