EC2 Instance with recurrent I/O errors, but fsck reports filesystems as clean

Question

I have a Linux (Ubuntu) instance running as m3.xlarge - this server is getting IO errors across multiple EBS volumes. I have unmounted the volumes and run fsck on them. One found some problems that were fixed, the others reported as clean. On restart, after a while (hour) I received more IO errors. As multiple volumes are involved, I wondered if there was an underlying hardware problem (I have multiple other very similar instances which are functioning as normal). I have stopped/started the instance to see if that would move it to a new host, but there is no way of telling.
Suggestions on finding the issue? Is there a way to confirm if an instance has moved to a new host?

Answer

Hi, thank you for your question. In most cases, starting and stopping your instance will migrate the instance to another physical host computer [1]. If you have stopped/started the instance multiple times and the I/O errors persist, it is likely that the problem lies within the instance itself, rather than the underlying host/hardware. Please refer to this guide for help troubleshooting common I/O errors within your EC2 instance [2].
             
Additionally, please ensure that your read/write IOPS are not exceeding the capacity of your EBS volumes. You can view your read/write IOPS by monitoring the VolumeReadOps and VolumeWriteOps volume metrics in CloudWatch [3].
                 
[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html  
[2] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html  
[3] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html

EC2 Instance with recurrent I/O errors, but fsck reports filesystems as clean

Relevant content