EBS volume I/O error

Question

I've created a new EBS volume, formatted as ext4 and mounted to a Ubuntu instance.
No significant read/write volumes, about 1% of the volume used, nothing special at all.
After 1 week, it has reported an I/O error and was made read-only.
The following details reported in the syslog:
_______________________________
end_request: I/O error, dev xvdf, sector 0
Buffer I/O error on device xvdf, logical block 0
lost page write due to I/O error on xvdf
EXT4-fs error (device xvdf): ext4_journal_start_sb:327: Detected aborted journal
EXT4-fs (xvdf): Remounting filesystem read-only
EXT4-fs (xvdf): previous I/O error to superblock detected
_______________________________

**This is a second time this happens over last 2 weeks.** The above volume has replaced an older volume with the same issue. The server has been in service for a few years with no such issues in the past. What might go wrong? What could I try?

Accepted Answer

The ext4 file system can be configured to be remounted as read-only after encountering an error. For reference, see ext4's manual: https://man7.org/linux/man-pages/man5/ext4.5.html

The relevant part of this documentation is this:
```
errors={continue|remount-ro|panic}
              Define the behavior when an error is encountered.  (Either
              ignore errors and just mark the file system erroneous and
              continue, or remount the file system read-only, or panic
              and halt the system.)  The default is set in the file
              system superblock, and can be changed using tune2fs(8).
```

According to Ubuntu's documentation: https://manpages.ubuntu.com/manpages/xenial/en/man8/mount.8.html, this is the default option for mounting ext4 file systems on Ubuntu.

To prevent a file system from remounting itself as read-only, change its mounting option to not use `errors=remount-ro`. One alternative is to use `errors=continue`, which will allow the volume to still be used after encountering an error.

See this documentation for instructions on how to mount an EBS volume: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

If you have configured an entry in `/etc/fstab` for this volume, you can change the option in there. It will prevent the volume from being remounted as read-only.

EBS volume I/O error

Relevant content