Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

EFS metadata IOPS surges

0

I have noticed that my EFS sometimes goes through long periods (hours, sometimes days) of high metadata IOPS. So much so that when this disk was bursting, then this would consume my burst credits very quickly (I wish I could add a screenshot here to show). I have had to change to expensive Elastic FS type to prevent this.

I am relatively sure this is not any process that I am in control of. What exactly is a metadata IOPS and can I prevent it running for so long on my EFS?

Topics
Storage
Tags
Amazon Elastic File System
Language
English
asked a year ago647 views
2 Answers
1

Check this answer https://repost.aws/questions/QUHVElgie1SOGPIyf1N8k45A/efs-high-metadata-io

EXPERT
answered a year ago
EXPERT
reviewed a year ago
EXPERT
reviewed a year ago
  • Cinton
    a year ago

    Hi Oleksil, thanks for responding. I did see that post and response from AWS, but if you have a look at it you can see there is no real applicable answer to my question. They suggest monitoring IOPS/connections etc through cloudwatch logs/event/alarms. I already do that. My question is what causes these erratic metadata surges? None of the above-mentioned methods can show this. Is there a way I can actually monitor what is causing it?

  • Oleksii Bebych EXPERT
    a year ago

    Common Causes of High Metadata IOPS

    1. File Scanning: Applications that frequently scan directories and files, such as backups, indexing services, or antivirus scans, can generate a lot of metadata IOPS.

    2. Automated Scripts: Scripts or cron jobs that frequently create, delete, or modify files.

    3. Application Behavior: Certain applications or workloads that heavily interact with the file system metadata.

    4. Misconfigured Applications: Applications that are not properly tuned or have bugs can cause excessive metadata operations.

0

Hi, you can analyze your application to identify if it performs large operations (for example, reading the contents of a large directory using readdir). Such large operations may consume a significant amount of metadata IOPS. It's recommended to optimize these operations to reduce the metadata IOPS consumption.

AWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content