By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

EFS Throughput and mount

0

I have an efs file system which is mounted across 4 to 5 Linux boxes. Here is the efs file system policy to move 7 days old files to IA and keep it in that state. But In each ec2 instance, there is a cronjob to execute a couple of rsync commands to copy files from efs to the local drive every 5 minutes.

I am having a problem now as its throughput is 100% and if i check IOPS metadata is using 100%. Any idea what to check?

Topics
Storage
Tags
Amazon Elastic File System
Language
English
Manu Sabu
asked 9 months ago398 views
1 Answer
0

To give a full answer would require understanding of the workloads that are running, and in particular what volume of data is being transferred in the cron jobs every five minutes, but it could be a case of the workloads being too much for the resources that it is being run on.

You mention lifecycling your data to IA, is this EFS Standard-IA or OneZone-IA? If EFS is running in Standard mode then you can increase throughput by recreating the filesystem in Max I/O mode https://docs.aws.amazon.com/efs/latest/ug/performance.html#performancemodes

Also,what throughput mode is the EFS filesystem running in https://docs.aws.amazon.com/efs/latest/ug/performance.html#throughput-modes

What are the EC2 instance types, and what are the EBS volume types, and is there any performance hit on these, or is it only the EFS filesystem that is showing a problem?

If EC2 or EBS are a bottleneck then consider an EBS-optimised instance type https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html and/or provisioned IOPS volumes https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html

If EC2 and EBS are comfortably within their limits then one other option that comes to mind is splitting the data across several EFS filesystems instead of just one, and thus spreading the load.

profile picture
EXPERT
Steve_M
answered 9 months ago
  • Manu Sabu
    9 months ago

    Okay. My situation is like this,

    The EFS storage has 26gb in use. 17GB in standard IA and 9gb in frequently access space. Inside EFS I have 6 directories. As mentioned earlier I have cronjob on 5 instances to do a sync for this directories from efs to local drive (rsync command for each directory) at Sametime( cron set with every 5 min). When we do rsync it will do the read operation for Metadata but I don't have any bulk write or update on all files. Maximum 100 files may updated in 30 min. Since it is read operation most of the files won't go to frequently access space once it moved to IA as they aren't checking this via POSIX way.

    All rsync jobs finishing with in 2 min on each box so it's not overlapping with next run

    In efs monitoring the Throughput of file system is 100% and when checking iops types it is saying Metadata is all time at 100%.

    So I am stuck with where to check without increasing resources. As this behavior started recently.

  • Steve_M EXPERT
    8 months ago

    https://repost.aws/knowledge-center/efs-choosing-correct-throughput-mode

    The baseline performance per GiB of file system storage is 50 KiB/s

    Your filesystem is 26GB which equates to 1.3MB/s.

    All EFS file systems, regardless of size, can burst up to 100 MiB/s of metered throughput, if they have burst credits.

    What does your burst credit balance look like?

    https://repost.aws/knowledge-center/efs-burst-credits

    If the credit balance of your file system drops to zero, then your permitted throughput rate drops to your baseline throughput. When driving at baseline throughput, you use credits at the same rate you earn them.

    You mention that this behaviour started recently. The above link also states:

    New file systems start out with 2.1 TiB of burst credits. This is why you can drive the new file system at burst throughput, despite having no data or metadata stored in the system. When the burst credits are used, then the performance of the new system is lowered.

    Could it be you have used up your burst credits, and so now you're maxed out by running constantly at 1.3MB/s?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content