- Newest
- Most votes
- Most comments
Hello,
-
To handle your 240TB data repository with FSx File Cache the best approach is to ensure that the total size of your data repository does not exceed the cache capacity.
-
If you have a single large file of 240TB in S3, you can configure FSx File Cache to handle it, but the cache will be fully utilized by this file, leaving no room for additional data.
-
To optimize this setup, consider breaking the large file into smaller segments if possible, and configure the cache to store only frequently accessed portions of the data.
-
For less critical or infrequently accessed data, access it directly from S3, bypassing the cache.
-
you can efficiently manage your data within the cache's capacity while ensuring quick access to the most important parts.
https://docs.aws.amazon.com/fsx/latest/FileCacheGuide/limits.html
Yes, you can set up a 240TB FSx File Cache connected to a 240TB data repository. The cache only saves the parts of files that you access, not the whole file. This means that if you have a 240TB file in S3, only the portions you actually use will be stored in the cache, making it easier to manage large files.
As long as your FSx File Cache is at least the same size as the total data you intend to cache, this configuration will function effectively. This allows you to manage large data repositories without needing to cache entire files all at once.
Relevant content
- asked 2 years ago
- asked 2 years ago
Thank you Parthasaradi, my another question is，when i preload my file into filecache system，how do i make sure that it wont start cache eviction. For example my filecache storage capacity is 20TB, if i wanna preload three different file paths, and each path got 10TB. After i preload path A and B, cache is full, when i preload path C, is there anyway i can retain the data i want to use in A and B. How the preload and eviction works in this situation?
Hello Yifei, Preload and frequently access the important data paths A and B to keep it in the cache. If needed, increase the cache size to avoid eviction.