내용으로 건너뛰기

S3 SMB share is 52TB actual data but shows as 2.7PB in CloudWatch

0

Basic SMB share S3 File Gateway - Software version - 1.27.13 US East 2

질문됨 일 년 전162회 조회

1개 답변
0

This discrepancy between your actual S3 data size (52TB) and what's showing in CloudWatch (2.7PB) is unusual and concerning. While there isn't a documented hard limit on the number of files an S3 File Gateway SMB share can hold (as shares are backed by S3, which is virtually unlimited), there are some important considerations that might explain what you're seeing:

  1. Metadata caching: S3 File Gateway maintains two types of cache - data cache and metadata cache. The metadata cache stores local copies of file metadata (name, size, attributes, ACLs) in the root disk. Depending on your gateway configuration, it can be set for 5M, 10M, or 20M metadata cache entries.

  2. Cache refresh issues: If changes are made directly in S3, they might not be immediately reflected in the SMB share view. This could potentially cause discrepancies in reporting. You can use the RefreshCache command to update the cache, but there might be latency involved.

  3. Performance considerations: With very large numbers of files or data volumes, there can be resource constraints related to the gateway VM's available memory, particularly for metadata handling.

  4. Storage class impact: The default storage class for objects put into an S3 bucket by the S3 File Gateway can be configured (S3_STANDARD, S3_INTELLIGENT_TIERING, S3_STANDARD_IA, or S3_ONEZONE_IA), which might affect how data is reported.

I would recommend:

  • Verifying your S3 storage size using S3 Storage Lens or the S3 console directly to confirm the actual storage used
  • Checking if the RefreshCache command has been run recently
  • Reviewing the gateway's configuration, particularly the metadata cache settings
  • Contacting AWS Support as this significant discrepancy might indicate an issue with the gateway or its reporting mechanism

This level of discrepancy (52TB vs 2.7PB) suggests there might be an underlying issue with how the data is being reported or calculated in CloudWatch.
Sources
S3 File Gateway file limit per SMB share | AWS re:Post
Changes made directly in S3 not reflected in SMB share after RefreshCache update | AWS re:Post
SMBFileShareInfo - Storage Gateway
Cutting-Edge Innovation, Yet No Simple Way to See Your S3 Storage Size? | AWS re:Post

답변함 일 년 전

전문가

검토됨 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.