S3 File Gateway - large files and robocopy error

Question

I have an S3 File Gateway on EC2 in the same region as an EC2 File server on Windows 2022. Both instances are r5b.xlarge

S3 Gateway Server has 4x 2000gb gp3 drives.

I am using robocopy on the Windows server to move 10's of TB of files to the S3 File gateway.   On large files I often see errors like below  90gig+ files usually

**2023/08/21 15:26:02 ERROR 64 (0x00000040) Copying File F:\somesourcefolder\somefile2.zip
The specified network name is no longer available.**

Robocopy command like below
robocopy F:\someLocalsource \S3GatewayServer\SomeFolder /J  /COPY:DAT /S /E /MT:8

Sometimes it retries and the copy works and sometimes it just goes onto the next files and skips the large file.  There's no indication of the S3 gateway instance being down. The same copy works fine between 2 windows servers in the same region.

I can see on the gateway cloudwatch graph that the dump to S3 is happening at just about the same speed as the smb copy to the gateway from the file server so I don't think it's getting too swamped.   I expect that the reported error is somewhat generic and doesn't really mean the network is down or the s3 gateway is down but don't know where to troubleshoot.

Accepted Answer

Hello,

Please check if the Cache%Dirty metric on the gateway to see if it is above 80% during the copy. Gateway starts throttling the reads and writes to the cache if the metric is above 80%. However,  if the large file copy still fails even with the *CachePercentDirty* metric being less than 80 percent, then your File Gateway might be hitting a client-side session timeout.

For SMB, you can increase this timeout using the PowerShell command below. Running this command sets the timeout to 1800 seconds. The default value is 60 Seconds.

***Set-SmbClientConfiguration -SessionTimeout 1800***

Also, you can alternatively set the SMB session timeout on the SMB client by updating the registry setting at *HKEY_LOCAL_MACHINE\SYSTEM\CURRENTCONTROLSET\SERVICES\ 
   LANMANWORKSTATION\PARAMETERS\SESSTIMEOUT*.

Ref - https://docs.aws.amazon.com/filegateway/latest/files3/troubleshooting-file-gateway-issues.html#backup-job-fails

If the above suggestion did not address the issue, please open a case with AWS support for further investigation.

S3 File Gateway - large files and robocopy error

相关内容