datasync transfer and verification errors to S3 fails with verification failed to S3 Put Object

Question

I am seeing an error that the execution failed with "Complete errorLogs with verification error, Tranfer and verification completed. verification detected mismatches. Files with mismatched are listed in cloudwatch Logs" Furtherchecking the cloudwatch logs we do see the following error.

"[ERROR] Deferred error: s0:c58 close (""s3://test//yesh/test.txt""): 40978 (s3 Put Object Failed)"

Any input on this error?

"[ERROR] Deferred error: s0:c58 close (""s3://test//yesh/test.txt""): 40978 (s3 Put Object Failed)"

Any input on this error?

Accepted Answer

This error occurs because the destination S3 bucket is encrypted with customer managed KMS keys and the IAM role and KMS policy do not have sufficient permissions to perform the Put object action. Though some customers added limited KMS keys it still fails as "kms:GenerateDataKey" is not present on both IAM role policy and KMS policy.

Resolution

Check the following:

- Check whether the S3 bucket is encryption enabled and the type of KMS.

- Make sure the IAM role policy and KMS key policy with this role has the following minimum permissions:
       "kms:Encrypt",
      "kms:Decrypt",
      "kms:ReEncrypt*",
      "kms:DescribeKey",
      "kms:GetPublicKey",
      "kms:ReEncrypt*",
      "kms:GenerateDataKey",

- Make sure the IAM role Trust policy is as per the Example 1: in this documentation:
https://docs.aws.amazon.com/datasync/latest/userguide/using-identity-based-policies.html

datasync transfer and verification errors to S3 fails with verification failed to S3 Put Object

Relevant content