How to get S3 to automatically calculate a sha256 checksum on file upload?

Question

I'm trying to do the following:

1. The client requests the server for a pre-signed URL. In the request body, the client also specifies the SHA256 hash of the file it wants to upload. This checksum is saved in the database before generating the pre-signed url.

2. The server sends the client the pre-signed URL, which was generated using the following command:

```typescript
    const command = new PutObjectCommand({
      Bucket: this.bucketName,
      Key: s3Key,
      // Include the SHA-256 of the file to ensure file integrity
      ChecksumSHA256: request.sha256Checksum, // base64 encoded
      ChecksumAlgorithm: "SHA256",
    })
```

3. This is where I notice a problem: Although I specified the sha256 checksum in the pre-signed URL, the client is able to upload any file to that URL i.e. if client sent sha256 checksum of file1.pdf, it is able to upload some_other_file.pdf to that URL. My expectation was that S3 would auto-reject the file if the checksums didn't match.. but that is not the case.

4. When this didn't work, I tried to include the `x-amz-checksum-sha256` header in the PUT request that uploads the file. That gave me a 'There were headers present in the request which were not signed` error.

The client has to call a 'confirm-upload' API after it is done uploading. Since the presigned-url allows any file to be uploaded, I want to verify the integrity of the file that was uploaded and also to verify that the client has uploaded the same file that it had claimed during pre-signed url generation.

So now, I want to know if there's a way for S3 to auto-calculate the SHA256 for the file on upload that I can retrieve using `HeadObjectCommand` or `GetObjectAttributesCommand` and compare with the value saved in the DB.

Note that I don't wish to use the CRC64 that AWS calculates.

re:Post Agent · Answer

Amazon S3 supports automatic checksum calculation for data integrity validation during uploads, including SHA-256. To achieve what you're trying to do, you need to make a few adjustments to your approach.

When generating a presigned URL with checksum validation, you need to ensure that the checksum is included as part of the signature calculation. This means the client must include the same checksum when uploading the file.

For your specific use case, you have two main options:

1. Use Content-MD5 header validation:
When generating the presigned URL, include the Content-MD5 header in the signed headers. This will require the client to provide the correct MD5 checksum when uploading, and S3 will reject uploads with mismatched checksums.

2. Use the ChecksumSHA256 parameter correctly:
When you include ChecksumSHA256 in your PutObjectCommand, you're telling S3 to expect this checksum in the upload. However, the client must also include this same checksum in the `x-amz-checksum-sha256` header when uploading.

To make the second approach work:
- When generating the presigned URL, make sure to include `x-amz-checksum-sha256` in the list of signed headers
- The client must include the same base64-encoded SHA-256 checksum in the `x-amz-checksum-sha256` header when uploading

After a successful upload, you can retrieve the stored SHA-256 checksum using the GetObjectAttributes or HeadObject operations. The checksum will be available in the object's metadata, and you can compare it with your database value.

If you're still having issues, ensure that:
1. The SHA-256 checksum is properly base64 encoded (not hex encoded)
2. The `x-amz-checksum-sha256` header is included in the signed headers list
3. The client is correctly including this header with the exact same value when uploading

This approach will ensure that S3 validates the integrity of the uploaded file against the expected SHA-256 checksum, and will reject uploads that don't match.  
**Sources**  
[Checking object integrity for data uploads in Amazon S3 - Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html)  
[Securing Amazon S3 presigned URLs for serverless applications | AWS Compute Blog](https://aws.amazon.com/blogs/compute/securing-amazon-s3-presigned-urls-for-serverless-applications/)  
[How do I calculate an AWS S3 compatible SHA-256 hash from a Blob in Angular? | AWS re:Post](https://repost.aws/questions/QU5irBWwpdQnKDUsBFRKAq_w/how-do-i-calculate-an-aws-s3-compatible-sha-256-hash-from-a-blob-in-angular)

How to get S3 to automatically calculate a sha256 checksum on file upload?

Add your answer

Relevant content