S3 SHA256 Checksum for Presigned URL in File Upload

Question

How should we implement the presigned url creation for S3 file upload and use SHA256 checksum? Is there any code snippet?

I generated my presigned url with the below python code

import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME'},
  ExpiresIn=3600)
print(url)

Then I use that url to upload the file with below code

import requests
headers = {'x-amz-checksum-sha256': 'CHECKSUM_VALUE'}
with open('file.txt', 'r') as object_file:
  object_text = object_file.read()
  response = requests.put(url, data=object_text, headers=headers)
  print(response.content)

However it the response comes back with

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message> .....

How to fix the issue? Also, how to correctly compute the CHECKSUM_VALUE to be inputted as the value of x-amz-checksum-sha256 header? A Python example would be great.

Answer

When using the method `generate_presigned_url` the expected parameters are : 
```
ACL, Body, Bucket, CacheControl, ContentDisposition, ContentEncoding, ContentLanguage, ContentLength, ContentMD5, ContentType, Expires, GrantFullControl, GrantRead, GrantReadACP, GrantWriteACP, Key, Metadata, ServerSideEncryption, StorageClass, WebsiteRedirectLocation, SSECustomerAlgorithm, SSECustomerKey, SSECustomerKeyMD5, SSEKMSKeyId, SSEKMSEncryptionContext, BucketKeyEnabled, RequestPayer, Tagging, ObjectLockMode, ObjectLockRetainUntilDate, ObjectLockLegalHoldStatus, ExpectedBucketOwner
```
Hence the method does not provide us the posibility to use [S3 checksum feature](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html).

Another alternative is to use the method [`generate_presigned_post`](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.generate_presigned_post) in which we can specify the headers `x-amz-checksum-algorithm` and `x-amz-checksum-sha256` in the `Fields` and `Conditions` attributes, so we can have a code similar to the following :

```
import boto3
from botocore.config import Config
import requests

fileToUpload = 'filename'

s3_client = boto3.client(
        's3',
        config=Config(signature_version='s3v4'),
        region_name = 'us-west-2', endpoint_url='https://s3.us-west-2.amazonaws.com') # please feel free to change the region name and the endpoint region

bucketname = 'mybucketname'
keyname = 'myprefixname/filename'
checksumAlgo = 'SHA256'
checksumValue = '4tD+FYWmPsYAnIAW/43aEXample0BaTiPA/4EzkUgkk='

#Generate the presigned URL
response = s3_client.generate_presigned_post(
    Bucket = bucketname,
    Key = keyname,
    Fields = {'x-amz-checksum-algorithm':checksumAlgo,'x-amz-checksum-sha256':checksumValue},
    Conditions = [{'x-amz-checksum-algorithm':checksumAlgo},{'x-amz-checksum-sha256':checksumValue}],
    ExpiresIn = 3600
    )

#Upload file to S3 using presigned URL
headers = {'x-amz-checksum-algorithm':checksumAlgo,'x-amz-checksum-sha256':checksumValue}
file = { 'file': open(fileToUpload, 'rb')}

http_response = requests.post(response['url'], data=response['fields'], files=file, headers=headers)
print(http_response.status_code)

```

Please note that in case of successful requests the above code returns [HTTP status code 204](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html).

Answer

Your generate_presigned_url MUST include the headers that will be contained at the request. The 'url' didn't include the header 'x-amz-checksum-sha256', therefore it's failing with SignatureDoesNotMatch.

So generating presign URL must be changed to include the checksum headers.

As I know, metadata should be something like below.

```
import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME', "Metadata": {"filex-protocol" : "HTTPS", "filex-tracking": "12345"},
  ExpiresIn=3600)
print(url)
```
Above will generate presigned URL with x-amz-meta-filex-protocol and x-amz-meta-filex-tracking headers to be included at the actual request call. Not sure for the x-checksum. But you can try some approaches.

You can optionally add "Fields" instead of "Metadata" in Params, specifying exact header name that you want.
```
import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME',  Fields: {
        'x-amz-checksum-sha256': 'SOME_GENERATED_VALUE'
    },
  ExpiresIn=3600)
print(url)
```

S3 SHA256 Checksum for Presigned URL in File Upload

Relevanter Inhalt