S3 SHA256 Checksum for Presigned URL in File Upload

0

How should we implement the presigned url creation for S3 file upload and use SHA256 checksum? Is there any code snippet?

I generated my presigned url with the below python code

import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME'},
  ExpiresIn=3600)
print(url)

Then I use that url to upload the file with below code

import requests
headers = {'x-amz-checksum-sha256': 'CHECKSUM_VALUE'}
with open('file.txt', 'r') as object_file:
  object_text = object_file.read()
  response = requests.put(url, data=object_text, headers=headers)
  print(response.content)

However it the response comes back with

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message> .....

How to fix the issue? Also, how to correctly compute the CHECKSUM_VALUE to be inputted as the value of x-amz-checksum-sha256 header? A Python example would be great.

asked 9 months ago1282 views
2 Answers
0

Your generate_presigned_url MUST include the headers that will be contained at the request. The 'url' didn't include the header 'x-amz-checksum-sha256', therefore it's failing with SignatureDoesNotMatch.

So generating presign URL must be changed to include the checksum headers.

As I know, metadata should be something like below.

import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME', "Metadata": {"filex-protocol" : "HTTPS", "filex-tracking": "12345"},
  ExpiresIn=3600)
print(url)

Above will generate presigned URL with x-amz-meta-filex-protocol and x-amz-meta-filex-tracking headers to be included at the actual request call. Not sure for the x-checksum. But you can try some approaches.

You can optionally add "Fields" instead of "Metadata" in Params, specifying exact header name that you want.

import boto3
url = boto3.client('s3').generate_presigned_url(
  ClientMethod='put_object', 
  Params={'Bucket': 'MY_BUCKET, 'Key': 'FILE_NAME',  Fields: {
        'x-amz-checksum-sha256': 'SOME_GENERATED_VALUE'
    },
  ExpiresIn=3600)
print(url)
V
answered 9 months ago
0

When using the method generate_presigned_url the expected parameters are :

ACL, Body, Bucket, CacheControl, ContentDisposition, ContentEncoding, ContentLanguage, ContentLength, ContentMD5, ContentType, Expires, GrantFullControl, GrantRead, GrantReadACP, GrantWriteACP, Key, Metadata, ServerSideEncryption, StorageClass, WebsiteRedirectLocation, SSECustomerAlgorithm, SSECustomerKey, SSECustomerKeyMD5, SSEKMSKeyId, SSEKMSEncryptionContext, BucketKeyEnabled, RequestPayer, Tagging, ObjectLockMode, ObjectLockRetainUntilDate, ObjectLockLegalHoldStatus, ExpectedBucketOwner

Hence the method does not provide us the posibility to use S3 checksum feature.

Another alternative is to use the method generate_presigned_post in which we can specify the headers x-amz-checksum-algorithm and x-amz-checksum-sha256 in the Fields and Conditions attributes, so we can have a code similar to the following :

import boto3
from botocore.config import Config
import requests

fileToUpload = 'filename'

s3_client = boto3.client(
        's3',
        config=Config(signature_version='s3v4'),
        region_name = 'us-west-2', endpoint_url='https://s3.us-west-2.amazonaws.com') # please feel free to change the region name and the endpoint region

bucketname = 'mybucketname'
keyname = 'myprefixname/filename'
checksumAlgo = 'SHA256'
checksumValue = '4tD+FYWmPsYAnIAW/43aEXample0BaTiPA/4EzkUgkk='

#Generate the presigned URL
response = s3_client.generate_presigned_post(
    Bucket = bucketname,
    Key = keyname,
    Fields = {'x-amz-checksum-algorithm':checksumAlgo,'x-amz-checksum-sha256':checksumValue},
    Conditions = [{'x-amz-checksum-algorithm':checksumAlgo},{'x-amz-checksum-sha256':checksumValue}],
    ExpiresIn = 3600
    )

#Upload file to S3 using presigned URL
headers = {'x-amz-checksum-algorithm':checksumAlgo,'x-amz-checksum-sha256':checksumValue}
file = { 'file': open(fileToUpload, 'rb')}

http_response = requests.post(response['url'], data=response['fields'], files=file, headers=headers)
print(http_response.status_code)

Please note that in case of successful requests the above code returns HTTP status code 204.

answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions