API_S3_POST_signature

0

Hello, i don't arrive to create an post request for create an multpi part upload. Today, i arrive to upload or download a little file on my bucket but, with an upgrade, i would like upload an big file. In the documentation i see this : "https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html" for create the request, i use this : "https://docs.aws.amazon.com/fr_fr/general/latest/gr/sigv4-signed-request-examples.html" with the different change but, i can arrive to make the request, i have the error "Signature problem" i try different solution but, the result it's the same. Do you havec any solution for fixe my bug, or an other python code ? It's really important for my application.

# Copyright 2010-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# This file is licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
#
# ABOUT THIS PYTHON SAMPLE: This sample is part of the AWS General Reference 
# Signing AWS API Requests top available at
# https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
#

# AWS Version 4 signing example

# GET file from S3 bucket

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
from urllib.parse import urlparse
from requests_toolbelt.utils import dump

# ************* REQUEST VALUES *************
bucket_name = <name bucket>
region = 'eu-west-3'
endpoint = f'https://{bucket_name}.s3.{region}.amazonaws.com/test_slicer.waypoints'
method = 'POST'
service = 's3'
host = f'{bucket_name}.s3.{region}.amazonaws.com'
request_parameters = 'uploads'

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    print(f'key : {key}')
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    #for i in range(0,len(kDate)):
        #print(int(kDate[i]))
    #print(" ");
    kRegion = sign(kDate, regionName)
    #for i in range(0,len(kDate)):
        #print(int(kRegion[i]))
    #print(" ");
    kService = sign(kRegion, serviceName)
    #for i in range(0,len(kDate)):
        #print(int(kService[i]))
    #print(" ");
    kSigning = sign(kService, 'aws4_request')
    #for i in range(0,len(kDate)):
        #print(int(kSigning[i]))
    #print(" ");
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = <my acces key>
secret_key =<my secret key>
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
#amzdate="20220105T164100Z"


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = urlparse(endpoint).path

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").


payload_hash = hashlib.sha256((canonical_querystring).encode('utf-8')).hexdigest()

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-content-sha256:' + payload_hash + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-content-sha256;x-amz-date'

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
#print(canonical_request)

# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)


# Sign the string_to_sign using the signing_key

print(string_to_sign)
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
signature_bis = sign(signing_key,string_to_sign)

for i in range(0,len(signature_bis)):
    print(signature_bis[i])
    
#print(signature)

# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, 'x-amz-content-sha256':payload_hash, 'Content-Length': '7'
}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

with open('test_slicer.waypoints', 'rb') as data:
    r = requests.put(request_url,headers=headers, data=request_parameters)

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
#r = requests.put(request_url, headers=headers)

print(r.request.url)
print(r.request.body)
print(r.request.headers)

data = dump.dump_all(r)
print(data.decode('utf-8'))

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

print(signature)
print(canonical_uri)
print(sys.getsizeof(type(data)))

asked 2 years ago546 views
1 Answer
0

The first thing captured my sight is 'Content-Length': '7' in TASK 4. This header with value is in the request but never on the part of canonical_querystring. You can only add headers that were included in the signature of canonical query string

AWS
V
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions