By using AWS re:Post, you agree to the Terms of Use

Signature_don't_match_API

0

Hello, I developpe an application. I would like create relation with my storage with the api. For this, i use c++ and the lib curl for create the link. But, whene i try to understand the signautre, i don't understand and, i don't find any simple solution for calcul the signature. I put my code un buttom.

curl_easy_setopt(curl,CURLOPT_URL,"http://<bucket>.s3.eu-west-3.amazonaws.com" ); 
        curl_easy_setopt(curl, CURLOPT_CAINFO, pCertFile);               
        curl_easy_setopt(curl, CURLOPT_PUT, 1L);                             
        sprintf(autorization,"Authorization: AWS4-HMAC-SHA256 Credential=<key id>/20220103/eu-west-3/s3/aws4_request, SignedHeaders=x-amz-content-sha256;host;x-amz-date;x-amz-meta-author, Signature=<Signature (this) >");           
        list = curl_slist_append(list,autorization);

       sprintf(autorization, "x-amz-content-sha256: <content (and this)>");
       list = curl_slist_append(list,autorization);

        sprintf(autorization,"x-amz-date: Mon, 03 Jan 2022 16:00:00 GMT");           
        list = curl_slist_append(list,autorization);

        sprintf(autorization,"content-Type: image/jpeg");           
        list = curl_slist_append(list,autorization);

        sprintf(autorization,"Content-Length: 94328");          
        list = curl_slist_append(list,autorization);

        sprintf(autorization,"x-amz-meta-author: Janet");           
        list = curl_slist_append(list,autorization);

        sprintf(autorization,"Expect: 100-continue");           
        list = curl_slist_append(list,autorization);

        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);          
        curl_easy_setopt(curl, CURLOPT_READDATA,fileToUpload);     

        res=  curl_easy_perform(curl);                            

        printf("%s",res);
        curl_easy_cleanup(curl);                                 

my code don't have any comment beacause, my commente was in french. If you know any code in c or c++ for calculate the signature, or if i have any error, can you signal me please ?

(On my code, the any string on ' " ' it's the header for the api)

5 Answers
1

Is there a valid reason why you decided to avoid using AWS SDK for C++ and create a new storage API client from scratch? AWS SDK library will make sure to add the necessary security context to the calls, so that you can focus on your business logic alone.

If you still want to create your own SigV4 algorithm implementation, there is a good example for Python in this article: Examples of the complete Signature Version 4 signing process (Python). You can use debugger to step through that example and compare all intermediate calculation values (and the signature itself) with your C++ code behavior and try to spot the difference.

Or you can take a look at AWS SDK for C++ implementation and see if you can adapt it to your custom code.

answered 7 months ago
1

x-amz-content-sha256 value doesn't look like a valid hash for an empty string, moreover, you should be using HTTPS endpoints, not HTTP!

Anyway, the below code worked for me to download a test.txt file from a demo bucket

# Copyright 2010-2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# This file is licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
#
# ABOUT THIS PYTHON SAMPLE: This sample is part of the AWS General Reference 
# Signing AWS API Requests top available at
# https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
#

# AWS Version 4 signing example

# GET file from S3 bucket

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
from urllib.parse import urlparse

# ************* REQUEST VALUES *************
bucket_name = '<your_bucket_name>'
region = 'us-west-2'
endpoint = f'https://{bucket_name}.s3.{region}.amazonaws.com/test.txt'
method = 'GET'
service = 's3'
host = urlparse(endpoint).hostname
request_parameters = ''

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = os.environ.get('AWS_ACCESS_KEY_ID')
secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY')
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = urlparse(endpoint).path

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-content-sha256:' + payload_hash + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-content-sha256;x-amz-date'

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, 'x-amz-content-sha256':payload_hash}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
answered 7 months ago
0

Hello, I don't use the SDK beacause is so biggest for my utilisation. I would like use just 3 or 4 function. For this, i implement the solution on my application. Thank's for your answer, i try with the python code but, i have the same error...

#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
#
# ABOUT THIS PYTHON SAMPLE: This sample is part of the AWS General Reference 
# Signing AWS API Requests top available at
# https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
#

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac 
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'PUT'
service = 's3'
host = '<bucket>.s3.amazonaws.com'
region = 'eu-west-3'
endpoint = 'http://<bucket>.s3.eu-west-3.amazonaws.com'
request_parameters = ''

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = '<acces_key>'
secret_key = '<secret key>'
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/' 

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-content-sha256:' + '5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7' + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-content-sha256;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-content-sha256':'5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7','x-amz-date':amzdate, 'Authorization':authorization_header}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

print('\n' +  signature)
print('\n' +  datestamp)
print('\n' +  region)
print('\n' +  service)
print('\n'+ string_to_sign)
print('\n'+ canonical_request)

I use the code in python for "Using GET with authentication information in the Query string (Python)" and it's don't work

answered 7 months ago
0

thank's for your reply ! It's work, i can create PUT and GET request. I try to translate in C++ but i have the same probleme "SignatureDoesNotMatch" and i don't understand why. I make the same request, the signature it's the same like the python when i fixe the the date.

#include <stdio.h>
#include <curl/curl.h>
#include <iostream>
#include "sha256_calc.h"
#include <ctime>
#include <hmac_sha256/hmac_sha256.h>
#include <vector>

using std::string;
using std::cout;
using std::endl;


#define BUCKET_NAME "<my bucket>"
#define REGION "eu-west-3"
#define METHODE_GET "GET"
#define SERVICE_S3 "s3"
#define ALGORITHM "AWS4-HMAC-SHA256"

#define ACCESS_KEY "<my acces key>"
#define SECRET_KEY "<my secret key>"

#define SHA256_HASH_SIZE 32

size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream) {
    size_t written = fwrite(ptr, size, nmemb, stream);
    return written;
}


string vector_to_string (std::vector<uint8_t> val)
{
    string retour = "";

    char res[2048];
    char intermedaire[2];

    for (int i = 0; i < SHA256_HASH_SIZE; i++)
    {
        //sprintf(intermedaire,"%x",val[i]);
        retour += val[i];
    }
    return retour;
}

std::vector<uint8_t> sign(string key,string msg)
{
     std::vector<uint8_t> out(SHA256_HASH_SIZE);
     hmac_sha256(key.data(), key.size(), msg.data(), msg.size(),out.data(), out.size());
     return out;
}


std::vector<uint8_t> getSignatureKey(string key,string dateStamp,string regionName,string serviceName)
{
    printf("key : %s \n" , key.c_str());

    std::vector<uint8_t> kDate = sign("AWS4" + key, dateStamp);
    for (int i = 0; i < SHA256_HASH_SIZE; i++)
    {
        printf("%d\n",kDate[i]);
    }
    printf("\n");

    string buff = vector_to_string (kDate);
    std::vector<uint8_t> kRegion = sign(buff, regionName);
    for (int i = 0; i < SHA256_HASH_SIZE; i++)
    {
        printf("%d\n",kRegion[i]);
    }
    printf("\n");

    buff = vector_to_string (kRegion);
    std::vector<uint8_t> kService = sign(buff, serviceName);
    for (int i = 0; i < SHA256_HASH_SIZE; i++)
    {
        printf("%d\n",kService[i]);
    }
    printf("\n");

    buff = vector_to_string (kService);


    std::vector<uint8_t> kSigning = sign(buff, "aws4_request");
    for (int i = 0; i < SHA256_HASH_SIZE; i++)
    {
        printf("%d\n",kSigning[i]);
    }
    printf("\n");

     return kSigning;
}


int main(void)
{
  CURL *curl;
  CURLcode res;
  curl_slist *chunk=NULL;

  char *file = (char*)"toto.waypoints";
  char end_point[2048];
  char host[1024];
  sprintf(host,"%s.s3.%s.amazonaws.com",BUCKET_NAME,REGION);
  sprintf(end_point,"https://%s/toto.waypoints",host);


  time_t now = time(0);
  tm *gmtm = gmtime(&now);

  char amzdate[32];
  char datestamp[32];
  sprintf(amzdate,"%04d%02d%02dT%02d%02d%02dZ",gmtm->tm_year+1900,gmtm->tm_mon+1,gmtm->tm_mday,gmtm->tm_hour,gmtm->tm_min,gmtm->tm_sec);
  //sprintf(amzdate,"%04d%02d%02dT%02d%02d%02dZ",1900+gmtm->tm_year,gmtm->tm_mon+1,gmtm->tm_mday,8,45,25);
  sprintf(datestamp,"%04d%02d%02d",1900+gmtm->tm_year,gmtm->tm_mon+1,gmtm->tm_mday);

  cout << gmtm->tm_hour << endl;


  string canonical_uri;
  canonical_uri = (string)"/" + file;

  string test = "";
  string payload_hash = sha256(test);

  string canonical_headers;
  canonical_headers = (string)"host:" + host + "\nx-amz-content-sha256:" + payload_hash + "\nx-amz-date:" + amzdate + "\n";

  string signed_header = "host;x-amz-content-sha256;x-amz-date";

  string canonicale_request;
  canonicale_request = (string) METHODE_GET + "\n" + canonical_uri + "\n\n" + canonical_headers + "\n" + signed_header + "\n" + payload_hash;

  printf ("%s\n",canonicale_request.c_str());

  string credential_scope;
  credential_scope = (string) datestamp + "/" + REGION + "/" + SERVICE_S3 + "/aws4_request";


  string string_to_sign;
  string_to_sign = (string) ALGORITHM +  "\n" + amzdate + "\n" + credential_scope + "\n" + sha256(canonicale_request);

  char *pCACertFile = (char*)"C:/Users/merop/Documents/build-test_api-Desktop_Qt_5_15_2_MinGW_32_bit-Debug/debug/cacert.pem";


  //string secret_key = "S4NeREO6sKegaXRQjZAEpAGrR0Fg3IiiCsz2rf5r";
  //string msg = "20220105";

  std::vector<uint8_t> signing_key ;
  signing_key  = getSignatureKey(SECRET_KEY,datestamp,REGION,SERVICE_S3);
  string buff = vector_to_string(signing_key);

  std::vector<uint8_t> signature(SHA256_HASH_SIZE);
  string string_to_sign_buff = string_to_sign;

  hmac_sha256(buff.data(), buff.size(), string_to_sign_buff.data(), string_to_sign_buff.size(),signature.data(), signature.size());

  printf("\n");
  for (int i = 0; i < SHA256_HASH_SIZE; i++)
  {
      printf("%d\n",signing_key[i]);
  }
  printf("\n");


  printf("\n %s \n",string_to_sign_buff.c_str());

  for (int i = 0; i < SHA256_HASH_SIZE; i++)
  {
      printf("%d\n",signature[i]);
  }


  FILE *fileOpen = fopen("<file directory>/toto.txt","wb");
  if (fileOpen)
  {
      printf("fichier ok \n");
  }
  else
  {
      printf("erreur\n");
  }

  FILE *f = fopen(pCACertFile,"r");


  if (f)
  {
      fclose(f);
  }
  else
  {
      printf("certificat abscent");
  }

  string buffUser="";
  curl=curl_easy_init();
  curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0);
  curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);

  curl_easy_setopt(curl, CURLOPT_URL, end_point);
  curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);

  buff = credential_scope;
  buff = ACCESS_KEY + string("/") + buff;
  printf("%s\n",buff.c_str());


  string test_val = "";
  char tampon[8];
  for (int i = 0; i < SHA256_HASH_SIZE; i++)
  {
      sprintf(tampon,"%02x",signature[i]);
      test_val += tampon;
  }

  buffUser = (string) "Authorization:"+ ALGORITHM +" Credential="+ ACCESS_KEY + "/" + credential_scope + ",SignedHeaders=" + signed_header + ",Signature=" + test_val;
  chunk = curl_slist_append(chunk,buffUser.c_str());

  buffUser = "x-amz-content-sha256:" + payload_hash;
  chunk = curl_slist_append(chunk,buffUser.c_str());

  buff = amzdate;
  buffUser = "x-amz-date:" + buff;
  chunk = curl_slist_append(chunk,buffUser.c_str());

  curl_easy_setopt(curl,CURLOPT_HTTPHEADER,chunk);

     //curl_easy_setopt(curl, CURLOPT_UPLOAD, true);
     //curl_easy_setopt(curl, CURLOPT_PROTOCOLS, CURLPROTO_HTTPS);
   curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
  curl_easy_setopt(curl,CURLOPT_WRITEDATA,fileOpen);


  //curl_easy_setopt(curl, CURLOPT_INFILESIZE, filesize('path_to_file'));
  //cout << result << endl;

    res = curl_easy_perform(curl);
    if(res != CURLE_OK)
      fprintf(stderr, "curl_easy_perform() failed: %s\n",
              curl_easy_strerror(res));

    curl_easy_cleanup(curl);
 // }
    fclose (fileOpen);
  printf("Fin programme \n");
  return 0;
}

[EDIT] I fixe the bug, the probleme was create by the library CURL, it's include the otpion automitcly. For fixe this problme, i replace curl_easy_setopt(curl, CURLOPT_PUT, 1); by curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");

After this, i have a new problem (in c++ and python, the application send nothing, i just have the name on my bucket and the file size is 0 octet....

answered 7 months ago
0
#
# This file is licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
# OF ANY KIND, either express or implied. See the License for the specific
# language governing permissions and limitations under the License.
#
# ABOUT THIS PYTHON SAMPLE: This sample is part of the AWS General Reference 
# Signing AWS API Requests top available at
# https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
#

# AWS Version 4 signing example

# GET file from S3 bucket

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests # pip install requests
from urllib.parse import urlparse
from requests_toolbelt.utils import dump

# ************* REQUEST VALUES *************
bucket_name = 'meropy'
region = 'eu-west-3'
endpoint = f'https://{bucket_name}.s3.{region}.amazonaws.com/test_slicer.waypoints'
method = 'PUT'
service = 's3'
host = f'{bucket_name}.s3.{region}.amazonaws.com'
request_parameters = 'test_slicer.waypoints'

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    print(f'key : {key}')
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    for i in range(0,len(kDate)):
        print(int(kDate[i]))
    print(" ");
    kRegion = sign(kDate, regionName)
    for i in range(0,len(kDate)):
        print(int(kRegion[i]))
    print(" ");
    kService = sign(kRegion, serviceName)
    for i in range(0,len(kDate)):
        print(int(kService[i]))
    print(" ");
    kSigning = sign(kService, 'aws4_request')
    for i in range(0,len(kDate)):
        print(int(kSigning[i]))
    print(" ");
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = 'AKIARYYNP37IMEEQMEGC'
secret_key = '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'
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
#amzdate="20220105T164100Z"


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = urlparse(endpoint).path

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").

# payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest() last code
with open("test_slicer.waypoints",'rb') as f:            # NEW code 
    test = f.read()                                                          # NEW code 
print(len(test))                                                            # NEW code 

payload_hash = hashlib.sha256((test)).hexdigest() # NEW code 

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-content-sha256:' + payload_hash + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-content-sha256;x-amz-date'

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
print(canonical_request)

# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)


# Sign the string_to_sign using the signing_key

print(string_to_sign)
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
signature_bis = sign(signing_key,string_to_sign)

for i in range(0,len(signature_bis)):
    print(signature_bis[i])
    
print(signature)

# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header, 'x-amz-content-sha256':payload_hash,'content-length':'1'}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
#r = requests.put(request_url, headers=headers) Last code
with open('test_slicer.waypoints', 'rb') as data:
    r = requests.put(request_url,headers=headers, data=data) # New code

print(r.request.url)
print(r.request.body)
print(r.request.headers)

data = dump.dump_all(r)
print(data.decode('utf-8'))

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

print(signature)
print(canonical_uri)

The program send 0o. The file was create but, the size is 0o.

[EDIT] I fixe the bug with add parameters. today, everything work !

Thank's four your repley and help!

answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions