Issue with AWS Signature Calculation

0

I am encountering an issue while trying to calculate the AWS Signature for my requests. I have been following the AWS documentation and various examples, but I keep getting the following error:

"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been 'POST\n/endpoints/BackgroundCleanup-1677106144/invocations\n\nhost:runtime.sagemaker.us-east-1.amazonaws.com\nx-amz-date:20230630T101411Z\n\nhost;x-amz-date\n94e97978f60d2c26082a8a6828db14ed05348a5763e381ddfb5c4973a1f2d429'

The String-to-Sign should have been 'AWS4-HMAC-SHA256\n20230630T101411Z\n20230630/us-east-1/sagemaker/aws4_request\ne9f815121dcd09a6a9c731118c623fd168ebab1590f3111529a929c2d606d456'"

import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.security.InvalidKeyException
import java.security.MessageDigest
import java.text.SimpleDateFormat
import java.util.Date
import java.util.TimeZone
import java.net.URLDecoder
import java.net.URLEncoder
import java.util.LinkedHashMap
import java.util.Map
import java.util.TreeMap
import java.io.UnsupportedEncodingException

// Defined in User Defined Variables
def access_key = vars.get("aws_access_key")
def secret_key = vars.get("aws_secret_key")
def service = vars.get("aws_service_name")
def host = vars.get("aws_host")
def region = vars.get("aws_region")

log.info("Access Key: " + access_key)

// Obtain data from the HTTP Request Sampler
def method = sampler.getMethod()
def url = sampler.getUrl()
def req_path = url.getPath()
def req_query_string = orderQuery(url)
def request_parameters = ''

sampler.getArguments().each { arg ->
    request_parameters = arg.getStringValue().substring(1)
}

// Create the variable x-amz-date
def now = new Date()
def amzFormat = new SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'")
def stampFormat = new SimpleDateFormat("yyyyMMdd")
amzFormat.setTimeZone(TimeZone.getTimeZone("UTC")) // server timezone
def amzDate = amzFormat.format(now)
def dateStamp = stampFormat.format(now)
vars.put("x_amz_date", amzDate)

// Create a Canonical Request
def canonical_uri = req_path
def canonical_querystring = req_query_string
def canonical_headers = "host:" + host + "\n" + "x-amz-date:" + amzDate + "\n"
def signed_headers = "host;x-amz-date"
def payload_hash = getHexDigest(request_parameters)
def canonical_request = method + "\n" + canonical_uri + "\n" + canonical_querystring + "\n" + canonical_headers + "\n" + signed_headers + "\n" + payload_hash

log.info("canonical_request: " + canonical_request)

// Create the String to Sign
def algorithm = "AWS4-HMAC-SHA256"
def credential_scope = dateStamp + "/" + region + "/" + service + "/" + "aws4_request"
def hash_canonical_request = getHexDigest(canonical_request)
def string_to_sign = algorithm + "\n" + amzDate + "\n" + credential_scope + "\n" + hash_canonical_request
log.info("string_to_sign: " + string_to_sign)

// Calculate the Signing Key
def signing_key = getSignatureKey(secret_key, dateStamp, region, service)
def signature = hmac_sha256Hex(signing_key, string_to_sign)

// Add Signing information to Variable
def authorization_header = algorithm + " " + "Credential=" + access_key + "/" + credential_scope + ", " + "SignedHeaders=" + signed_headers + ", " + "Signature=" + signature
vars.put("aws_authorization", authorization_header)

def hmac_sha256(secretKey, data) {
    Mac mac = Mac.getInstance("HmacSHA256")
    SecretKeySpec secretKeySpec = new SecretKeySpec(secretKey, "HmacSHA256")
    mac.init(secretKeySpec)
    byte[] digest = mac.doFinal(data.getBytes())
    return digest
}

def hmac_sha256Hex(secretKey, data) {
    def result = hmac_sha256(secretKey, data)
    return result.encodeHex()
}

def getSignatureKey(key, dateStamp, regionName, serviceName) {
    def kDate = hmac_sha256(("AWS4" + key).getBytes(), dateStamp)
    def kRegion = hmac_sha256(kDate, regionName)
    def kService = hmac_sha256(kRegion, serviceName)
    def kSigning = hmac_sha256(kService, "aws4_request")
    return kSigning
}

def getHexDigest(text) {
    def md = MessageDigest.getInstance("SHA-256")
    md.update(text.getBytes())
    return md.digest().encodeHex()
}

public static String orderQuery(URL url) throws UnsupportedEncodingException {
    def orderQueryString = ""
    Map<String, String> queryPairs = new LinkedHashMap<>()
    String queryParams = url.getQuery()

    if (queryParams != null) {
        String[] pairs = queryParams.split("&")

        for (String pair : pairs) {
            int idx = pair.indexOf("=")
            queryPairs.put(URLDecoder.decode(pair.substring(0, idx), "UTF-8"), URLDecoder.decode(pair.substring(idx + 1), "UTF-8"))
        }
        def orderQueryArray = new TreeMap<>(queryPairs)
        orderQueryString = urlEncodeUTF8(orderQueryArray)
    }
    return orderQueryString
}

public static String urlEncodeUTF8(String s) {
    try {
        return URLEncoder.encode(s, "UTF-8")
    } catch (UnsupportedEncodingException e) {
        throw new UnsupportedOperationException(e)
    }
}

public static String urlEncodeUTF8(Map<?, ?> map) {
    StringBuilder sb = new StringBuilder()
    for (Map.Entry<?, ?> entry : map.entrySet()) {
        if (sb.length() > 0) {
            sb.append("&")
        }
        sb.append(String.format("%s=%s",
                urlEncodeUTF8(entry.getKey().toString()),
                urlEncodeUTF8(entry.getValue().toString())
        ))
    }
    return sb.toString()
}

log.info("req_path: " + req_path)
req_query_string = orderQuery(url)
log.info("req_query_string: " + req_query_string)
log.info("host: " + host)
log.info("amzDate: " + amzDate)

I have verified that the values for attributes such as aws_access_key, aws_secret_key, aws_service_name, aws_host, and aws_region are correct.

Could you please review my code and let me know if there's any mistake or if I'm missing any important steps in the signature calculation process? Any guidance or suggestions to resolve this issue would be greatly appreciated.

asked 10 months ago749 views
2 Answers
0

Hi, why don't you use SigV4 function provided by Python package botocore.auth? It will make your life much simpler: sigV4 is not easy to implement from scratch.

You have a code sample on how to use it at https://docs.aws.amazon.com/vpc-lattice/latest/ug/sigv4-authenticated-requests.html#sigv4-authenticated-requests-python

If you really want to implement it on you own, go to the source code of botocore.auth at https://github.com/boto/botocore/blob/develop/botocore/auth.py to see how it's done.

And to fully understand, the full spec is here: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html

Hope it helps

Didier

profile pictureAWS
EXPERT
answered 10 months ago
profile picture
EXPERT
reviewed 10 months ago
0

I am trying to perform load tests on the sagemaker endpoint hence using the jmeter tool which only accepts Java Language How else can I achieve this load test/performance test on the endpoint

answered 10 months ago
  • The code that you shared was not Java. But I updated my answer above for Java so that you can accept it

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions