Generation of presigned URL to S3 with temporary credentials - InvalidAccessKeyId

0

I have been working on adding support for presigned URLs in aws-iot-device-sdk-embedded-C and I believe I'm very close.

I started from the demo to download from S3 with a temporary credentials user profile, and I adjusted my signature comparing with the CLI (aws s3 presign) with same data.

The signature is now OK but still while the temporary AccessKeyId is able to download the file, when I use the (printed to terminal) presigned URL I get instead <Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message>

I'm working on https://github.com/giuspen/aws-iot-device-sdk-embedded-C/blob/GP_http_demo_s3_download_test_signature2/demos/http/http_demo_s3_download/http_demo_s3_download.c

I'm puzzled, is there an additional procedure to record the AccessKeyId in order to generate the presigned URL?

EXAMPLE LOG:

[INFO] [DEMO] [http_demo_s3_download.c:1752] HTTP Client Synchronous S3 download demo using temporary credentials fetched from iot credential provider:
c10nfdpw0ux517.credentials.iot.eu-central-1.amazonaws.com
[INFO] [DEMO] [http_demo_s3_download.c:910] Establishing a TLS session with c10nfdpw0ux517.credentials.iot.eu-central-1.amazonaws.com:443.
[INFO] [DEMO] [http_demo_s3_download.c:766] AWS IoT credentials will expire after this timestamp: 2023-10-02T13:30:09Z.
[INFO] [DEMO] [http_demo_s3_download.c:650] AWS IoT credential provider response: HTTP/1.1 200 OK
content-type: application/json
content-length: 1165
date: Mon, 02 Oct 2023 12:30:09 GMT
x-amzn-RequestId: fd0bb5c9-754f-9486-6cfa-2529899e17c8
{"credentials":{"accessKeyId":"ASIAVBKNXEL5HP2B4VXX","secretAccessKey":"****************************************","sessionToken":"IQoJb3JpZ2luX2VjEA0aDGV1LWNlbnRyYWwtMSJHMEUCIFsKZ0d7JLu+P1S0AfpaXUGuW5/agg0/3+joQt+D2oirAiEA+fCfmAqo9zt8Xb5CcuJj7He77UQQcgq/HMaZNJljQKcq3gMIFhAAGgwzNDY0NDQ3MzUyMjYiDCk8oHzTPPEymfx9Kiq7AwvmDkvcH5RPb9RWtAoci/WO/UhI4D1CQ5kg2X7KEUbpdk/ig8W6hSMhEcFWYEfMNpryhicDB/XoJzpjfvPxccOewA6Rd/y+2qFzNOOc2Cba3GtovdGctYJBtvex6du367kJoYks6PtaKpxFXyA6oEHWQ2X0yy9Ai+bZdPX9woHEJOQ7x9Orr6msJo47wSFgF9R5JlPk2n1mvLuI1kQlzPbgakTSwkSYb0hR91Sbd9V9kWQzQsK1GI3TktJ9YkPM0EIP5FMlLtwBPa0IYNMpHT1mz5LnXg/Imoepb6sZtsfW/M1LhkbgL//8sTeE5HHUXIL0RbRRSEQSctFPETIozfmKDp+yGNeOgFXDKwuY7aeKTzw1URnKHSM9mN5d5BVCdIjFu4obUgOLrL1tSVK3ai+54d1pXplNkcT1RDuZUKPvdj1eNDb46Ykd70LbtuQCHZNJQLeIxOQ/XbH3slf49UXp6Gwr8ENk888U+w8W3Emi1uZ6OkM4X00DKZaUksQCH1/my7tAwOP/RZNypXiQj6d33COeu2W92NvTt74xH8XrUEBOxxAQuDTasOTf8SK4E+y7Q9GoB0tb9CQiMNHv6qgGOpoB3LgWRNLccumU7fynfuXkdXoi3utGH7UpWqs8zz2mImgSJccDFg8NEyT9x+yhdcitTiJ7lugAFNgKg1qTpphL6i0CRDExX6JyYYwXU1l3BxSxLMSehYlLWpJV+/SCOVqur9ps+7dC3M3+hjuSG3jusANCGbbINzink/mQmXSHDnGL8zcbV6YpLPEH+DVXuz4cFW3FT2jbkjuV4g==","expiration":"2023-10-02T13:30:09Z"}}.
[INFO] [DEMO] [http_demo_s3_download.c:961] Establishing a TLS session with otahalo.s3.eu-central-1.amazonaws.com:443.
[INFO] [DEMO] [http_demo_s3_download.c:1478] Getting file object size from host...
[INFO] [DEMO] [http_demo_s3_download.c:1652] Received successful response from server (Status Code: 206).
[INFO] [DEMO] [http_demo_s3_download.c:1698] The file is 58 bytes long.
[INFO] [DEMO] [http_demo_s3_download.c:1176] Downloading bytes 0-57, out of 58 total bytes, from otahalo.s3.eu-central-1.amazonaws.com...:  
[INFO] [DEMO] [http_demo_s3_download.c:1206] Response Body:
{
"filename"="update-bundle-imx8mnevk-v01-00-01.raucb"
}
[INFO] [DEMO] [http_demo_s3_download.c:1307] Getting presigned URL...
[INFO] [DEMO] [http_demo_s3_download.c:1410] ota_temp_url=
https://otahalo.s3.eu-central-1.amazonaws.com/ota-v01-00-01.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAVBKNXEL5HP2B4VXX%2F20231002%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20231002T123009Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEA0aDGV1LWNlbnRyYWwtMSJHMEUCIFsKZ0d7JLu%2BP1S0AfpaXUGuW5%2Fagg0%2F3%2BjoQt%2BD2oirAiEA%2BfCfmAqo9zt8Xb5CcuJj7He77UQQcgq%2FHMaZNJljQKcq3gMIFhAAGgwzNDY0NDQ3MzUyMjYiDCk8oHzTPPEymfx9Kiq7AwvmDkvcH5RPb9RWtAoci%2FWO%2FUhI4D1CQ5kg2X7KEUbpdk%2Fig8W6hSMhEcFWYEfMNpryhicDB%2FXoJzpjfvPxccOewA6Rd%2Fy%2B2qFzNOOc2Cba3GtovdGctYJBtvex6du367kJoYks6PtaKpxFXyA6oEHWQ2X0yy9Ai%2BbZdPX9woHEJOQ7x9Orr6msJo47wSFgF9R5JlPk2n1mvLuI1kQlzPbgakTSwkSYb0hR91Sbd9V9kWQzQsK1GI3TktJ9YkPM0EIP5FMlLtwBPa0IYNMpHT1mz5LnXg%2FImoepb6sZtsfW%2FM1LhkbgL%2F%2F8sTeE5HHUXIL0RbRRSEQSctFPETIozfmKDp%2ByGNeOgFXDKwuY7aeKTzw1URnKHSM9mN5d5BVCdIjFu4obUgOLrL1tSVK3ai%2B54d1pXplNkcT1RDuZUKPvdj1eNDb46Ykd70LbtuQCHZNJQLeIxOQ%2FXbH3slf49UXp6Gwr8ENk888U%2Bw8W3Emi1uZ6OkM4X00DKZaUksQCH1%2Fmy7tAwOP%2FRZNypXiQj6d33COeu2W92NvTt74xH8XrUEBOxxAQuDTasOTf8SK4E%2By7Q9GoB0tb9CQiMNHv6qgGOpoB3LgWRNLccumU7fynfuXkdXoi3utGH7UpWqs8zz2mImgSJccDFg8NEyT9x%2ByhdcitTiJ7lugAFNgKg1qTpphL6i0CRDExX6JyYYwXU1l3BxSxLMSehYlLWpJV%2B%2FSCOVqur9ps%2B7dC3M3%2BhjuSG3jusANCGbbINzink%2FmQmXSHDnGL8zcbV6YpLPEH%2BDVXuz4cFW3FT2jbkjuV4g%3D%3D%22%2C%22expiration%22%3A%222023-10-02T13%3A30%3A09Z%22%7D%7D&X-Amz-Signature=fd138ce95b8bf62db8ba1a0cf1aea3147a48e4c95231b44e2c4a6a30aebd52b7
[INFO] [DEMO] [http_demo_s3_download.c:1856] Demo iteration 1 is successful.
[INFO] [DEMO] [http_demo_s3_download.c:1875] Demo completed successfully.

NOW If I use the presigned URL:

<Error>
  <Code>InvalidAccessKeyId</Code>
  <Message>The AWS Access Key Id you provided does not exist in our records.</Message>
  <AWSAccessKeyId>ASIAVBKNXEL5HP2B4VXX</AWSAccessKeyId>
  <RequestId>1ZXWN5V7S7N41VHX</RequestId>
  <HostId>3lXR0Y18JE21ljldrxdoXZrZ1PCR6kC5MAEp9G+RcVms+fA1kRpn0S8aVJ9mFP/6ydlvCXHlFs0=</HostId>
</Error>
profile picture
asked 7 months ago532 views
2 Answers
1

Hi Giuseppe. I'm not sure what's wrong. Maybe I misunderstand your goals and intentions, but I think you should consider an alternate approach.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html

the object owner may share objects with others by creating a presigned URL

I don't think the IoT device should have the rights to create a presigned URL to download the object. Why not instead grant the device the ability to download the object directly? That's how AWS IoT Greengrass works, incidentally. It uses it's X.509 certificate to get temporary credentials from the AWS IoT Core credentials provider. And the corresponding IAM role grants the device permissions to get certain S3 objects.

There are two major approaches:

  1. The device uses credential provider to exchange it's X.509 certificate for IAM credentials that let it access the S3 object. That's what the HTTP download demo in the C-SDK does.
  2. The device obtains a presigned URL (usually from an MQTT subscription). That's what the HTTP upload demo in the C-SDK does.

You seem to be blending the two approaches.

profile pictureAWS
EXPERT
Greg_B
answered 7 months ago
  • Hi Greg, indeed if we could just download the object from the S3 I would be done, but I have a requirement to be able to pass a presigned URL with temporary validity to another process, this is out of my hands and I need to find a solution. I found good documentation on presigning the URL https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html which I followed and seems working (compared with CLI aws s3 presign) but it looks like in addition to signing the URL there is something else that I'm missing. I may have to go looking in detail into the source code of the CLI

0
Accepted Answer

I was able with help to fix the bug as discussed on https://github.com/aws/aws-iot-device-sdk-embedded-C/issues/1887 and I will soon make a proper demo for aws-iot-device-sdk-embedded-C

profile picture
answered 7 months ago
profile picture
EXPERT
reviewed a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions