Generation of presigned URL to S3 with temporary credentials - InvalidAccessKeyId

0

I have been working on adding support for presigned URLs in aws-iot-device-sdk-embedded-C and I believe I'm very close.

I started from the demo to download from S3 with a temporary credentials user profile, and I adjusted my signature comparing with the CLI (aws s3 presign) with same data.

The signature is now OK but still while the temporary AccessKeyId is able to download the file, when I use the (printed to terminal) presigned URL I get instead <Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message>

I'm working on https://github.com/giuspen/aws-iot-device-sdk-embedded-C/blob/GP_http_demo_s3_download_test_signature2/demos/http/http_demo_s3_download/http_demo_s3_download.c

I'm puzzled, is there an additional procedure to record the AccessKeyId in order to generate the presigned URL?

EXAMPLE LOG:

[INFO] [DEMO] [http_demo_s3_download.c:1752] HTTP Client Synchronous S3 download demo using temporary credentials fetched from iot credential provider:
c10nfdpw0ux517.credentials.iot.eu-central-1.amazonaws.com
[INFO] [DEMO] [http_demo_s3_download.c:910] Establishing a TLS session with c10nfdpw0ux517.credentials.iot.eu-central-1.amazonaws.com:443.
[INFO] [DEMO] [http_demo_s3_download.c:766] AWS IoT credentials will expire after this timestamp: 2023-10-02T13:30:09Z.
[INFO] [DEMO] [http_demo_s3_download.c:650] AWS IoT credential provider response: HTTP/1.1 200 OK
content-type: application/json
content-length: 1165
date: Mon, 02 Oct 2023 12:30:09 GMT
x-amzn-RequestId: fd0bb5c9-754f-9486-6cfa-2529899e17c8
{"credentials":{"accessKeyId":"ASIAVBKNXEL5HP2B4VXX","secretAccessKey":"****************************************","sessionToken":"IQoJb3JpZ2luX2VjEA0aDGV1LWNlbnRyYWwtMSJHMEUCIFsKZ0d7JLu+P1S0AfpaXUGuW5/agg0/3+joQt+D2oirAiEA+fCfmAqo9zt8Xb5CcuJj7He77UQQcgq/HMaZNJljQKcq3gMIFhAAGgwzNDY0NDQ3MzUyMjYiDCk8oHzTPPEymfx9Kiq7AwvmDkvcH5RPb9RWtAoci/WO/UhI4D1CQ5kg2X7KEUbpdk/ig8W6hSMhEcFWYEfMNpryhicDB/XoJzpjfvPxccOewA6Rd/y+2qFzNOOc2Cba3GtovdGctYJBtvex6du367kJoYks6PtaKpxFXyA6oEHWQ2X0yy9Ai+bZdPX9woHEJOQ7x9Orr6msJo47wSFgF9R5JlPk2n1mvLuI1kQlzPbgakTSwkSYb0hR91Sbd9V9kWQzQsK1GI3TktJ9YkPM0EIP5FMlLtwBPa0IYNMpHT1mz5LnXg/Imoepb6sZtsfW/M1LhkbgL//8sTeE5HHUXIL0RbRRSEQSctFPETIozfmKDp+yGNeOgFXDKwuY7aeKTzw1URnKHSM9mN5d5BVCdIjFu4obUgOLrL1tSVK3ai+54d1pXplNkcT1RDuZUKPvdj1eNDb46Ykd70LbtuQCHZNJQLeIxOQ/XbH3slf49UXp6Gwr8ENk888U+w8W3Emi1uZ6OkM4X00DKZaUksQCH1/my7tAwOP/RZNypXiQj6d33COeu2W92NvTt74xH8XrUEBOxxAQuDTasOTf8SK4E+y7Q9GoB0tb9CQiMNHv6qgGOpoB3LgWRNLccumU7fynfuXkdXoi3utGH7UpWqs8zz2mImgSJccDFg8NEyT9x+yhdcitTiJ7lugAFNgKg1qTpphL6i0CRDExX6JyYYwXU1l3BxSxLMSehYlLWpJV+/SCOVqur9ps+7dC3M3+hjuSG3jusANCGbbINzink/mQmXSHDnGL8zcbV6YpLPEH+DVXuz4cFW3FT2jbkjuV4g==","expiration":"2023-10-02T13:30:09Z"}}.
[INFO] [DEMO] [http_demo_s3_download.c:961] Establishing a TLS session with otahalo.s3.eu-central-1.amazonaws.com:443.
[INFO] [DEMO] [http_demo_s3_download.c:1478] Getting file object size from host...
[INFO] [DEMO] [http_demo_s3_download.c:1652] Received successful response from server (Status Code: 206).
[INFO] [DEMO] [http_demo_s3_download.c:1698] The file is 58 bytes long.
[INFO] [DEMO] [http_demo_s3_download.c:1176] Downloading bytes 0-57, out of 58 total bytes, from otahalo.s3.eu-central-1.amazonaws.com...:  
[INFO] [DEMO] [http_demo_s3_download.c:1206] Response Body:
{
"filename"="update-bundle-imx8mnevk-v01-00-01.raucb"
}
[INFO] [DEMO] [http_demo_s3_download.c:1307] Getting presigned URL...
[INFO] [DEMO] [http_demo_s3_download.c:1410] ota_temp_url=
https://otahalo.s3.eu-central-1.amazonaws.com/ota-v01-00-01.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAVBKNXEL5HP2B4VXX%2F20231002%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20231002T123009Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEA0aDGV1LWNlbnRyYWwtMSJHMEUCIFsKZ0d7JLu%2BP1S0AfpaXUGuW5%2Fagg0%2F3%2BjoQt%2BD2oirAiEA%2BfCfmAqo9zt8Xb5CcuJj7He77UQQcgq%2FHMaZNJljQKcq3gMIFhAAGgwzNDY0NDQ3MzUyMjYiDCk8oHzTPPEymfx9Kiq7AwvmDkvcH5RPb9RWtAoci%2FWO%2FUhI4D1CQ5kg2X7KEUbpdk%2Fig8W6hSMhEcFWYEfMNpryhicDB%2FXoJzpjfvPxccOewA6Rd%2Fy%2B2qFzNOOc2Cba3GtovdGctYJBtvex6du367kJoYks6PtaKpxFXyA6oEHWQ2X0yy9Ai%2BbZdPX9woHEJOQ7x9Orr6msJo47wSFgF9R5JlPk2n1mvLuI1kQlzPbgakTSwkSYb0hR91Sbd9V9kWQzQsK1GI3TktJ9YkPM0EIP5FMlLtwBPa0IYNMpHT1mz5LnXg%2FImoepb6sZtsfW%2FM1LhkbgL%2F%2F8sTeE5HHUXIL0RbRRSEQSctFPETIozfmKDp%2ByGNeOgFXDKwuY7aeKTzw1URnKHSM9mN5d5BVCdIjFu4obUgOLrL1tSVK3ai%2B54d1pXplNkcT1RDuZUKPvdj1eNDb46Ykd70LbtuQCHZNJQLeIxOQ%2FXbH3slf49UXp6Gwr8ENk888U%2Bw8W3Emi1uZ6OkM4X00DKZaUksQCH1%2Fmy7tAwOP%2FRZNypXiQj6d33COeu2W92NvTt74xH8XrUEBOxxAQuDTasOTf8SK4E%2By7Q9GoB0tb9CQiMNHv6qgGOpoB3LgWRNLccumU7fynfuXkdXoi3utGH7UpWqs8zz2mImgSJccDFg8NEyT9x%2ByhdcitTiJ7lugAFNgKg1qTpphL6i0CRDExX6JyYYwXU1l3BxSxLMSehYlLWpJV%2B%2FSCOVqur9ps%2B7dC3M3%2BhjuSG3jusANCGbbINzink%2FmQmXSHDnGL8zcbV6YpLPEH%2BDVXuz4cFW3FT2jbkjuV4g%3D%3D%22%2C%22expiration%22%3A%222023-10-02T13%3A30%3A09Z%22%7D%7D&X-Amz-Signature=fd138ce95b8bf62db8ba1a0cf1aea3147a48e4c95231b44e2c4a6a30aebd52b7
[INFO] [DEMO] [http_demo_s3_download.c:1856] Demo iteration 1 is successful.
[INFO] [DEMO] [http_demo_s3_download.c:1875] Demo completed successfully.

NOW If I use the presigned URL:

<Error>
  <Code>InvalidAccessKeyId</Code>
  <Message>The AWS Access Key Id you provided does not exist in our records.</Message>
  <AWSAccessKeyId>ASIAVBKNXEL5HP2B4VXX</AWSAccessKeyId>
  <RequestId>1ZXWN5V7S7N41VHX</RequestId>
  <HostId>3lXR0Y18JE21ljldrxdoXZrZ1PCR6kC5MAEp9G+RcVms+fA1kRpn0S8aVJ9mFP/6ydlvCXHlFs0=</HostId>
</Error>
profile picture
질문됨 7달 전558회 조회
2개 답변
1

Hi Giuseppe. I'm not sure what's wrong. Maybe I misunderstand your goals and intentions, but I think you should consider an alternate approach.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html

the object owner may share objects with others by creating a presigned URL

I don't think the IoT device should have the rights to create a presigned URL to download the object. Why not instead grant the device the ability to download the object directly? That's how AWS IoT Greengrass works, incidentally. It uses it's X.509 certificate to get temporary credentials from the AWS IoT Core credentials provider. And the corresponding IAM role grants the device permissions to get certain S3 objects.

There are two major approaches:

  1. The device uses credential provider to exchange it's X.509 certificate for IAM credentials that let it access the S3 object. That's what the HTTP download demo in the C-SDK does.
  2. The device obtains a presigned URL (usually from an MQTT subscription). That's what the HTTP upload demo in the C-SDK does.

You seem to be blending the two approaches.

profile pictureAWS
전문가
Greg_B
답변함 7달 전
  • Hi Greg, indeed if we could just download the object from the S3 I would be done, but I have a requirement to be able to pass a presigned URL with temporary validity to another process, this is out of my hands and I need to find a solution. I found good documentation on presigning the URL https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html which I followed and seems working (compared with CLI aws s3 presign) but it looks like in addition to signing the URL there is something else that I'm missing. I may have to go looking in detail into the source code of the CLI

0
수락된 답변

I was able with help to fix the bug as discussed on https://github.com/aws/aws-iot-device-sdk-embedded-C/issues/1887 and I will soon make a proper demo for aws-iot-device-sdk-embedded-C

profile picture
답변함 7달 전
profile picture
전문가
검토됨 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인