NoSuchUpload error on uploading a part file of multi-part upload via presigned url

0

I am getting NoSuchUpload error when uploading a part of multi-part upload file via PUT request to presigned url. The error message says, "The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed." The upload id is not invalid as I have checked the same via list-multipart-uploads command on AWS CLI. And neither I have aborted or completed the upload.

I am getting the following error on multi part upload

<?xml version="1.0"?>

<Error>
<Code>NoSuchUpload</Code>
<Message>The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.</Message>
<UploadId>1kPnI95Sy9xim3DzudwQ4Yno1wIrKT.Lv.wzZ6wqXTM792QfKYZZLavSWOrQxCAgc9mj3E09Nos2xJu_YvaRzAIjD4sx6hO1pOoBNWvzfoFf_Tabbt9d62ebjrKgHHfN</UploadId>
<RequestId>BNK1E884Y30TM5MF</RequestId>
<HostId>Iz2brROW9q4ym9UnxLZwoBZp+Af8KkXmFfTm2C86tRHIW1r5w/LWAKU0wSg2bQS4c5K0Xo/yL1A=</HostId>
</Error>

I am trying to upload a file to s3 bucket via multi-part upload method. I am using boto3 python SDK to do the same. I generated the upload_id for a 20MB file with key <uuid4>/files/test-user-data/<uuid4>_0001.mp4 using create_multipart_upload method. Then I generated presigned-url for each 5MB chunk of the file as follows:

params = {'Bucket': <bucket_name>, 'Key': <key>, 'UploadId': <upload_id>, 'PartNumber': <chunk_id>}

s3_client.generate_presigned_url(ClientMethod='upload_part', Params=params, ExpiresIn=3600)

I got the following presigned url:

https://<bucket_name>.s3.amazonaws.com/3be4b390-f01c-4cfb-bac0-ecf1534a335a/files/3be4b390-f01c-4cfb-bac0-ecf1534a335a/files/test-user-data/725f5643-6dc8-4d48-ad7b-d73479aa5752_25bceabd-343e-4d9c-82ab-0577dc551a69_0001.mp4?uploadId=1kPnI95Sy9xim3DzudwQ4Yno1wIrKT.Lv.wzZ6wqXTM792QfKYZZLavSWOrQxCAgc9mj3E09Nos2xJu_YvaRzAIjD4sx6hO1pOoBNWvzfoFf_Tabbt9d62ebjrKgHHfN&partNumber=1&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATTAUJC2AI5OKS5GA%2F20221119%2Fap-south-1%2Fs3%2Faws4_request&X-Amz-Date=20221119T041617Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=aa26c2263d36fd3e47e05077fa32c4631cefc42c0b7c009341f9b52804cbc97e

Then to the presigned url, I sent a put request as follows:

s3_response = requests.put(url=<presigned_url>, files={'file': <chunk>})

Here chunk is a bytes object. I expected the file to upload successfully. Initially I doubted that I may be sending the incorrect upload_id as per the error message. But I eliminated that probability after writing an automated test case.

1 Answer
0

Hello,

Instead of generating the presigned-url for each 5MB chunk of the file, Generate a presigned URL to initiate the multiart upload. Follow the steps as below:

  1. Create a new file, and split it into parts.
  2. Generate a presigned URL to initiate the multiart upload.
  3. Use the presigned URL to initiate the upload.
  4. Generate a presigned URL for each part, using a part number.
  5. Use the URLs to send the PutPart requests. Keep track of the Etag that is returned for the part number.
  6. Combine all of the parts and corresponding ETAGs to form the request body.
  7. Generate a presigned URL to complete the MP upload.
  8. Complete the multipart upload by sending the request with the presigned complete multipart upload URL.
AWS
SUPPORT ENGINEER
answered a year ago
  • I'm running into this same issue. Your answer seems somewhat contradictory, as you say not to generate a presigned url for each chunk, yet in step 4 in your instructions is to generate a presigned URL for each part using a part number.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions