Uploading executable file to S3 fails checksum validation

0

Hello, I'm working on a multipart upload from Angular using pre-signed URLs. I've tested the following cases:

  1. Upload a text file or an image - checksum validation works.
  2. Upload an executable (windows or linux) - checksum validation fails.
  3. Upload executable file with checksum disabled, file upload works fine.
  4. Download the file I uploaded in step 3 and check the checksum of the downloaded file vs the checksum of my original copy, it's the same.

Any idea about what could cause this behavior?

Thank you.

  • Could you check if the MD5 hash generated using the Angular application is the same as the one generated in your computer? (see this https://aws.amazon.com/premiumsupport/knowledge-center/data-integrity-s3/).

    If they are the same, please, share the code you are using to upload the file.

    S3 doesn't do any distinction between file types.

  • I double checked the MD5 using md5sum and it's correct. If it wasn't I would also expect all the files to fail. What happens is that only some executable files fail to upload.

    In the end we gave up on this and we're just using a lambda to calculate the checksum for the entire file after upload. We need this for other reasons anyway.

asked 2 years ago607 views
1 Answer
0

you dont say which API call/step is giving the error, nor how you generated the checksum to compare, nor which S3 attribute you are checking, nor exactly what you are signing with the "pre-signed URL". When doing multipart -- OR when the size of the upload is large enough, the "E-TAG" of the assembled file on S3 may not be the same as a simple MD5 of the original file. You need to do checksums of each piece uploaded in order to calculate the ETAG of the entire object in S3. It may not be possible to use a presigned URL for MultiPart uploads -- each PART is distinct and requires its own signing.

DALDEI
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions