Uploading executable file to S3 fails checksum validation

0

Hello, I'm working on a multipart upload from Angular using pre-signed URLs. I've tested the following cases:

  1. Upload a text file or an image - checksum validation works.
  2. Upload an executable (windows or linux) - checksum validation fails.
  3. Upload executable file with checksum disabled, file upload works fine.
  4. Download the file I uploaded in step 3 and check the checksum of the downloaded file vs the checksum of my original copy, it's the same.

Any idea about what could cause this behavior?

Thank you.

  • Could you check if the MD5 hash generated using the Angular application is the same as the one generated in your computer? (see this https://aws.amazon.com/premiumsupport/knowledge-center/data-integrity-s3/).

    If they are the same, please, share the code you are using to upload the file.

    S3 doesn't do any distinction between file types.

  • I double checked the MD5 using md5sum and it's correct. If it wasn't I would also expect all the files to fail. What happens is that only some executable files fail to upload.

    In the end we gave up on this and we're just using a lambda to calculate the checksum for the entire file after upload. We need this for other reasons anyway.

preguntada hace 2 años630 visualizaciones
1 Respuesta
0

you dont say which API call/step is giving the error, nor how you generated the checksum to compare, nor which S3 attribute you are checking, nor exactly what you are signing with the "pre-signed URL". When doing multipart -- OR when the size of the upload is large enough, the "E-TAG" of the assembled file on S3 may not be the same as a simple MD5 of the original file. You need to do checksums of each piece uploaded in order to calculate the ETAG of the entire object in S3. It may not be possible to use a presigned URL for MultiPart uploads -- each PART is distinct and requires its own signing.

DALDEI
respondido hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas