By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

File downloaded from s3 using cloudfront is corrupted

4

Hello. I have an app that download files from s3 using cloudfront. I use plain HTTP request with node.js(electron), got npm package. In most cases it works well. But some users have errors with the downloaded file because its corrupted. No download errors (success code is received, file is downloaded) This problem can come and go even for 1 user. so today its corrupted but tomorrrow its ok. So my question is - how can I know what is the problem? And second question: is there any way to 100% download full working file?

Topics
StorageNetworking & Content Delivery
Tags
StorageAmazon CloudFront
Language
English
rePost-User-2739165
asked 10 months ago268 views
4 Answers
2

The same issue on edges:

  • ARN56-P1
  • ARN54-C1
Egor
answered 10 months ago
  • Egor
    10 months ago

    examples of currupted input:

    | hexdump -c
100  0000000   '   u   s   e       s   t   r   i   c   t   '  \0  \0  \0   $ 0
10000010  \0  \0  \0  \0  \0  \0  \0 016   i   p   -   3   0   -   3   0
40000020   -   4   6   -   6   3  \0  \0  \0  \0  \0  \0   3   c   6   9
00000030   b   0   7   2   2   4   8   0   7   a   6   2   9   e   c   5
50000040   e   4   6   1   8   9   9   a   7   b   '   ;  \n   v   a   r
 0000050       h   o   s   t   n   a   m   e       =       '   '   ;  \n

    as you see- \0 character is currupted character

    legal content:

    'use strict';

var codeVersion = '32a86883c83c69b07224807a629ec5e461899a7b';
var hostname = '';
0

Just throwing it out there as an idea, could this be related to cloudfront compression? Gzip or BR?

Sometimes cloudfront will compress dynamic objects some times it will not.

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html

profile picture
EXPERT
Gary Mclean
answered 10 months ago
  • Gary Mclean EXPERT
    10 months ago

    Agree I see that too.

  • Gary Mclean EXPERT
    10 months ago

    \0 is ASCI NUL in hex

  • Gary Mclean EXPERT
    10 months ago

    The ASCII NUL character is used to denote the end of the string in C or C++

  • Egor
    10 months ago

    as you see example above - there is 3 part in content: part of correct content, part of corrupted content and finally correct content. Looks like internal error of some Cloudfront's edges.

  • rePost-User-2739165
    10 months ago

    thank you for comment. Do you know can I disable it?

-2

Hi, I don't know why you're receiving corrupted files intermittently but does the problem exist if the user downloads a file, finds the file is correct and then downloads the file again straight away, i.e. does the problem exist consistently for a period of time?

A workaround that can be good practice anyway is to upload an object containing a hash of the file alongside the actual file object. Download both the file object and hash object in your client and then compare the original hash to a hash of the object you downloaded. If they differ then try to download the file again.

Shane Anderson
answered 10 months ago
-3

Few ideas-

  1. You can use HTTPS if you might think someone in the middle is meddling with the traffic (local daemons on the client, proxies, ISP, etc.)
  2. Is there a chance you are using custom error page which masks an error?
  3. You can use CloudFront logs to make sure the size of the response makes sense, and that the client didn't disconnected before the entire file downloaded.
AWS
Nati_C
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content