CloudFront is dropping Content-Length (might relate to Content-Encoding)

1

while s3 url - https://test-content-length.s3.amazonaws.com/sample.pdf has Content-Length response header its CloudFront url - https://d3je9z6q60prpt.cloudfront.net/sample.pdf doesn't have Content-Length

  • why is it?
  • is there a way to set CloudFront distribution to return Content-Length?
  • if so, please provide details
  • if not, please explain how we can solve it

Thanks, Michal

additional edit after the first response - curl request should imitate the way a browser request this url with Accept-Encoding: gzip, deflate, br

curl https://d3je9z6q60prpt.cloudfront.net/sample.pdf -I -H "Accept-Encoding: gzip, deflate, br"
Michal
asked 10 months ago1318 views
3 Answers
0

Both responses include Content-Length, do you and I have different ways of checking?

S3

curl https://test-content-length.s3.amazonaws.com/sample.pdf -I
HTTP/1.1 200 OK
x-amz-id-2: R8XPq5+/2riIlM+GLUJVrWepJ5yYL8xWZSQrqTLZ70gnIHiqxqllgB9LfX6kmAlNtDz2chSH7Mc=
x-amz-request-id: 479BMV3A70J0F17B
Date: Mon, 03 Jul 2023 13:28:57 GMT
Last-Modified: Mon, 03 Jul 2023 12:53:14 GMT
ETag: "4b41a3475132bd861b30a878e30aa56a"
x-amz-server-side-encryption: AES256
Accept-Ranges: bytes
Content-Type: application/pdf
Server: AmazonS3
Content-Length: 3028

CloudFront

curl https://d3je9z6q60prpt.cloudfront.net/sample.pdf -I
HTTP/2 200
content-type: application/pdf
content-length: 3028
date: Mon, 03 Jul 2023 13:29:20 GMT
last-modified: Mon, 03 Jul 2023 12:53:14 GMT
etag: "4b41a3475132bd861b30a878e30aa56a"
x-amz-server-side-encryption: AES256
accept-ranges: bytes
server: AmazonS3
vary: Accept-Encoding
x-cache: Miss from cloudfront
via: 1.1 9bf92639763e2dc7c1ea8eb4814a239e.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT20-C1
x-amz-cf-id: FNExZ36GtBm1afRWJJUY0Fax8q0Tda6JTJW5kvRQTzeDD3UQsxVlwg==
profile picture
EXPERT
answered 10 months ago
  • as I mentioned, it relate to gzip when u hit the CloudFront url from browser there is Accept-Encoding: gzip, deflate, br and then you don't have Content-Length can you add the Accept-Encoding to your curl? curl https://d3je9z6q60prpt.cloudfront.net/sample.pdf -I -H "Accept-Encoding: gzip, deflate, br"

  • Thank you for sharing how to check. We have confirmed that content-length is not included in the response. The reason Content-Length is lost in CloudFront is because the content is compressed. Content-Length represents the length of the origin content before compression, not the length after compression. If a client is requesting to receive compressed content, including a Content-Length header will cause the actual size of the content to not match the uncompressed one. If Content-Length is required, it may be possible by way of disabling compression. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html

  • Hi Riku, Thanks for your explanation, but why CloudFront works different than s3 or any other WebServer / CDN service? When trying the same from s3 - Content-Length is returned.. curl https://test-content-length.s3.amazonaws.com/sample.pdf -I -H "Accept-Encoding: gzip, deflate, br"

  • The specifications behind CloudFront are not publicly available, so the exact information is not known. My guess is that even if you set "Accept-Encoding: gzip, deflate, br" when accessing S3, the Content-Length in the response will return the same value. This leads me to believe that the specification returns the pre-compressed Content-Length only when accessing the origin directly. In other words, it is easy to understand if you think in terms of where the origin content is received. When accessing CloudFront as shown in the figure below, it is CloudFront that receives the origin content. So I expect that Content-Length is there up to the point where it reaches CloudFront. origin

  • it sounds like a bug with CloudFront when CloudFront handles the gzip, it set only Content-Encoding: gzip header and skip the Content-Length

    this is not best practice of on-the-fly gzip compression when comparing it to IIS

    who can we reach out to report on this? it's blocking our clients from getting adequate response from CloudFront and we are considering moving those clients to use Akamai CDN so they would be unblocked by this missing functionality

    Thanks, Michal

0

Updates: I contacted AWS and according to them this is the current behavior

using compression instead of using Content-Length header, we add the "Transfer-Encoding" header with the value as "Chunked"

Because compression is happening on the fly, Cloudfront is unable to get the final length of the compressed content. Adding a Content-Length header would require storing the full response and calculating its length and then returning it back to the client which is not possible since it would cause latency if the original response itself was large. That is why when using compression instead of using Content-Length header, we add the "Transfer-Encoding" header with the value as "Chunked". This indicates that the data is sent in a series of chunks. If you are using HTTP/1.1, the Content-Length header is omitted and the "Transfer-Encoding: Chunked" is added instead, in the response headers. However, if you are using HTTP/2 for compressed contented, neither "Content-Length" nor "Transfer-Encoding" is used. HTTP/2 uses its own mechanism to convey to the client that the response is chunked.

I suggested them to provide settings or either not compressing file types or defines file types that requires Content-Length - they opened internal ticket

Thank you for your suggestions. I have gone ahead and made a feature request to our internal Cloudfront team and whilst I do not have an ETA to share, updates are often posted in the Cloudfront Console, Blog and AWS Announcement pages. You can track new features and updates for AWS services at https://aws.amazon.com/new/

Michal
answered 10 months ago
0

Content-Encoding is also missing and it's causing some problems with audit tools that review the webs and detect the content as not compressed.

answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions