By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Every second request to CloudFront fails with 504, every other request works.

0

I have a weird situation where every second request fails with 504, and every other request succeeds (the expected response for https://www.voxviva.app/ is to get a 307 that points to https://www.voxviva.app/en )

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 307 
content-type: text/html
date: Thu, 14 Sep 2023 10:46:49 GMT
location: /en
strict-transport-security: max-age=15724800; includeSubDomains
x-cache: Miss from cloudfront
via: 1.1 f6d693d2da795055838b1c110137b89c.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: HibGevOKUNtICW0AlFTX_ypxUm-_t-gfUJQ0cPbVqws_50Ux9-hnkA==

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 504 
content-type: text/html
content-length: 1033
server: CloudFront
date: Thu, 14 Sep 2023 10:47:24 GMT
x-cache: Error from cloudfront
via: 1.1 f0f40cec74c0c7e87f64d4eee853867a.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: DyhaxIAhEwepIWRNMpkwxuyD60BrqwW3lLBYspaCQCIT25884sTnFg==

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 307 
content-type: text/html
date: Thu, 14 Sep 2023 10:49:17 GMT
location: /en
strict-transport-security: max-age=15724800; includeSubDomains
x-cache: Miss from cloudfront
via: 1.1 457123b46297b2878de5b87c0fb16090.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: lVrIvOoyggScR20IvL8oL5jK5DBLFsiZ__HcDejeDurM_Mgfm4y8bg==

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 504 
content-type: text/html
content-length: 1033
server: CloudFront
date: Thu, 14 Sep 2023 10:49:52 GMT
x-cache: Error from cloudfront
via: 1.1 87edbe6dffa6ef8010cc5cc902bcafa0.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: iiLiUBbo1sUfsXJg6NMPCMjeCcnhXpBsBa_wAW25I_OeveVi4BkV9g==

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 307 
content-type: text/html
date: Thu, 14 Sep 2023 10:50:18 GMT
location: /en
strict-transport-security: max-age=15724800; includeSubDomains
x-cache: Miss from cloudfront
via: 1.1 136fd1617e777ad27200226c42b35398.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: Q3wcsmS61YliZP5lvexVa44FfzTa62ThG6Anzsb6uXa1-RV-xsUe4g==

dolf@astro:~$ curl -I https://www.voxviva.app/
HTTP/2 504 
content-type: text/html
content-length: 1033
server: CloudFront
date: Thu, 14 Sep 2023 10:50:51 GMT
x-cache: Error from cloudfront
via: 1.1 f0f40cec74c0c7e87f64d4eee853867a.cloudfront.net (CloudFront)
x-amz-cf-pop: JNB50-C1
alt-svc: h3=":443"; ma=86400
x-amz-cf-id: PN08JbiWNnC2dcUZzA197oN09MxHbe639Pp-_tuaLWHtq5ah2x6IDg==

I have fiddled with settings and read documentation, and could find nothing that fixes this.

I have also tried the advice at https://repost.aws/knowledge-center/cloudfront-troubleshoot-504-errors to test the speed of my origin (which is public.voxviva.app ), and the "Total time" printed is always under 1 second:

curl -w "DNS Lookup Time: %{time_namelookup} \nConnect time: %{time_connect} \nTLS Setup: %{time_appconnect} \nRedirect Time: %{time_redirect} \nTime to first byte: %{time_starttransfer} \nTotal time: %{time_total} \n" -o /dev/null https://public.voxviva.app

Any advice would be greatly appreciated.

  • Dmytro Sirant EXPERT
    8 months ago

    I can't reproduce the problem. Tried over 20 times and each time I got 307 code. I see that there is only one IP address returned, so it's not a case.

  • Rudolf Byker
    8 months ago

    I ran curl -I https://www.voxviva.app/ from a few more places:

    • From our office network in South Africa, I get 307 on every second request, and 504 on every other request.
    • From my home network in South Africa, I got the same as at the office.
    • From a (non-AWS) VM in Germany, and I get 307 every time.
    • From a AWS EC2 VM in es-east-1, and get 307 every time.
  • Rudolf Byker
    8 months ago

    I asked a few other people in South Africa to test using the following script:

    #!/usr/bin/env sh

while true
do
    TIME=$(date +%Y-%m-%dT%H:%M:%S)
    CODE=$(curl -sI https://www.voxviva.app | head -n 1 | cut -d ' ' -f 2)
    echo "$TIME","$CODE"
done

    This was run on several different ISPs, operating systems, hardware, etc. Only one person got consistent 307s. The rest all either see the responses toggle between 504 and 307 like I do, or they see multiple 504s followed by multiple 307s, repeatedly.

  • Rudolf Byker
    7 months ago

    The 504 problem seems to have disappeared. I asked everyone who tested previously to test again, and everyone gets consistent 307s now. We did not change anything in our own infrastructure or in our AWS configuration. So maybe this was just a temporary glitch at one of the edge locations in South Africa.

Topics
Networking & Content Delivery
Tags
Amazon CloudFront
Language
English
profile picture
Rudolf Byker
asked 8 months ago179 views
1 Answer
0

Hi,

Assuming that your origin is an S3 bucket, can you check that:

  • NO custom headers are defined in the origin
  • s3:GetObjectVersion is granted to Principal:*

Some issues like yours have been reported when the above was not satisfied. So, you may face (or not...) similar problem

Best,

Didier

profile pictureAWS
EXPERT
Didier_Durand
answered 8 months ago
  • Rudolf Byker
    8 months ago

    Hi, thanks for your time. I mentioned specifically that my origin is NOT an S3 bucket. Also, I have configured it to not send any headers to the origin (except those that CloudFront automatically adds, like Host). Everything works, except this one URL which returns 307. The URLs that return 200 never have this issue. Also, I never get timeouts when accessing the origin directly.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content