CloudFront make website WordPress redirect to page alphabet after time duration

Hello everyone,

We are facing a challenging issue despite our efforts to search for official documentation and Stack Overflow discussions related to CloudFront.

We have two WordPress sites that utilize CloudFront and cache policy settings, enabling compression support for both gzip and Brotli, and setting cache control to expire after one day. Underneath CloudFront, we have an ALB and a target group that connects to an EC2 Apache server. The Apache web server does not use any compression method and does not enable accept-encoding. This means that CloudFront handles compression as well as accept-encoding. I have reached this conclusion based on my testing using the curl command under the EC2 instance's localhost:

curl -I -H 'Accept-Encoding: gzip,deflate' -H 'HOST: mydomain.com' localhost

The response does not include the content-encoding header:

HTTP/1.1 200 OK
Date: Fri, 07 Jul 2023 03:24:34 GMT
Server: Apache/2.4.37 () PHP/7.4.30
Upgrade: h2,h2c
Connection: Upgrade
X-Powered-By: PHP/7.4.30
Link: https://mydomain.com/wp-json/; rel="https://api.w.org/"
Link: https://mydomain.com/wp-json/wp/v2/pages/2077; rel="alternate"; type="application/json"
Link: https://mydomain.com/; rel=shortlink
Content-Type: text/html; charset=UTF-8

And here when request through CloudFront:

curl -I --location -sH 'Accept-encoding: gzip' 'https://mydomain.com/'

HTTP/2 200 
content-type: text/html; charset=UTF-8
date: Thu, 06 Jul 2023 03:45:13 GMT
server: Apache/2.4.37 () PHP/7.4.30
apigw-requestid: Hn3PggQgjoEEJxQ=
x-powered-by: PHP/7.4.30
link: <https://mydomain.com/wp-json/>; rel="https://api.w.org/"
link: <https://mydomain.com/wp-json/wp/v2/pages/2077>; rel="alternate"; type="application/json"
link: <https://mydomain.com/>; rel=shortlink
content-encoding: gzip
vary: Accept-Encoding
x-cache: Hit from cloudfront
via: 1.1 3581c46f65a0770e1977ff05d5b7ad80.cloudfront.net (CloudFront)
x-amz-cf-pop: DUB56-P1
x-amz-cf-id: _jujzCQv6PFQl_n6EvWCnPV5Kut3kg5i0W41J1JSAkZbtyjDpwl0EQ==
age: 85804

The issue I'm encountering here is that after a certain period (around 3 weeks), both of these websites encounter an error. The observed error is being redirected to a page sorted alphabetically. It seems there is a problem with the cache, as it fails to clear the cache after one day, and instead of fetching the content from the backend, it returns a pre-cached page sorted alphabetically.

I have researched some Stack Overflow posts related to requesting support for gzip or Brotli, such as the following:

https://stackoverflow.com/questions/68054148/why-does-cloudfront-sometimes-serve-gzip-instead-of-br-when-both-are-enabled

However, I am curious about why CloudFront behaves in this way, which I find difficult to understand. Clearing the cache indeed resolves the issue, but it cannot be done until a new issue is discovered.

I suppose the issue here could be that only gzip compression is enabled while Brotli compression is disabled. Gzip is a more commonly used compression method and supported by many browsers. However, Brotli provides better compression and can result in smaller file sizes, improving page load speeds. Using both methods in parallel can optimize compression performance and be compatible with various browsers. Can someone explain why this happens? Thank you.