CloudFront + API Gateway AWS_IAM Authorization

I have an endpoint in API Gateway that has AWS_IAM Authorization enabled. If the endpoint is called directly everything works OK, provided that the request is signed correctly.

I am trying to put a CloudFront distribution in front of it, I have configured origin and cache behavior; the Authorization header is allowlisted and forwarded to API Gateway.

When I try to call the endpoint using the CloudFront url I get a signature error like this:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method.

I suspect that the issue is related to the fact that the client is signing the request for the CloudFront URL but when the signature is checked on the API Gateway/IAM side the url doesn't match, is that so?

What am I missing? I know I could reach for a Lambda@Edge to essentially re-sign the request but since I followed these instructions it should be possible without right?