API Gateway cares about my Authorization header when it shouldn't

Question

My setup is:  
* Private API Gateway REST API with a GET method that invokes a Lambda  
* A VPC from which the API will be invoked  
* VPC endpoint for API Gateway (execute-api). The endpoint (as well as the rest of the VPC infrastructure) is created via CloudFormation.  
  
There is no Authorization set on the method. It's just a simple test API as a proof of concept. However it will need to accept a header called "Authorization", which is something specific to our company. I expect this header to be passed through to the Lambda as part of the integration.  
  
The problem is that when I supply an Authorization header, API Gateway rejects the request with the following message (taken from API Gateway CloudWatch logs):  
  
```  
IncompleteSignatureException Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=\[the header content here]  
```  
  
If I remove the Authorization header, the request goes through to the Lambda and I get the expected response.  
  
Now the interesting part is this: if I delete the execute-api VPC endpoint and create it manually via the console, everything works fine - my Authorization header is passed through to the Lambda without problems. But if I create the endpoint via CloudFormation, API Gateway suddenly cares about that header's content. But the endpoint itself appears to be the same when created manually or via CF: dns is enabled, same 3 subnets, same security group, same resource policy (allow all access for everyone to everything). I can see no difference whatsoever. But evidently something somewhere is different, but I have no idea what. I've recreated it manually and via CF several times and the result is always as described above.  
  
What I have found, not sure how relevant it is, is that the amazon headers logged by API Gateway are slightly different for the manual vs CF endpoints (this is with the Authorization header removed, as the CF one rejects before logging headers). With the CF endpoint, the "Method request headers" logged include "x-amzn-vpce-config=0" and "x-amzn-vpce-policy-url=MQ==". With the manual endpoint, I get "x-amzn-vpce-config=1" and the policy-url header isn't included. Googling these headers gives me absolutely nothing.  
  
Also possibly worth noting, I've tried both specifying the endpoint in the API (in the console, API settings, you can select the endpoint for a private API) and leaving it blank. Redeployed API gateway after. Neither change has any effect on the behaviour described above.  
  
Does anyone know why it's behaving like this?

Answer

I managed to get it working. This is an AWS bug.  
  
This is a snippet of the CF to setup this endpoint:

```
ApiGatewayVPCEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    PolicyDocument:
      Statement:
        - Action: '*'
          Effect: Allow
          Resource: '*'
          Principal: '*'
```

This is that policy shown in the console:

```
{
    "Statement": [
        {
            "Action": "*",
            "Effect": "Allow",
            "Resource": "*",
            "Principal": "*"
        }
    ]
}
```

This is the policy returned when calling aws ec2 describe-vpc-endpoints

```
"PolicyDocument": "{"Statement":[{"Action":"*","Resource":"*","Effect":"Allow","Principal":"*"}]}"
```

Now if I create the endpoint manually, this is the policy in the console:

```
{
    "Statement": [
        {
            "Action": "*",
            "Effect": "Allow",
            "Resource": "*",
            "Principal": "*"
        }
    ]
}
```

And this is the policy returned when calling aws ec2 describe-vpc-endpoints

```
"PolicyDocument": "{
  "Statement": [
    {
      "Action": "*", 
      "Effect": "Allow", 
      "Principal": "*", 
      "Resource": "*"
    }
  ]
}"
```

Basically the same except for some whitespace and newlines you'd get when prettifying the JSON, but of course it's irrelevant to the actual JSON data. Or so you'd think. Actually that is the reason why it is failing in the YAML version. When AWS transforms the YAML to JSON, it doesn't put newlines in there. And somehow that causes the policy to not work. If I deploy this:

```
ApiGatewayVPCEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    PolicyDocument: '
    {
      "Statement": [
        {
          "Action": "*",
          "Effect": "Allow",
          "Resource": "*",
          "Principal": "*"
        }
      ]
    }'
```

Then it works perfectly. And even if I supply the JSON on a single line, with no whitespace, it still works fine (describe-vpc-endpoints shows newlines and whitespace added).  
  
Obviously (I'd like to say) the issue here isn't whether irrelevant newlines exist in the JSON or not, but rather how that JSON is interpreted. It seems like there is string parsing going on here rather than converting the JSON to an actual object (or whatever method you might normally use to access some path in a JSON string), which is getting tripped up by lack of newline characters. Anyway, just a guess, but what isn't a guess is that this is a bug.  
  
Edited by: mspo2 on Oct 18, 2021 7:27 AM

API Gateway cares about my Authorization header when it shouldn't

Relevant questions

Return a custom header from lambda authorizer in API-gateway (HTTP api)

Is it possible to send json data in body of GET method defined in API Gateway REST API ?

Calling Private API from Lambda in VPC

Deploy Lambda Function and API Gateway REST

How to get traffic from a public API Gateway to a private one?

When to invoke a lambda directly or via API Gateway

API Gateway cares about my Authorization header when it shouldn't

Build a REST API with API Gateway private integration: Tutorial

Create API GW Websocket API that is only accessible from within a VPC.

{"message":"forbidden"} when accessing private API Gateway