mTLS in API Gateway fails some certificates with 403 and x-amzn-ErrorType=ForbiddenException

I have been trying to help one of our customers to get service working with mTLS, but I keep hitting a brick wall trying to understand what is going on. Customer might not be able to replace the cert they are using due to shared usage with other vendors.

What happens : I have tested the API with my own self-signed key successfully. When customer calls API, a 403 {"message":"Request Refused"} with header x-amzn-ErrorType: ForbiddenException is given. Customers cert seems to be identical to my own except for some X509v3 extensions and reads fine with openssl. There is furthermore no complaints from API Gateway on the trust store like I have seen on broken certs before.

What is tested: First I created a certificate as identical as possible to the customer cert, and it failed the same way for me. Checked Prerequisites for mutual TLS and everything checked out although not much detail is given for : "- The certificate must meet X.509 syntax requirements." Since I knew that my first test certificate worked as expected I know that service is configured correctly.

Secondly I created a lot of different certificates with different combinations of X509v3 extensions and tested them. Only the ones not having keyUsage was accepted by mTLS.

openssl req -x509 -newkey rsa:4096 -keyout key3.pem -out cert3.pem -sha256 -days 180 -nodes -subj "/C=XX/O=XXXX/OU=XXXXX/CN=MTLSTest3" -addext "basicConstraints=CA:false" -addext "extendedKeyUsage=serverAuth,clientAuth" -addext "keyUsage=critical,digitalSignature" NOT WORKING: X509v3 extensions: X509v3 Key Usage: critical Digital Signature X509v3 Basic Constraints: CA:FALSE X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication

openssl req -x509 -newkey rsa:4096 -keyout key11.pem -out cert11.pem -sha256 -days 180 -nodes -subj "/C=XX/O=XXXX/OU=XXXXX/CN=MTLSTest10" -addext "basicConstraints=CA:false" -addext "extendedKeyUsage=serverAuth,clientAuth" WORKING: X509v3 extensions: X509v3 Basic Constraints: CA:FALSE X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication

For now there is no Lambda authoriser, just the S3 trusted store containing my own self-sign test certificates and the customers self-signed certs.

Questions:

• Why do API Gateway reject certificates containing keyUsage ? As far as I can understand it is a valid X509v3 syntax.
• Is there anywhere I can find a more comprehensive requirement list for certificates to be accepted ?
• Is there a better way to troubleshoot mTLS ? I can not ask the customer to send private keys, and the execution logs do not say anything outside what is already known from response-header and -body.