Cannot resolve host of RDS endpoint in private subnet via VPN client endpoint

I have an AWS VPC VPN client endpoint setup to connect to 2 private subnets. Inside these private subnets is a RDS instance and an EC2 instance running an application server (aka "control plane server"). The private subnets are provided access to the external internet (So servers can download packages and such) via a public subnet with a NAT -> internet gateway.

I have successfully connected to the VPN from my laptop and even SSH-ed into the "control plane server". However from my laptop, connected to the VPN, I cannot connect to the RDS endpoint. I get the error:

lookup <rds instance ID>.<random>.us-east-2.rds.amazonaws.com on [2600:4040:5710:9100::1]:53: no such host

This seems to be an error related to looking up the RDS endpoint's IP address. To debug this I used the dig tool from my laptop and from within an SSH session of the "control plane server".

I found that from my laptop, whether or not I'm connected to the VPN, dig <rds instance ID>.<random>.us-east-2.rds.amazonaws.com returns 0 answers. However my laptop isn't completely clueless about this URL. I can ask for the name servers and dig returns the name servers ns-573.awsdns-07.net. awsdns-hostmaster.amazon.com..

If I SSH into the "control plane server" I actually get an A record back for the RDS endpoint URL. It's an IP address in the 10.1.2.0/24 subnet. I also get back the same name server results.

I have tried disabling split-tunnel mode on the VPN and I get the same dig results from my laptop.

I cannot exactly give my entire network configuration with all the security groups and such, but I followed this RDS over VPN official AWS guide almost exactly. The only modifications were adding a public subnet with a NAT -> IGW and the modification described in the following paragraph.

I had one question about the guide however, to me the security group rules laid out regarding VPN client CIDRs didn't make sense.

The guide says the CIDR in the security group rule is the CIDR which VPN clients will get IPs from. The security group uses 122..... However the VPN configuration uses 192..... So I changed the security group rule to match the actual VPN CIDR. Was this a mistake?

Am I missing anything about how I can get the AWS DNS servers to give my private subnet IPs when connected via the VPN? My hypothesis is that when my laptop makes a request to the AWS DNS server for RDS it sees I am connecting from an external network, and not the private subnet from which the RDS endpoint IP is allocated. So it refuses to leak information and says there are no results.