Bastion Host (Public Subnet) unable to reach RDS (Private Subnet) in my VPC


Hi Forum,

first of all: I really enjoy developing and learning new stuff with AWS. But it seems like I've hit a wall here!

I've setup a basic web application stack through AWS CDK and I want to my database to be located in private subnets.
I want to be able to connect to my DB locally through a bastion host which is located in my public subnets currently.

I'm able to connect to my bastion host via Session Manager.
But my bastion host is not able to reach my db:
ERROR 2005 (HY000): Unknown MySQL server host ''

  • Bastion host is inside "Bastion host security group"
  • RDS security group allows connections on port 3306 for Bastion host security group
  • NACL has default setting
  • Route tables in private subnets set to destination: with target:NAT Gateways located in public subnets

I've tried to ping - ping is sent but it never returns
It seems like bastion host can resolve the ip but it never get's an answer
Looks like the firewall eats the request - but I'm no expert

I've used AWS CDK to set up all resources.
This is my VPC setup (Java):

    val vpc = new Vpc(this, CdkUtils.generateId("vpc"), VpcProps.builder()  

Public & Private subnets, Route tables, NACL, Nat Gateways and Internet Gateways are all setup automatically. I've read somewhere else that by default public and private subnets should be able to communicate as long as security groups are set up properly.

What am I doing wrong here?

Thank you very much in advance

Edited by: freehugz on Mar 19, 2021 8:15 AM

asked 3 years ago642 views
2 Answers

Routing done by VPC cannot be the issue, because all destinations within a single VPC are hardwired to be directly reachable via a local route that cannot be overridden. Only the route table of the operating system on the bastion host could have an effect. The VPC-level mechanisms for blocking the traffic would be NACLs (which by default permit all traffic, and you said those are not changed) and the security groups, where the connections have to be permitted in two places: 1) in an outbound rule on one of the security groups attached to your bastion host and 2) in an inbound rule on the security group attached to your RDS instance.

You mentioned having the inbound rule in place (assuming that your RDS instance is listening on port 3306), so you might want to check that the corresponding outbound connections are permitted from the bastion host towards the RDS instance's security group. Any local software firewall you might have running on the bastion host may also restrict outbound traffic for tcp/3306.

The error message sounds more like a DNS resolution issue. You said you tested name resolution by pinging the DNS name, but just to be sure, you could narrow the problem down by doing a telnet to port 3306 on the IP address of the RDS instance. If that doesn't work, then something is failing at the network level. Since routing is not an issue within a VPC, the places to permit the MySQL traffic would be: 1) outbound rules on the bastion host's local software firewall, 2) outbound rules in the bastion host's security group, 3) NACLs attached to the subnets hosting the bastion host and the RDS instance (with default NACLs permitting all traffic in all directions), and 4) inbound rules on the RDS instance's security group.

If telnetting to the IP address works, then you either have a DNS problem (try adding a dot to the end of the name to specify it's a FQDN and prevent any DNS suffixes from being appended), or the MySQL driver/library/client app requires a different syntax for specifying the destination (such as by specifying the port number explicitly).

answered 3 years ago

Hi LeoMk,

thank you very much for replying!

I made the most stupid mistake, damn!
I had to exclude the port from the 'host' parameter in my mysql connection statement.

You saved me a lot of trouble :)

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions