How can I connect to an Amazon RDS DB instance from an Amazon SageMaker notebook instance that's in a different VPC?

Resolution

1.    Create a VPC peering connection.

2.    After the VPC peering connection is active, update the route tables. The RDS DB instance subnet and the SageMaker notebook instance subnet must have a route to each other. For example, for the following CIDR blocks:

SageMaker VPC CIDR block: 192.168.0.0/16

RDS DB instance VPC CIDR block: 10.0.0.0/24

The Amazon RDS DB Instance subnet route table looks like this:

The SageMaker notebook instance subnet route table looks like this:

3.    Confirm that both security groups are configured correctly. To allow traffic from the notebook instance to the RDS DB instance:

On the notebook instance security group: Be sure that there's an outbound rule that allows traffic to the RDS DB instance.

On the DB instance security group: Be sure that there's an inbound rule that allows the notebook instance's security group, VPC CIDR block, or subnet CIDR block.

4.    To test the connection, run the following command in a terminal on the SageMaker notebook instance. Replace the following values in the example:

mydatabase.c5y9vfc8igjj.ap-southeast-2.rds.amazonaws.com: the DB instance endpoint

3306: the DB instance port

curl -v mydatabase.c5y9vfc8igjj.ap-southeast-2.rds.amazonaws.com:3306

Troubleshooting

If the notebook instance has direct internet access activated, then the test command might fail (for example, "Connection timed out"). If this happens, run the following command to check the notebook instance's routing table at the operating system level:

route -n

When direct internet access is activated, the routing table looks similar to the following:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0
10.0.32.0       0.0.0.0         255.255.224.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.255.0   U     0      0        0 veth_def_agent
169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0
172.16.0.0      0.0.0.0         255.255.224.0   U     0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a785eca34a3
192.168.0.0     0.0.0.0         255.255.128.0   U     0      0        0 eth2
192.168.0.0     192.168.0.1     255.255.0.0     UG    0      0        0 eth2

In this example, the DB instance's VPC CIDR block is 10.0.0.0/24. The operating system's routing table doesn't have a route to 10.0.0.0/24. Instead, traffic to the RDS VPC CIDR block uses the primary network interface (eth0), which handles public traffic. To resolve the connection issue, modify the routing table to use eth2, which is the notebook instance's VPC elastic network interface:

1.    Note the SageMaker VPC router address. In this example, 192.168.0.0/16 (the notebook instance's VPC CIDR block) is routed to 192.168.0.1. This means that 192.168.0.1 is the VPC router address.

2.    Add the route to the routing table in the notebook instance terminal. Replace these values in the example:

10.0.0.0/24: the RDS DB instance's VPC CIDR block

192.168.0.1: the SageMaker notebook instance's VPC router address

sudo ip route add 10.0.0.0/24 via 192.168.0.1 dev eth2

3.    Confirm that the new route is in the routing table:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0
10.0.0.0        192.168.0.1     255.255.255.0   UG    0      0        0 eth2
10.0.32.0       0.0.0.0         255.255.224.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.255.0   U     0      0        0 veth_def_agent
169.254.169.254 0.0.0.0         255.255.255.255 UH    0      0        0 eth0
172.16.0.0      0.0.0.0         255.255.224.0   U     0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a785eca34a3
192.168.0.0     0.0.0.0         255.255.128.0   U     0      0        0 eth2
192.168.0.0     192.168.0.1     255.255.0.0     UG    0      0        0 eth2

4.    Test the connection again.

Note: Changes to the operating system routing table don't persist between notebook instance sessions. This means that you lose the changes when you stop and start the SageMaker notebook instance. To mitigate this, use a lifecycle configuration to add the route every time that you start the notebook instance.

Related information

Connect a notebook instance to resources in a VPC

Connect to SageMaker through a VPC interface endpoint