How do I troubleshoot network connectivity from my Amazon SageMaker notebook instance that has direct internet access deactivated?

Short description

Configure SageMaker notebook instances to use the public internet or an Amazon VPC. If you're using an Amazon VPC, then the SageMaker notebook instance uses the virtual private cloud (VPC) instead of the public internet to train or deploy models.

Note: After you create a SageMaker notebook instance, you can't change the network configuration settings. This applies to both Public internet only and VPC only SageMaker notebooks. If your SageMaker notebook requires changes to the network settings, then create a new notebook instance.

To train or deploy models from a notebook instance in VPC mode that has direct internet access deactivated, confirm the following:

• Your VPC has a NAT Gateway.
• Your instance security group allows outbound connections.

If you don't want traffic from the notebook to go over the internet, then use VPC endpoints to connect to services like SageMaker API and SageMaker Runtime. For more information, see Connect to SageMaker within your VPC.

Resolution

Prerequisite: Create a private and public subnet in the same AWS Region as the VPC of the notebook instance.

Use a NAT gateway to activate internet access from a notebook instance in an Amazon VPC with direct internet deactivated

To use a NAT gateway to activate internet access from a notebook instance in an Amazon VPC with direct internet deactivated, complete these steps:

1. Open the Amazon VPC console.
2. In the navigation pane, choose NAT Gateways.
3. Choose Create NAT Gateway and complete these steps:
   (Optional) Specify a name for the NAT gateway.
   Choose the public subnet.
   For Elastic IP allocation ID, associate an Elastic IP address with the NAT gateway. Allocate an Elastic IP address if you don't have one.
4. Choose Create a NAT Gateway.
5. Add the NAT gateway to the route table of the private subnet.

For more information about how to use SageMaker with your Amazon VPC, see Connect a notebook instance in a VPC to external resources.

Perform NAT gateway troubleshooting checks

If your SageMaker notebook instance has networking issues similar to "connection timeout" errors, then complete these steps to check your NAT gateway configuration:

• Verify whether your VPC has an existing NAT gateway associated with it.
• Verify whether your VPC and NAT gateway are in the same Region.
• Verify that your NAT gateway is created within a public subnet.
• Verify whether the NAT gateway is associated as a target in the route table for a private subnet.
• Verify that your destination allows inbound and outbound traffic.
• Verify whether your Amazon VPC security group allows outbound connections.

For more information about NAT gateways, see NAT gateways.

Use a VPC endpoint to activate access to AWS services from a notebook instance with direct internet deactivated

To use a VPC endpoint to activate access to AWS services from a notebook instance with direct internet deactivated, complete these steps:

1. Open the Amazon VPC console.
2. In the navigation pane, choose Endpoints.
3. Choose Create endpoint.
4. For Service category, choose AWS services.
5. For Service name, select the service.
6. For VPC, select the VPC in the same Region as your SageMaker notebook instance.
7. Select the route table associated with the private subnet.
8. Choose Create endpoint.

For more information about VPC endpoints, see Access an AWS service using an interface VPC endpoint.

Perform VPC endpoint troubleshooting checks

If your SageMaker notebook instance has networking issues similar to "connection timeout" errors, then compete these steps to check your VPC endpoint configuration:

• Verify that your Amazon VPC has an existing VPC endpoint associated to the AWS service that you want to connect to. For a full list of supported VPC endpoints, see AWS services that integrate with AWS PrivateLink.
• Verify that both your Amazon VPC and the AWS service to connect to are located in the same Region.
• Verify that the VPC endpoint is associated with a private subnet route table.

Related information

Customize your SageMaker notebook instances with lifecycle configurations and the option to turn off internet access

How do I set up a NAT gateway for a private subnet in Amazon VPC?

Understanding Amazon SageMaker notebook instance networking configurations and advanced routing options