EC2 Connectivity problem on port 443

Confirm that Network ACLs associated with the private subnet allow outbound traffic on port 443.

Are you using a NAT Gateway or NAT Instance for internet access?

AWS service endpoints are on the public internet. To connect to the internet from a private subnet over IPv4, you'll first need to create a NAT gateway with a public IP address in a public subnet in the same VPC. In the route table attached to the private subnet, change the route towards 0.0.0.0/0 to use the NAT gateway as its next hop. Then your connections will work.

There's a simple block diagram of the design in documentation: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-example-private-subnets-nat.html. You don't need the Application Load Balancer for your bastion hosts, but you do need at least one NAT gateway.

To save on costs, you should additionally create VPC endpoints of the gateway type for S3 and DynamoDB and associate them with all the route tables of your VPC. This will provide systems in your VPC with a shortcut to S3 and DynamoDB which is completely free to use. If you connect to S3 through a NAT gateway, it'll technically work, but the difference is that the NAT gateway charges you for every byte of traffic passing through it, while running it through a VPC gateway endpoint is free of charge. Note specifically to use the "gateway" type of endpoint; interface endpoints incur similar costs as NAT gateways. The diagram in the documentation also shows the "S3 gateway" symbol to the left.

1. Check Security Groups and NACLs Again

Security Groups: Ensure the security group attached to the private instance allows outbound traffic on port 443 and that the bastion host's security group allows inbound traffic on port 443.

**Network ACLs: **Verify that the Network ACLs for both the public and private subnets allow both inbound and outbound traffic on port 443. Remember that NACLs are stateless, so both inbound and outbound rules need to be explicitly allowed.

2. Check Route Tables

Private Subnet Route Table: Ensure that the private subnet's route table is correctly configured to route traffic to the Internet Gateway via the NAT Gateway or the NAT instance. Without this, your private instance won’t be able to access the internet on port 443.

3. NAT Gateway/Instance Configuration

If you’re using a NAT Gateway or NAT Instance for internet access, ensure it’s properly configured and associated with the private subnet. NAT Instance Security Group: If using a NAT instance, check its security group to ensure it allows outbound traffic on port 443.

NAT Gateway Health: If using a NAT Gateway, ensure it’s healthy and functioning correctly.

4. Check VPC Flow Logs

Enable VPC Flow Logs on your VPC or on the specific subnets. These logs can help you see whether traffic is being allowed or denied, and where the traffic is getting dropped. Check if there are any DENY entries for traffic on port 443.

5. Test Connectivity from the Private Instance

Curl/Wget Test: SSH into your private instance and use curl or wget to test connectivity to external services on port 443:

curl -v https://www.example.com
wget https://www.example.com

6. Check Instance Configuration

Instance Metadata Service (IMDS) Configuration: Ensure that the EC2 instance has access to the IMDS if IAM roles are being used for access. Incorrect IAM role attachment or metadata access issues could cause the CLI to hang when trying to access services.

Network Interface: Check the network interface of the private instance to ensure it’s attached to the correct subnet and that it’s healthy.

7. Application-Level Issues

**SSL/TLS Issues: **If the issue is specifically related to SSL/TLS connections, there might be an SSL/TLS issue or configuration problem at the application level. Consider testing with different SSL/TLS versions or ciphers.

Firewall/Anti-Virus: Ensure there’s no software firewall or anti-virus on the private instance that could be blocking outbound connections on port 443.

8. Reachability Analyzer

Although you mentioned using the Reachability Analyzer, try running it again with a focus on outbound traffic from the private instance to the internet on port 443. This can help identify any overlooked issues.

9. Proxy Settings Check if there are any proxy settings that might be misconfigured, either on the instance itself or within the VPC.

10. Check System Logs

Review the system logs (/var/log/messages, /var/log/syslog, /var/log/awslogs.log, etc.) on the private instance for any errors or warnings related to networking or SSL/TLS connections.

11. DNS Resolution

Ensure that DNS resolution is working correctly on the private instance. Try pinging a well-known domain or using nslookup or dig to confirm DNS resolution.

12. Consider Technical Support

If the issue persists after trying all the above steps, and considering the complexity, it may be worth escalating to AWS Technical Support for more in-depth analysis.