forgot to mention, also checked VPC ACL,its outbound rule has rule 100 to allow any.
OK will answer the question myself:
opened support ticket with AWS and the support answered my question:
"In order for your NAT Gateway to work properly, you will have to place it in a public subnet ie. the subnet should have an internet gateway attached to it and you should have subnet specific route tables where the 0.0.0.0/0 on your private subnet is routed to your NAT Gateway and the 0.0.0.0/0 on your public subnet is routed to the Internet Gateway."
Worked mostly in Azure ( where Internet gateway and NAT gateway were setup directly by Azure Vnet ), I didn't realize how private network should be configured in AWS.
I'm glad that you were able to solve the issue. Please let us know if you need any further help.
Ec2 Not connecting to InternetAccepted Answerasked 6 months ago
SSH connection to a newly created Ubuntu EC2 instance failsAccepted Answerasked 2 months ago
How can Ec2 Instance in private subnet access internet via vpc endpoint?Accepted Answerasked 3 months ago
VPN Connection Public IPAccepted AnswerEXPERTasked 3 years ago
Client VPN access to VPCasked 2 years ago
Access to AWS Opensearch in a VPCAccepted Answerasked 5 months ago
VPN client endpoint interfaces have public IP, how to remove?Accepted Answerasked 2 years ago
How can I access resources in a peered VPC over site to site VPN?Accepted Answerasked a month ago
EC2 VM doesn't have Internet connectionasked 3 years ago
Advice on creating VPC for EC2 to use IPSec connectionasked 6 months ago