- Newest
- Most votes
- Most comments
This may be difficult to troubleshoot without knowing more about the organisation; HCSN is a UK healthcare network that operates across both private and unrouted public IP address space. When you connect to the HSCN network via Direct Connect (DX), you would typically be given a small number of HSCN routable IP addresses (normally just a /30), which cannot overlap with your VPC CIDR range. Since this isn't big enough to create an additional VPC CIDR block from, you would normally create static NATs from your HSCN-side router to private IP VPC addresses in the Amazon VPC (in this case, to your EC2 instance).
If you have been given a bigger block (at least a /27), then you can create an additional CIDR block in the VPC, create two /28 routable subnets (1 per AZ), and then have DX route to those addresses. To scale, you would run something like an NLB for inbound connections, and a NAT Gateway for outbound connections.
The currently defined VIF is private, but that's not to say it should be something else - the people who provide the link to the HSCN network have no clue as to how to set up the networking inside AWS, so I've just been guessing up until this point. The VIFs (as there are two connections for redundancy) are showing "available" with a green tick.
The architecture I was hoping would be quite simple: we run a service on an EC2 instance and we need to pull information via an API from a server inside the HSCN network.
To that end: we have a physical connection from HSCN to the zone where our EC2 instance is located. I've defined a Direct Connect Gateway, and 2 private Virtual Private Interfaces, all of which show as available.
I can see in a "public" Virtual Private Interfaces page that I can specify CIDR prefixes to advertise, but the range we've been asked to advertise is a /29 network, and the smallest that I'm allowed to specify in "public" VIF is /24. Not only that but the range isn't associated with the VPC that our EC2 instance is in, so the definition of our CIDR just can't be entered at all.
I can't create a VPC with the right CIDR, and I can't add the CIDR to our existing VPC.
I'm guessing that I need to define some kind of private NAT or something, but I have no idea really.
I was asked to create a gateway with IP addresses in the 10.x.x.x range (AWS side and HSCN side), our EC2 instance is in our default VPC with a 172.x.x.x range. So I have no idea where a public IP address range should go.
So, for HSCN you definitely don't want to go down a Public VIF route; this is where you are able to connect to publicly available IP addresses in AWS from your network. HSCN is a closed network with separate routing from the Internet, so Private VIF is correct.
The challenge is that because of the number and scale of networks, you can't just assume that you can route the entire VPC CIDR range onto the HSCN - this is also the reason why you only get a /29 address space - which introduces challenges of how you integrate this into an Amazon VPC construct.
Given the complexity of this, I would recommend looking to contact the AWS UK public sector account team; we can give you specific information around routes to connecting to HCSN, both 1st and 3rd party partner options that manage away a lot of the complexity of doing this.
You can contact the team here: https://aws.amazon.com/government-education/contact/ - make sure you flag that this is for the attention of the UK public sector healthcare team.
This knowledge article goes through the details https://repost.aws/knowledge-center/direct-connect-vpc-bgp
- Please elaborate more on your architecture.
- Whether you're using public, private, or transit VIF?
- Is the VIF in UP state or is it in DOWN state?
Essentially, you want to advertise your on-prem CIDR to AWS and VPC CIDR to on-prem.
You first mentioned that the VIF is private, "The currently defined VIF is private". You then mentioned about public VIF, I can see in a "public" Virtual Private Interfaces page that I can specify CIDR prefixes to advertise.
Since you're trying to connect your on-prem to your VPC, then I guess it's a private VIF. Private VIF is used to connect to VPC resources (EC2 in your case). See below knowledge article for more details https://repost.aws/knowledge-center/public-private-interface-dx Also, You cannot create a public virtual interface to a Direct Connect gateway, see https://docs.aws.amazon.com/directconnect/latest/UserGuide/virtualgateways.html
With private VIF, there is no public IP addresses in the picture.
What you need is advertise your default VPC CIDR 172.31.0.0/16 by adding this in the allowed prefixes. https://docs.aws.amazon.com/directconnect/latest/UserGuide/multi-account-associate-vgw.html#allowed-prefixes
I hope this help!
Relevant content
- asked 8 months ago
AWS OFFICIALUpdated 5 months ago- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 9 months ago
This seems like it is what I need to do, I will do some research on setting up "static NATs".
I was hoping that quite a few people would have gone through this process, as the NHS is huge and there must be 1000s of external systems that need to connect into it, i.e. a well trodden path, so things like this would be common(ish) knowledge.
Since that doesn't appear to be the case, I will comment back once I do have it working, to give as many details as I can for my fellow travellers.
So, for HSCN you definitely don't want to go down a Public VIF route; this is where you are able to connect to publicly available IP addresses in AWS from your network. HSCN is a closed network with separate routing from the Internet, so Private VIF is correct.
The challenge is that because of the number and scale of networks, you can't just assume that you can route the entire VPC CIDR range onto the HSCN - this is also the reason why you only get a /29 address space - which introduces challenges of how you integrate this into an Amazon VPC construct.
Given the complexity of this, I would recommend looking to contact the AWS UK public sector account team; we can give you specific information around routes to connecting to HCSN, both 1st and 3rd party partner options that manage away a lot of the complexity of doing this.
You can contact the team here: https://aws.amazon.com/government-education/contact/ - make sure you flag that this is for the attention of the UK public sector healthcare team.
OK, so first thing was to get our IP address allocation increased from /29 to /28. Once that was done, adding the subnet to our VPC was possible.
After adding the new CIDR to the VPC, I was able to update the Virtual Private Gateway to advertise the new IP range, and the connection provider was happy that it was all OK from their end.
So I added a route table in the VPC that only had the 2 local VPC subnets and the CIDRs of the gateways, and a 0.0.0.0/0 rule going through the Virtual Gateway.
I create a new Network Interface on the /28 subnet and attached it to our instance.
I added the new /28 subnet to this route table and ran the Reachability Analyzer between my EC2 instance and known IP addresses inside the HSCN (specifically their DNS servers) and all checked out OK.
However, adding routes on the (Debian) EC2 instance to point the gateway CIDRs and the HSCN IP addresses to the new network interface and then using traceroute failed miserably, not even reaching the Virtual Private Gateway.
Just one last thing was needed to make everything work, adding in the "via <gateway IP address in the assigned CIDR range>" to the routes on our EC2 instance, e.g.
ip route add 155.231.231.1 dev eth1 src 195.156.107.124 via 195.156.107.113
(155.231.231.1 is a DNS server inside the HSCN network, 195.156.107.124 is the host's IP address on the assigned CIDR range, and 195.156.107.113, I presume the vpg's ip address).
I didn't see the 195.156.106.113 IP noted anywhere in the config for the VPC's route table, nor the VPG definition nor any of the VIFs or Direct Connection. I raised a support call with AWS and after about a week they casually mentioned it in a "have you tried this...?" response. It was pingable, so I presume if you have the same issue, you can ping each IP address in the CIDR range and whatever responds back, that isn't your instance's IP address, is the gateway's IP address on the CIDR range.
Or maybe someone knows where this IP address is noted in the config pages?
(note the IP ranges have been updated to random numbers, just the range sizes and relative IP addresses within those ranges are accurate).