Routing across regions with ALB and ECS for data localisation

0

I am trying to achieve the following setup:

  • Two VPCs, one in Europe and one in a non-European region (call this Region R), both belonging to the same AWS account.
  • Two ALBs, one in Europe and one in Region R.
  • (A single domain name which directs traffic to the closest of the two ALBs.)
  • Two ECS clusters, one in Europe and one in Region R.
  • A path-based rule in both ALBs which directs traffic on the /eu/* path to the ECS cluster in Europe.

The goal of this is to enable a European shard of my application (for data sovereignty/localisation reasons) which is still accessible globally under a single domain. In production, I would be using HTTP header checks in the ALB listener rule to perform the routing to the European cluster, but for simplicity's sake when testing I'm using a path-based rule.

The issue I'm struggling with is that neither ECS nor ALB seem set up to make this possible. I have a VPC each in Europe and in Region R, and they're successfully peered, but:

  1. I can't seem to create an ECS Service in Europe with a load balancer target group in Region R.
  2. I can't create a listener rule on the ALB in Region R which forwards requests to a target group in Europe.

I can't tell from the documentation whether I should be able to do any of those things and haven't found the right incantations, or whether they are all simply not possible. For example, when trying to set up (1) using Terraform, I get an error like this:

Error: updating ECS Service (arn:aws:ecs:[region-r]:[account-id]:service/multi-region-experiment/multi-region-experiment-web):
InvalidParameterException: Unable to assume role and validate the specified targetGroupArn.
Please verify that the ECS service role being passed has the proper permissions.

I think this is a generic error and not actually to do with the service role, but again, I can't find anything useful about this in the docs.

So my specific questions are:

  1. Is it possible to use ECS service load balancing and ALBs across regions? If so, where am I going wrong?
  2. Is this entirely the wrong approach to enabling cross-region request routing? If so, what is the recommended approach?

My ultimate goal is that a user can make requests to my API, and no matter where in the world they're making requests from, they are served from a specific region where that user's account data lives.

Related questions:

  • Multi-region ECS suggests using CloudMap with multiple namespaces. I have not yet investigated this approach, but will give it a go.
  • ECS services in different accounts suggests using "networking and routing tables" and multiple layers of API gateway. This isn't clear to me. I did consider attempting to target the /eu/* rule in Region R at the IP address of the ALB in Europe, but understand that the IP addresses associated with an ALB can change unexpectedly, and I would need some other machinery to detect this and update the rule dynamically.

Common wisdom seems to be "regions are meant to be isolated, don't try to create dependencies between them" which is fair advice, but I'd really like to not have to create a separate domain name for the EU shard of the app.

EDIT: I realised that I did not follow the steps in this article to properly set up peering. I will do that and see if it changes anything, but to the extent that I understand what is going on, it doesn't seem like this will affect my usage of ECS services and ALB target groups across regions. If anything, the peering setup would have prevented network requests successfully reaching the target instances, but I haven't even got that far.

2 Answers
0
Accepted Answer

I don't think this can be achieved with just ALB and ECS. However, using CloudFront at the root of the domain should make this possible. Lambda@edge origin request functions can manipulate a request's origin based on headers or other request characteristics, and could also even do database lookups to help decide. So I think this might be the way to go!

Reference architecture: https://docs.aws.amazon.com/architecture-diagrams/latest/multi-region-api-gateway-with-cloudfront/multi-region-api-gateway-with-cloudfront.html

Lambda@edge origin selection example: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-content-based-S3-origin-request-trigger

answered 10 months ago
0
  • (A single domain name which directs traffic to the closest of the two ALBs.)

Assuming you're using Route 53 for DNS, what's described here is Geoproximity Routing https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy-geoproximity.html So locations that are really close to Europe but not actually in Europe will still route to the European load balancer, which is why I assume you need the extra smarts using HTTP header checks in the ALB listener rule.

Is there a reason you can't use Geolocation Routing instead? https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy-geo.html You would have a default record directing all traffic to the Region R load balancer, except for queries originating in Europe which will be directed to the European load balancer.

profile picture
EXPERT
Steve_M
answered 10 months ago
  • Thanks for your comment! Yeah I've definitely considered just using geo-routing. However, that doesn't let me direct traffic from outside EU, to the /eu path, into the EU cluster. While that is the uncommon scenario, and most EU-bound traffic will originate from inside the EU, I don't want to have to force users to use VPNs in order to access their account data when roaming if I don't have to.

  • How about using the SNI extension for TLS?

    An unauthenticated user hits www.yourapplication.com where the www record is using Route 53 geolocation, as per my previous message.

    Once a user is authenticated, the app is setup so that they use www-e.yourapplication.com (for a European user) or www-g.yourapplication.com (global, i.e. non-European user). These are both CNAMEs pointing to the www record, so the user will still hit the same load balancer depending on where they are in the world, but then the ALB looks at the SNI and if it's www-e then redirect to the ECS cluster serving up the /eu path.

    This probably has some holes in it, and it definitely needs refining, but at a high level I hope you can see what I'm trying to get at.

  • That's a reasonable approach, I just don't know how I would actually do the "then redirect to the ECS cluster serving up the /eu path" bit!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions