Cluster-mesh AWS-GCP cross cloud cluster communication

While establishing communication between Amazon EKS and Google Kubernetes Engine (GKE) clusters across different cloud providers is a complex task, there are a few considerations and potential solutions to address your situation.

First, it's important to note that direct VPC peering between AWS and GCP is not natively supported. However, you've already taken a good first step by setting up a VPC tunnel and virtual private network between the two environments.

For cluster-to-cluster communication, especially for services like peer discovery, you'll need to ensure that the necessary network routes and security groups/firewall rules are properly configured to allow traffic between the clusters. This includes ensuring that the Kubernetes network policies in both clusters are set up to allow the required communication.

One approach to consider is using a service mesh solution like Istio, which can help manage cross-cluster communication and service discovery. However, implementing a service mesh across different cloud providers can be challenging and may require additional configuration.

Another option to explore is using Amazon VPC Lattice, which is designed for connecting services across multiple accounts and VPCs. While it's primarily an AWS service, it could potentially be used in conjunction with your VPN tunnel to GCP to improve service discovery and connectivity between your EKS and GKE clusters.

For the specific issue of peer discovery, you might need to implement a custom solution or use a third-party tool that's designed to work across different cloud environments. This could involve setting up a centralized discovery service that both clusters can reach, or using a distributed key-value store like etcd that spans both clusters.

Given the complexity of cross-cloud Kubernetes communication, you may also want to consider using a multi-cloud Kubernetes management platform that's specifically designed to handle these types of scenarios.

Remember to carefully consider the security implications of allowing cross-cluster communication, and ensure that you're following best practices for securing your Kubernetes environments across both cloud providers.

If you continue to face issues, you might need to dive deeper into the network configurations on both sides, ensuring that all necessary ports are open and that DNS resolution is working correctly across the VPN tunnel.

Sources
Secure Cross-Cluster Communication in EKS with VPC Lattice and Pod Identity IAM Session Tags | Containers
Enable secure cross-cluster connectivity with Amazon VPC Lattice - Amazon EKS

Hi,

You may be interested in this fully detailled example implementing Site-to-site VPN between GCP and AWS with dynamic BGP routing: https://oleg-pershin.medium.com/site-to-site-vpn-between-gcp-and-aws-with-dynamic-bgp-routing-7d7e0366036d

Best,

Didier