EKS Multi Cluster Strategy


We want to separate applications by accounts for security purpose and billing as well. What's best approach whether create central cluster with workloads in individual accounts communicating through networking medium (TGW, VPC peering / subnet sharing) or separate cluster for each workload ?

asked a year ago1510 views
1 Answer

There are many ways to design and build a multi-tenant SaaS solution on Amazon EKS, each with its own set of trade-offs. You might employ a cluster-per-tenant model to isolate their tenants. This would have a simple isolation story, but may also come with a hefty price tag.

Other is shared compute model where all tenants are combined within the cluster and namespace and isolation is handled at the application level. As this achieves a great deal of operational and cost efficiency, it also represents a less compelling isolation model.

In the middle of these two extremes is namespace-per-tenant isolation, where each tenant is deployed into the same cluster but separated from one another using namespaces and a series of native and add-on Kubernetes constructs. This is what’s commonly referred to as a “silo” model where tenant resources are not shared by tenants.

The below document will walk through the key architectural elements of the EKS sample architecture. How to isolate tenants within an EKS cluster, automate tenant onboarding, manage tenant identities, and support routing of tenant workloads. https://aws.amazon.com/blogs/apn/building-a-multi-tenant-saas-solution-using-amazon-eks/

Further you can refer the below Document for Multi-tenant design considerations for Amazon EKS clusters https://aws.amazon.com/blogs/containers/multi-tenant-design-considerations-for-amazon-eks-clusters/

Below is a public guide that covers both the Soft and Hard multi-tenancy and the EKS Shared Responsibility Model: https://aws.github.io/aws-eks-best-practices/ https://aws.github.io/aws-eks-best-practices/security/docs/multitenancy/

Hard-tenant design can be achieved creating different namespaces, roles + role bindings, and network policies for this logical separation between tenants.

A basic best practice for handling multiple tenants is to assign each tenant a separate namespace. However, keep in mind that, for example in some cases, the same team might need multiple namespaces for hosting different builds of their application. You will be able to block the traffic between namespaces using different network policies.

In order to provide proper isolation from users k8s uses a RBAC authorization model, which AWS applies via IAM Users and Roles. It is important to consider while sharing a Cluster, is to set Resource Quotas between the users. This will provide a fair access to the resources that they need (CPU, storage, memory, etc).

--- EKS Best Practices Guides // Soft Multi-tenancy - https://aws.github.io/aws-eks-best-practices/security/docs/multitenancy/#soft-multi-tenancy --- EKS Best Practices Guides // Hard Multi-tenancy - https://aws.github.io/aws-eks-best-practices/security/docs/multitenancy/#hard-multi-tenancy

--- Multi-tenant design considerations for Amazon EKS clusters - https://aws.amazon.com/blogs/containers/multi-tenant-design-considerations-for-amazon-eks-clusters/

When yo use multiple namespaces - https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#when-to-use-multiple-namespaces Network Policies - https://kubernetes.io/docs/concepts/services-networking/network-policies/ Using RBAC Authorization - https://kubernetes.io/docs/reference/access-authn-authz/rbac/ RoleBinding and ClusterRoleBinding - https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding Cluster authentication - https://docs.aws.amazon.com/eks/latest/userguide/managing-auth.html Managing users or IAM roles for your cluster - https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html Resource Quotas - https://kubernetes.io/docs/concepts/policy/resource-quotas/

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions