re:Invent 2025 - From monolith to microservices: Migrate and modernize with Amazon EKS

Many software teams reach a point where their monolithic application, once simple and productive, starts working against them. Deployment cycles slow down, one failing test blocks the entire release, and integrating new capabilities like generative AI feels out of reach without major restructuring. In this post, we'll walk through the architectural journey presented by Nirmal Mehta, Principal Specialist Solutions Architect and Containers Tech Lead at AWS, and Isaac Mosquera, WW Specialist AppMod, from extracting the first microservice to building a fully automated, multi-tenant architecture on Amazon Elastic Kubernetes Service (Amazon EKS).

Breaking Down the Monolith

Monolithic applications are not inherently bad. They're simple, understandable, and efficient at small scale. But as teams grow from a handful of engineers to hundreds, coordination overhead compounds. A single failed quality assurance (QA) test halts the entire release pipeline. Every deployment requires synchronizing multiple teams across the same build-test-release cycle. And when leadership asks for generative AI integration, the architecture simply cannot move fast enough to deliver it.

The Strangler Fig pattern offers a measured starting point. Rather than rewriting everything at once, you identify discrete functional services and extract them one at a time, leaving the rest of the monolith intact. As familiarity with microservices grows, you accelerate the pace. The session traced this pattern from a monolith running in a corporate data center to individual services like payment, billing, and metrics running on Amazon Elastic Compute Cloud (Amazon EC2) instances in AWS.

Containers address the environment consistency problem that surfaces at this stage. When dependencies and runtime configurations are packaged with the application, a service that works on a developer's laptop behaves the same way in production. But as the number of services grows into the hundreds or thousands, you need orchestration. Kubernetes provides service discovery, load balancing, auto scaling, and rolling deployments out of the box. Managing an open-source Kubernetes control plane introduces its own operational complexity, which is where Amazon EKS comes in. EKS provides a fully managed, production-ready control plane with native integration with AWS services including IAM, security groups, and database services.

For workloads that cannot move to the cloud due to compliance or data residency requirements, EKS Hybrid Nodes lets you connect on-premises VMware or bare-metal nodes to an EKS cluster. Your on-premises resources participate in the same cluster alongside cloud-hosted workloads, managed through a single control plane.

Scaling Multi-Tenancy Safely

Once you have microservices running on EKS, the next challenge is growth. For ISVs and SaaS providers, onboarding each new customer as a dedicated EKS cluster is appealing from a compliance perspective, but it does not scale. Each new customer sale generates more operational work, more clusters to manage, and more upgrade cycles to coordinate.

The move toward multi-tenancy starts with Kubernetes namespaces, which create logical boundaries that allow multiple tenants to share a cluster without mixing workloads. There is an important caveat: Kubernetes networking is flat by default, meaning a pod in one namespace can freely communicate with pods in another. Kubernetes network policies let you define explicit rules about which pods can talk to which others, closing off cross-tenant traffic. Combined with the AWS VPC CNI plugin, which assigns a dedicated Elastic Network Interface (ENI) to each pod, you can also apply AWS security groups at the pod level. Your existing AWS network controls then apply to Kubernetes pods the same way they apply to EC2 instances.

Namespace isolation also addresses the noisy neighbor problem: one tenant's runaway workload consuming resources and starving others. Kubernetes resource quotas set hard limits on what a namespace can consume in total, covering CPU, memory, and pod count. Limit ranges add per-container floors and ceilings within a namespace. Together, these controls let you allocate tenant resources predictably without over-provisioning at the cluster level.

EKS Auto Mode handles compute management. Rather than creating and managing multiple node groups for different workload types, Auto Mode lets you declare infrastructure policies using Karpenter-based manifests. You can specify instance types, pricing models (Spot vs. On-Demand), and resource ceilings per tenant, aligned with their service level agreements (SLAs). A tenant contracted for best-effort availability might receive Spot Instances only. A tenant with stricter SLA requirements gets a mix of Spot and On-Demand, with Auto Mode handling provisioning decisions automatically.

For access to AWS resources like Amazon S3, Amazon RDS, and Amazon DynamoDB, hardcoding credentials in containers or environment variables creates compliance and security risks. EKS Pod Identity maps Kubernetes service accounts to AWS IAM roles, so pods receive temporary, automatically rotated credentials scoped to exactly the resources they need. No static keys, no credential sharing between tenants, and full auditability through AWS CloudTrail using the same tooling you already use to manage IAM across your AWS accounts.

From Manual Pipelines to Automated Provisioning

The biggest obstacle to scaling multi-tenancy is usually coordination, not technology. Onboarding a new tenant can take weeks or months even when individual steps are nominally automated. Each step depends on a different team accepting a ticket, running a pipeline, and handing off to the next. The automation is real, but the handoffs between teams are manual. That coordination overhead is what consumes so much time.

The session made the case that Kubernetes itself can solve this problem. EKS already manages complex infrastructure through declarative manifests: Auto Mode provisions EC2 nodes, ingress controllers create load balancers, and the CNI configures networking, all triggered by workload scheduling. That same model can extend to provisioning any AWS resource.

AWS Controllers for Kubernetes (ACK) provides Kubernetes controllers for AWS services including Amazon S3, Amazon RDS, Amazon DynamoDB, AWS IAM, Amazon SageMaker, and Amazon Bedrock. Tenant resources are defined as Kubernetes Custom Resource Definitions (CRDs) alongside namespace and workload manifests. ACK watches for those CRDs and calls the appropriate AWS APIs to create or update resources. Tenant onboarding becomes a single Git commit that provisions the namespace, network policies, resource quotas, IAM role bindings, S3 buckets, databases, and for tenants requiring AI capabilities, Bedrock model endpoints, all in one pipeline.

Paired with a GitOps controller like Argo CD, every infrastructure change flows through Git with a full audit trail. Policy enforcement via Open Policy Agent (OPA) or Kyverno sits at the Kubernetes admission controller layer, rejecting any manifest that does not meet declared rules before it reaches the cluster. You can require that all S3 buckets have public access blocked, that pods handling sensitive data run as non-root, and that every tenant resource carries the correct ownership tags. These rules are declarative, version-controlled, and enforced at deploy time rather than discovered through manual review after the fact.

The architecture that emerges separates a control plane cluster, which manages tenant provisioning, IAM policies, AWS resource creation, and policy enforcement, from application plane clusters that run the actual workloads. Adding GPU resources in a data center, new AWS services, or generative AI capabilities for specific tenants means updating control plane manifests. The pipeline itself does not change.

The journey from monolith to a scalable, compliant, multi-tenant microservices architecture involves incremental steps that each build on the last. Starting with the Strangler Fig pattern, adopting containers and Amazon EKS, layering in namespace isolation, resource quotas, network policies, and EKS Pod Identity, and extending Kubernetes into a full infrastructure automation system with ACK and GitOps gives engineering teams the ability to scale without scaling headcount linearly.

At re:Invent 2025, AWS announced EKS Capabilities, a new feature that provides managed controllers for Argo CD, ACK, and kro (Kubernetes Resource Orchestrator) directly within EKS, making this control plane architecture significantly easier to adopt. Details are covered in session CNS378.

Watch the full session: AWS re:Invent 2025 - From monolith to microservices: Migrate and modernize with Amazon EKS (CNS210)