AWS re:Invent 2024 - Demystifying containers on AWS: What's right for you?

Imagine waking up one morning with a critical decision to make: which container service should you use on AWS? Your team is waiting, management is asking questions, and you need to determine if you should prioritize flexibility, agility, or cost savings. This is the scenario Julia Furst Morgado and Cobus Bernard addressed in their session.

As Julia put it, quoting David Allen: "You can do anything, but not everything". This perfectly captures the container service selection challenge on AWS. The cloud offers endless possibilities to run containers, but each choice involves trade-offs. The decision depends on your needs, your team's expertise, and the challenges you're facing.

The Shared Responsibility Model of Container Management

Before diving into specific services, Cobus walked through how the responsibility model shifts across different container deployment options. He presented a stack visualization showing how AWS progressively takes over more responsibilities as you move up the abstraction layers.

Running containers directly on Amazon EC2 means you manage everything except the physical hardware – networking, pipeline, operating system, and more. With Amazon ECS or Amazon EKS on Amazon EC2, you get orchestration frameworks but still manage the Amazon EC2 instances and their operating systems.

Moving up to AWS Fargate with either Amazon ECS or Amazon EKS eliminates the need to worry about instances. You simply specify RAM and CPU requirements, and AWS handles the rest, though you still manage what happens inside the container.

At the highest level of abstraction, AWS App Runner lets you either provide a container or connect to your source code repository. In the latter case, AWS detects your language, builds the container, and runs it for you, letting you focus primarily on your application code.

The AWS Container Service Groupings

Julia explained that AWS container services can be divided into three groups based on how you interact with them.

The first group includes services where you bring your container image and AWS runs it for you: Amazon ECS, Amazon EKS, AWS App Runner, and AWS Lambda (for specific use cases). These services offer different levels of flexibility and control.

The second group consists of services where you can run containers, but you primarily choose them for their added features. These include AWS App Runner, AWS Amplify (for mobile and front-end applications), and AWS Elastic Beanstalk (for web application deployment and management).

The third group represents services where containers run under the hood, managed entirely by AWS. Many AWS managed services use containers internally to make development faster and more user-friendly.

Julia noted that some services overlap across these groups, which can cause confusion about their exact roles.

The Three Layers of Container Management

To understand container services better, Julia broke down container management into three key layers that work together to support containerized applications.

The capacity layer serves as the foundation, providing raw computing power - CPU, memory, and infrastructure resources. Within AWS, this layer is represented by Amazon EC2, offering virtual machines with full control over configuration, and AWS Fargate, a serverless compute option that abstracts the infrastructure entirely.

The orchestration layer functions like a traffic controller, managing how containers are deployed and scaled across your environment. Amazon ECS is a lightweight, AWS-native orchestrator tightly integrated with the AWS ecosystem. Amazon EKS provides a fully managed Kubernetes service running vanilla Kubernetes. ROSA (Red Hat OpenShift Service on AWS) is designed for hybrid and multi-cloud environments. For extending beyond AWS regions, Amazon ECS Anywhere and Amazon EKS Anywhere let you run containers on-premises or at the edge.

The provisioning layer simplifies interaction with the orchestrator, reducing friction for developers who don't want to deal with deployment configurations or CI/CD pipelines. This layer includes services like AWS App Runner, AWS Elastic Beanstalk, AWS Amplify, and Amazon Lightsail.

Julia also highlighted AWS Lambda as a unique "wild card" service that encompasses all three layers in itself. It provisions compute resources automatically, manages the lifecycle of your code execution, and provides a simple event-driven interface.

Two Approaches to Selecting Container Services

Organizations typically follow one of two approaches when selecting container services. The tactical approach, or fit-for-purpose solution, focuses primarily on the provisioning layer. It's driven by practical solutions to get applications running quickly, common among startups and smaller teams with specific use cases. This approach often doesn't consider broader organizational impact or long-term goals.

The strategic approach is more platform-focused, centered on organizational-level discussions. It aims to build platforms to help developers deploy containerized applications and seeks to standardize deployment across the company. This approach balances long-term needs like scalability, control, and developer experience but requires more operational expertise to implement successfully.

Optimizing for Cost

Cobus shared several cost optimization strategies for container services. When comparing Amazon EC2 and Amazon Lightsail, he explained that Lightsail offers fixed monthly costs, while Amazon EC2 uses per-second billing. He also clarified that Amazon ECS itself doesn't incur additional costs - you only pay for the compute resources.

Amazon Lightsail offers two main options: a Virtual Private Server (VPS) where you choose a size based on resource needs, or a Container Service where you deploy containers without direct access to the host.

AWS Lambda with containers can simplify deployment for certain workloads. Cobus highlighted the significant size difference: a 250MB uncompressed limit for code uploads versus 10GB for container images. AWS Lambda also comes with a generous free tier of up to 1 million requests, 500GB-seconds, and 32 million seconds of compute time.

For substantial cost savings, Cobus recommended spot instances. Both Amazon EC2 Spot and AWS Fargate Spot can dramatically reduce costs, with AWS Fargate Spot running at about 25% of the full AWS Fargate price. You can define instance types with weights for flexible scaling, making this option ideal for batch processing and workloads that can handle interruptions.

Optimizing for Reduced Complexity and Operational Effort

Julia explained options for reducing complexity, starting with AWS App Runner. This service abstracts away nearly all operational complexity by handling scaling, load balancing, and updates automatically. While it comes at a higher cost, the simplicity it provides can be well worth it for teams focused on rapid development.

Amazon ECS with AWS Fargate offers another powerful option. You define task definitions, and AWS handles scaling and provisioning. This combination works well for most containerized workloads, though at a higher cost compared to Amazon EC2-based solutions.

For those who need more control, Amazon EC2 Fleet provides complete management over underlying compute resources. You manage instances, scaling, and updates, but can balance cost and performance with Savings Plans or Reserved Instances. Julia also suggested using spot instances for traffic spikes or dynamic workloads to maximize efficiency.

Optimizing for Flexibility and Scalability

For workloads that need to adapt to varying demands, Julia recommended Amazon EKS with Amazon EC2. This combination brings Kubernetes features together with AWS reliability and simplicity. Amazon EKS manages the control plane, security patches, and Kubernetes API availability, while you control networking, storage, and access policies. It's highly modular with access to both AWS services and CNCF open-source projects, though the trade-off is greater complexity in setup and maintenance.

Amazon EKS with AWS Fargate removes the need to manage nodes or Amazon EC2 instances. AWS handles infrastructure provisioning, upgrades, patches, and maintenance. This option provides simplified scaling with effectively unlimited compute resources and pre-configured CPU/memory combinations for easier resource matching. While it offers better secure isolation, it comes at a higher cost compared to Amazon EC2-based options.

Julia also highlighted Amazon EKS Auto Mode, released at the end of 2024. This feature simplifies Amazon EKS management by automating node provisioning, scaling, and patching. It's ideal for workloads requiring Amazon EC2 instance flexibility with reduced management overhead – offering "the best of both worlds" with Amazon EC2 instances managed by AWS.

EKS Extensions and Integrations

Amazon EKS can be extended with numerous tools and integrations. These include add-ons that come with the cluster or from independent software vendors, AWS Controllers for Kubernetes (ACK) for easier interaction with AWS services, and Amazon EKS Blueprints for CDK or Terraform with ready-to-use templates. For observability, you can implement Amazon Managed Prometheus and Grafana, while Karpenter helps with optimized resource utilization.

Real-World Cost Examples

Cobus shared detailed cost comparisons across different services to help attendees understand the financial implications of their choices.

Starting with the basics, Amazon EC2 (t3a.nano) runs about $3.50/month with 2 vCPUs (burstable) and 0.5GB RAM. With this option, you handle patching, server maintenance, load monitoring, load balancing, and SSL certificates yourself.

Moving up slightly, Amazon Lightsail costs $7.00/month for 0.25 vCPU and 0.5GB RAM. AWS handles patching and host maintenance, but you still manage scaling and potentially load balancing.

For those wanting less operational overhead, Amazon ECS with AWS Fargate runs approximately $10.00/month. This option requires less maintenance, letting you focus on container content and application features. It includes automated scaling and gives you the option to implement AWS Fargate/AWS Fargate Spot ratio for cost optimization.

AWS App Runner comes in at $17.25/month but includes load balancer functionality and custom domain support with Amazon Certificate Manager. You can configure it to scale down to zero or turn off during certain periods to optimize costs.

Finally, AWS Lambda with containers offers the lowest direct cost, with a free tier that includes 1 million requests/month. However, you'll need to consider Amazon API Gateway costs if you're exposing your application as an API.

When to Choose Which Service

Cobus provided pragmatic guidance on when to choose different services based on specific scenarios and limitations.

With Amazon Lightsail, he noted it supports a maximum of 10 container definitions per deployment. Additionally, all containers redeploy with each update, even if only one changed. This might not be ideal if you need to maintain state in your containers or have frequent updates to individual components.

When considering a move from Amazon ECS with Amazon EC2 to AWS Fargate, Cobus suggested making this transition when maintaining and patching servers becomes too burdensome. He shared a personal anecdote about managing 180 servers with Chef at a startup, eventually realizing that simply building new Amazon EC2 images weekly was more efficient than maintaining complex configuration management.

For situations when even Amazon ECS with AWS Fargate seems too complex, moving to AWS App Runner might be appropriate. This service is ideal when you want to focus exclusively on code rather than infrastructure.

The True Cost of Container Services

Perhaps the most valuable insight from the presentation was that infrastructure costs are only part of the equation. Cobus highlighted that developer time is often overlooked when calculating costs. With an average software developer salary of approximately $132,000/year (about $68/hour), the time spent managing infrastructure can quickly exceed the direct savings from choosing a cheaper but more complex solution.

Additional hidden costs include business losses from downtime and the risk of single points of failure in both systems and personnel. Cobus illustrated this with a cautionary tale about writing bash scripts for container orchestration early in his career, only to discover the limitations when specific edge cases occurred.

Conclusion

Selecting the right container service on AWS depends on your specific needs, team expertise, and long-term goals. The presenters emphasized that there's no one-size-fits-all solution – each service offers unique trade-offs between control, simplicity, cost, and flexibility.

As you evaluate options, consider not just the infrastructure costs but also the operational overhead, developer productivity, and business requirements. Sometimes paying a premium for a managed service can result in significant overall savings when you factor in developer time and opportunity costs.

For those interested in learning more, Cobus and Julia have created a repository with examples using Amazon Lightsail, AWS App Runner, Amazon ECS with Amazon EC2, and AWS Fargate, available at community.aws. These examples include both Terraform and CDK versions, allowing you to experiment with different infrastructure setups before deciding which approach best fits your needs.

The full session recording is available on the AWS YouTube channel, featuring Cobus and Julia's detailed explanations and insights.