Suggestion for Kubernetes Deployment Strategy

Kubernetes will try to schedule these 12 replicas across your available nodes. If you have multiple m6g.2xlarge instances in your cluster, Kubernetes may distribute the replicas across these instances. However, if you only have one instance or if other instances are also heavily utilized, Kubernetes may end up scheduling more than four containers on a single instance, leading to the aforementioned resource overallocation issue.

When more than four containers are running on a single m6g.2xlarge instance, they will have to share the available CPU cores. Since each container is designed to utilize 2 cores, having to share means they won't perform as well as they would if they had exclusive access to 2 cores each. This can lead to slower processing times and overall reduced efficiency.

You can use pod anti-affinity rule to ensure that no more than four containers of a specific type are scheduled on the same EC2 instance. Additionally, set resource limits in your container specifications to allocate two CPU cores per container.

In this setup, each container receives the necessary resources, and Kubernetes will automatically schedule additional containers on new EC2 instances when the resource limits are reached. Additionally, consider adding more EC2 instances to your cluster to ensure sufficient CPU resources for all containers.

Key sources:

• Pod Topology Spread Constraints
• pod anti-affinity
• Requests and limits

To address your requirements, you can leverage Kubernetes' resource requests, and limits, along with the cluster autoscaler. Here's a step-by-step strategy:

1. Resource Requests and Limits:

   • In your Deployment or DaemonSet manifest, set the resources.requests and resources.limits for CPU and memory based on your workload requirements.
   • For example, if each of your background tasks requires 2 CPU cores, set resources.requests.cpu: 2 and resources.limits.cpu: 2.
   • Kubernetes will schedule pods based on the requested resources and prevent over-provisioning on a single node.

2. Cluster Autoscaler:

   • Deploy the Cluster Autoscaler in your Kubernetes cluster to automatically scale the number of nodes (EC2 instances) based on resource demands.
   • When the cluster detects that more pods need to be scheduled, and the existing nodes don't have sufficient resources, the Cluster Autoscaler will launch new EC2 instances.

3. Separate Deployments or Namespaces:

   • If you plan to run different algorithms alongside your background tasks, consider creating separate Deployments or Namespaces for each type of workload.
   • This separation will allow you to apply different resource requests, limits, and node selectors for each workload, ensuring better isolation and resource management.

4. Monitoring and Alerting:

   • Set up monitoring and alerting for your Kubernetes cluster to track resource utilization, pod scheduling failures, and other relevant metrics.
   • This will help you identify potential issues and take proactive measures to maintain optimal performance.

By following this strategy, you can ensure that only four pods are scheduled per EC2 instance, maintaining optimal performance for your background tasks. Additional containers will be scheduled on new EC2 instances automatically provisioned by the Cluster Autoscaler. Furthermore, you can isolate different workloads using separate Deployments or Namespaces, allowing for better resource management and optimization.

Note that this approach assumes you're running Kubernetes on AWS EC2 instances. If you're using a managed Kubernetes service like Amazon Elastic Kubernetes Service (EKS), some of the steps may be slightly different, but the overall principles remain the same.