How to increase the ulimit of pods in AWS cluster

To increase the ulimit for cRPD pods in an AWS cluster, you need to modify the pod or container specifications rather than changing the host node settings or making temporary changes within the pod. Here's how you can approach this:

1. Modify your Helm chart to include ulimit settings for the cRPD containers. You can do this by adding securityContext configurations to your container spec.

2. In your Helm chart, under the cRPD container definition, add a securityContext block with ulimits. For example:

   securityContext:
     ulimits:
     - name: stack
       hard: 16384
       soft: 16384

3. This configuration should be added to the container spec in your Helm chart, likely under the crpd section or wherever you define the cRPD container.

4. The name field specifies the type of ulimit you want to set. In this case, it's "stack". Other valid values include "core", "cpu", "data", "fsize", "locks", "memlock", "msgqueue", "nice", "nofile", "nproc", "rss", "rtprio", "rttime", "sigpending", and "stack".

5. The hard and soft fields set the hard and soft limits respectively. Both are required and should be integers.

6. After modifying your Helm chart, redeploy your pods using the updated chart. The new ulimit settings will be applied to the cRPD containers.

Remember that these settings will persist across pod restarts, unlike changes made by exec-ing into the pod. Also, note that ulimit settings are not applicable to jobs running on Fargate resources, so ensure your pods are running on EC2 instances if you're using Amazon EKS.

If you need to set different ulimits for different types (like "nofile" or "nproc"), you can add multiple entries under the ulimits section.
Sources
Ulimit - AWS Batch
AwsEcsTaskDefinitionContainerDefinitionsUlimitsDetails - AWS Security Hub