HealthOmics Task Memory and Concurrent Task Limit Increase Request

Based on your request for increased memory and concurrent task limits in AWS HealthOmics, I can provide some information about the current limitations and potential solutions.

For memory requirements, AWS HealthOmics supports a wide range of memory configurations. The default configuration is 1 vCPU and 1 GiB of memory (omics.c.large), but the service can support much higher memory allocations. For intensive genomic analysis tasks like HaplotypeCaller on large WGS datasets, you can request higher memory configurations. The service supports memory-optimized instances that can provide up to 1536 GiB of memory for the most demanding tasks.

Regarding your specific request to increase from 30GB to 256GB of memory, this is within the supported range of AWS HealthOmics. However, it's important to note that container resource allocations are hard limits - if a task runs out of memory or attempts to use additional vCPUs, it will generate an error log and exit, which appears to be what you're experiencing with your HaplotypeCallerShard tasks.

For concurrent tasks, AWS HealthOmics does have limitations on the maximum number of concurrent tasks. When a task is in "Pending" status, it may remain there if your account has reached the maximum number of concurrent tasks allowed or if the run is part of a run group that has reached its resource maximum values.

For your specific use case processing large WGS datasets like ERR3239334 (~27GB), you might benefit from:

1. Using memory-optimized instance types that provide the 256GB of memory you're requesting
2. Considering GPU-accelerated instances for certain tasks, as they've shown significant performance improvements in genomic pipelines
3. Optimizing your workflow to better handle the scatter-gather pattern with 64 shards

To address your failed runs, note that when a task fails due to resource constraints, HealthOmics transitions it to "Stopping" status (deprovisioning resources) and then to "Failed" status. If the error is a service error and the workflow supports retries, HealthOmics will attempt to process the task again with a new task ID.

For your request to increase concurrent tasks from 25 to 100+, you would need to contact AWS Support as this would likely require a service quota increase specific to your account.
Sources
Compute and memory requirements for HealthOmics tasks - AWS HealthOmics
Task lifecycle in a HealthOmics run - AWS HealthOmics
Benchmarking PacBio whole genome sequencing variant pipeline analysis with AWS HealthOmics workflows | AWS Public Sector Blog