- Newest
- Most votes
- Most comments
This error typically occurs when the available resources are less than what is required by the task. While you may not be explicitly using a lot of memory, it is possible that one of the Python libraries that you or your operator are using may be exceeding what is available in the AWS Fargate container used by MWAA.
One option is to try refactoring your tasks to use different libraries or operators. You may also want to test your DAGs using the MWAA local runner where you can more directly monitor what resources each task is requiring.
Another option is to reduce the number of tasks per worker, as it is possible that the aggregation of tasks might be consuming excessive resources. See celery.worker_autoscale setting in the MWAA Documentation
Relevant content
- asked 2 years ago
- asked 5 months ago
- asked 2 years ago
- asked 7 months ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago