Negsignal.SIGKILL in MWAA

0

I keep getting the error message Task exited with return code Negsignal.SIGKILL even on tasks not requiring much memory or compute on my airflow DAGs. I have already upgraded from small to medium. Want to know if there's some configuration I can change before upgrading to large or considering another option.

posta 2 anni fa5485 visualizzazioni
1 Risposta
1

This error typically occurs when the available resources are less than what is required by the task. While you may not be explicitly using a lot of memory, it is possible that one of the Python libraries that you or your operator are using may be exceeding what is available in the AWS Fargate container used by MWAA.

One option is to try refactoring your tasks to use different libraries or operators. You may also want to test your DAGs using the MWAA local runner where you can more directly monitor what resources each task is requiring.

Another option is to reduce the number of tasks per worker, as it is possible that the aggregation of tasks might be consuming excessive resources. See celery.worker_autoscale setting in the MWAA Documentation

AWS
John_J
con risposta 2 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande