Adding our own worker to MWAA

0

Hi,
I was wondering if it was possible (or in the roadmap) to add our own Worker to MWAA ?

We are currently running 95% of our dags on MWAA with small workers.
but the 5% left are more compute heavy and crash MWAA worker even if we scale up.

We have some machines at the office that we could use has worker to compute those dags.
Is that something we can (/will be able to) do?

We could tell the heavy compute dags to go in a specific queue and our worker at home would listen only to those queue.

Thanks for your feedbacks in advance

LouisAW
已提問 3 年前檢視次數 827 次
2 個答案
0

Hi!

Worker pool support is not on the immediate roadmap for MWAA, so the recommendation is to run those few high-compute workloads on AWS Batch, ECS, or EKS--an example of the latter at https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-eks-example.html

Thanks!

AWS
John_J
已回答 3 年前
0

Thanks !
I'll take a look.

LouisAW
已回答 3 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南