Adding our own worker to MWAA

0

Hi,
I was wondering if it was possible (or in the roadmap) to add our own Worker to MWAA ?

We are currently running 95% of our dags on MWAA with small workers.
but the 5% left are more compute heavy and crash MWAA worker even if we scale up.

We have some machines at the office that we could use has worker to compute those dags.
Is that something we can (/will be able to) do?

We could tell the heavy compute dags to go in a specific queue and our worker at home would listen only to those queue.

Thanks for your feedbacks in advance

LouisAW
질문됨 3년 전822회 조회
2개 답변
0

Hi!

Worker pool support is not on the immediate roadmap for MWAA, so the recommendation is to run those few high-compute workloads on AWS Batch, ECS, or EKS--an example of the latter at https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-eks-example.html

Thanks!

AWS
John_J
답변함 3년 전
0

Thanks !
I'll take a look.

LouisAW
답변함 3년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인