MWAA - Fargate worker spinup time

0

Hi team, I'm curious to know if anyone knows the approximate amount of time it takes for MWAA to request extra workers from Fargate and how long it takes for the fargate worker to be ready to start taking on tasks. Searching around the internet I see anywhere from 10 minutes to 1 minute. From what I've seen I'd guess its like 5 minutes?

Does anyone have documentation on this or can give me an estimation of the process from MWAA request to the handoff of a MWAA task to the worker?

Thanks

2回答
0

The time it takes for an Amazon ECS fargate instance to become ready depends on various factors, including the startup time of your Docker container, the size and complexity of your task. Fargate does not cache images, and therefore the whole image is pulled from the registry when a task runs. It is recommended to use larger task sizes with additional vCPUs. This is to reduce the time taken to extract the image. To further answer your question, we require details that are non-public information. Please open a support case with AWS using the following link [1].

[1] https://console.aws.amazon.com/support/home#/case/create

[+] https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/pull-behavior.html#fargate-pull-behavior

[+] What is AWS Fargate? - https://docs.aws.amazon.com/AmazonECS/latest/userguide/what-is-fargate.html

[+] Best Practices - Speeding up task launch - https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/task.html

[+] Get started with Amazon Managed Workflows for Apache Airflow - https://docs.aws.amazon.com/mwaa/latest/userguide/get-started.html

AWS
回答済み 5ヶ月前
0

Hi there,

I don't think it warrants a support case at the moment. I just noticed it seemed like it was not an insignificant amount of time for the instances to spin up and was curious.

I thought that perhaps the airflow fargate instances would be about the same size, but I guess requirements.txt and any other add-ons could raise the start time.

Thanks for your response.

回答済み 5ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ