how to choose ml.g4dn.* instances in sagemaker processing jobs

1

I have to perform some data manipulation for which the sagemaker "processing job" would fit perfectly. Such jobs would benefit from GPU and thus I was looking to use instances from the ml.g4dn family for cost efficiency. Unfortunately, I cant see them available in the dropdown when creating a processing job from the aws dashboard, only when creating training jobs. I previously requested the limit increase to the aws support, and i was told it was not necessary and up to 20 instances could be run in the chosen region.

Am I missing anything? do i have to enable the instance family somewhere else?

thanks

質問済み 2年前1100ビュー
1回答
1

Hi there, thanks for reaching out about your issue. Are you able to use the AWS CLI or SDKs to start Processing jobs with the ml.g4dn instance types? Also, could you clarify whether AWS support increased your ml.g4dn quota for both Processing and Training or not?

AWS
回答済み 2年前
  • i think i tried via cli before engaging with the support team, but ill try again today both with the CLI and the SDK as I'll start working on the pipelines we need. I'll let you know asap

    concerning the aws support, they first told me "I can confirm you that the ml.g4dn.xlarge should be already available up to 20 concurrent instances. As soon as I receive any response from our service team, I will let you know immediately." then, the service team confirmed: "I just receive a response from our service team. They confirmed me that the ml.g4dn.xlarge instance at a current quota of 20 is available for processing jobs in the EU (Ireland) region."

  • ok, so running via cli it works, while the dropdown is still empty. I would assume that via the sdk it should work too. But I still cant see the instance type in the UI. I can workaround it for a while, but Id still have users would need it via the dashboard. Any suggestion how to move from here?

  • Thanks for trying out CLI, and SDK should work as well. The console issue you have surfaced here seems to me a bug. We should list all supported instance types regardless of quotas. I have raised an internal ticket to look into this issue. In the meantime, I would recommend leveraging CLI or SDK.

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ