Service quotas on Sagemaker Jumpstart endpoints?

0

I am trying to deploy a proof-of-concept application using text-to-text LLM for a document knowledge repository. Every time I try to deploy a Sagemaker Jumpstart model I get some sort of a provisioning error wherein I have to subscribe to the model, and wait, or request a service quota limit increase for that particular type of endpoint, and wait for "some time". Is there any model that is actually available for this task without having to wait for an indeterminate amount of time for provisioning? Do I need to upgrade my support plan, and if so would that get me faster provisioning? It seems like it would be pretty hard to use Sagemaker services if none of the models are practically available. Has anyone else run into this issue, and if so, what was the magic sauce to fix it.?

esk456
已提問 4 個月前檢視次數 1577 次
1 個回答
2

Hello,

Few suggestions that may help with deploying SageMaker Jumpstart models:

  • Some of the more popular models do reach capacity limits temporarily, so trying again later may work. You could also try less commonly used models that may have availability.
  • Upgrading your support plan would not affect provisioning times - the model availability depends more on overall usage and quotas for that specific model.
  • You could consider deploying the model to your own infrastructure using techniques like model packaging. This avoids SageMaker's shared capacity but requires managing the infrastructure yourself.
  • Reach out to AWS Support - they may be able to check quota limits and provisioning queues for specific models. Sometimes a limit increase just requires a request if the use case merits it.
  • As an alternative, you could look at deploying a model using Amazon SageMaker Neo or AWS Lambda. These have their own performance and cost tradeoffs to consider.
  • I hope these suggestions help provide some workarounds until model availability improves. SageMaker Jumpstart can be very useful once a model is deployed

Thanks

Abhinav

已回答 4 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南