Service quotas on Sagemaker Jumpstart endpoints?

0

I am trying to deploy a proof-of-concept application using text-to-text LLM for a document knowledge repository. Every time I try to deploy a Sagemaker Jumpstart model I get some sort of a provisioning error wherein I have to subscribe to the model, and wait, or request a service quota limit increase for that particular type of endpoint, and wait for "some time". Is there any model that is actually available for this task without having to wait for an indeterminate amount of time for provisioning? Do I need to upgrade my support plan, and if so would that get me faster provisioning? It seems like it would be pretty hard to use Sagemaker services if none of the models are practically available. Has anyone else run into this issue, and if so, what was the magic sauce to fix it.?

esk456
asked 9 months ago1802 views
1 Answer
2

Hello,

Few suggestions that may help with deploying SageMaker Jumpstart models:

  • Some of the more popular models do reach capacity limits temporarily, so trying again later may work. You could also try less commonly used models that may have availability.
  • Upgrading your support plan would not affect provisioning times - the model availability depends more on overall usage and quotas for that specific model.
  • You could consider deploying the model to your own infrastructure using techniques like model packaging. This avoids SageMaker's shared capacity but requires managing the infrastructure yourself.
  • Reach out to AWS Support - they may be able to check quota limits and provisioning queues for specific models. Sometimes a limit increase just requires a request if the use case merits it.
  • As an alternative, you could look at deploying a model using Amazon SageMaker Neo or AWS Lambda. These have their own performance and cost tradeoffs to consider.
  • I hope these suggestions help provide some workarounds until model availability improves. SageMaker Jumpstart can be very useful once a model is deployed

Thanks

Abhinav

answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions