Need guidance on which GPU sagemaker instance is suitable for my huggingface model

0

We want to use huggingface whisper large hindi model on amazon sagemaker instance, https://huggingface.co/vasista22/whisper-hindi-large-v2. The model requires atleast 16GB GPU Memory, and we want to use it along with other hugging face models so we might need additional gpu memory. We are facing Resource Limit Exceeded error. Please let us know which resource to use, and please increase my limit accordingly.

1 Answer
0

Hi,

When deploying large models, the ideal situation is that you can fit the model on a single GPU. This is the best option with respect to performance as it eliminates the overhead of communication between GPU devices. For some models it is simply impossible to fit them on a single GPU due to model size. For other models, they may fit on a single GPU, but it may be more cost effective to partition the model across multiple cheaper GPUs.

In your case, you may consider an instance like g4dn.xlarge if using for inference. Details for using GPU can refer official document.

Hope it helps.

AWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions