How can I deploy a Jumpstart foundational model like llama2 and an endpoint through cloudformation?

0

I have created a LLM inference endpoint using sagemaker studio. Making the endpoint with the studio involves choosing the model from various jumpstart models, configuring permissions, and selecting configurations for the EC2 instance it is running on. However the whole setup takes just a couple minutes. (https://aws.amazon.com/blogs/machine-learning/learn-how-to-build-and-deploy-tool-using-llm-agents-using-aws-sagemaker-jumpstart-foundation-models/)

How can I repeat this with cloudformation? I see that a way to do this is with a python sdk (https://sagemaker.readthedocs.io/en/v2.75.1/overview.html#deploy-a-pre-trained-model-directly-to-a-sagemaker-endpoint) but I'd like to make this into a stack I can stand up or tear down.

2 Risposte
1

Hi Tim,

Here is an example of a CDK deployment of a GenAI model through SageMaker JumpStart - https://aws.amazon.com/blogs/machine-learning/deploy-generative-ai-models-from-amazon-sagemaker-jumpstart-using-the-aws-cdk/

You may want to go through this blog to get a fair idea of how you can make use of CDK (which uses CloudFormation behind the scenes) to deploy the infra and code.

Thanks, Arjun

AWS
con risposta 5 mesi fa
0

Hello Tim,

You can leverage the following AWS Machine Learning Blog which discusses the deployment deployment of different models provided by Amazon SageMaker JumpStart (such as Stable Diffusion model for image generation, the FLAN-T5-XL model for natural language understanding (NLU) and text generation from Hugging Face) via AWS CDK.

[+] Deploy generative AI models from Amazon SageMaker JumpStart using the AWS CDK

Here’s the CDK sample referred in the above blog - https://github.com/aws-samples/generative-ai-sagemaker-cdk-demo

Hope this helps!

AWS
con risposta 5 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande