How can I deploy a Jumpstart foundational model like llama2 and an endpoint through cloudformation?

0

I have created a LLM inference endpoint using sagemaker studio. Making the endpoint with the studio involves choosing the model from various jumpstart models, configuring permissions, and selecting configurations for the EC2 instance it is running on. However the whole setup takes just a couple minutes. (https://aws.amazon.com/blogs/machine-learning/learn-how-to-build-and-deploy-tool-using-llm-agents-using-aws-sagemaker-jumpstart-foundation-models/)

How can I repeat this with cloudformation? I see that a way to do this is with a python sdk (https://sagemaker.readthedocs.io/en/v2.75.1/overview.html#deploy-a-pre-trained-model-directly-to-a-sagemaker-endpoint) but I'd like to make this into a stack I can stand up or tear down.

2개 답변
1

Hi Tim,

Here is an example of a CDK deployment of a GenAI model through SageMaker JumpStart - https://aws.amazon.com/blogs/machine-learning/deploy-generative-ai-models-from-amazon-sagemaker-jumpstart-using-the-aws-cdk/

You may want to go through this blog to get a fair idea of how you can make use of CDK (which uses CloudFormation behind the scenes) to deploy the infra and code.

Thanks, Arjun

AWS
답변함 5달 전
0

Hello Tim,

You can leverage the following AWS Machine Learning Blog which discusses the deployment deployment of different models provided by Amazon SageMaker JumpStart (such as Stable Diffusion model for image generation, the FLAN-T5-XL model for natural language understanding (NLU) and text generation from Hugging Face) via AWS CDK.

[+] Deploy generative AI models from Amazon SageMaker JumpStart using the AWS CDK

Here’s the CDK sample referred in the above blog - https://github.com/aws-samples/generative-ai-sagemaker-cdk-demo

Hope this helps!

AWS
답변함 5달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠