Skip to content

Questions tagged with Amazon SageMaker Deployment

Amazon SageMaker provides a broad selection of machine learning (ML) infrastructure and model deployment options to help meet your needs, whether real time or batch. Once you deploy a model, SageMaker creates persistent endpoints to integrate into your applications to make ML predictions (also known as inference). It supports the entire spectrum of inference, from low latency (a few milliseconds) and high throughput (hundreds of thousands of inference requests per second) to long-running inference for use cases such as natural language processing (NLP) and computer vision (CV). Whether you bring your own models and containers or use those provided by AWS, you can implement MLOps best practices using SageMaker to reduce the operational burden of managing ML models at scale.

Content language: English

Filter questions
Select tags to filter
Sort by
Sort by most recent
Filter Questions by:

Browse through the questions and answers listed below or filter and sort to narrow down your results.

94 results
We are experiencing recurring InsufficientInstanceCapacity errors with a SageMaker real-time inference endpoint in ca-central-1 (Canada Central). Our workload is an LLM inference endpoint that requir...
3
answers
0
votes
104
views

asked a month ago

**Context** * Base image (AWS DLC): 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-gpu-py312-cu124-ubuntu22.04 * This had been stable for months. ...
1
answers
0
votes
140
views

asked 7 months ago

I have deployed a model in AWS Sagemaker to generate embeddings. I used huggingface-tei image with version 1.8.2. It deployed well. Initially, It works fine and after sometimes it returns None in plac...
1
answers
0
votes
120
views

asked 8 months ago

Hey everyone, I am facing some issue in deploying my endpoint on Sagemaker (later to be invoked by a downstream service - ECS). Here's my code in deploying it: ``` import os, tarfile, uuid, boto3, s...
1
answers
0
votes
233
views

asked a year ago

Below is my docker file. I can run everything locally serving bentoml just fine however in cloud watch the Sagemaker endpoint never actually initializes it loops. I have tried multiple configurations ...
2
answers
0
votes
127
views

asked a year ago

I have created a serverless endpoint for inference using my own ECR image. However, creation fails with no CloudWatch logs... I do everything I can ...
1
answers
0
votes
212
views

asked a year ago

![Enter image description here](/media/postImages/original/IMiXyVYzfRQp6i7mAmRVTbfQ) The figure above illustrates the directory tree for model_data model.tar.gz, where inference.py has dependency on...
2
answers
0
votes
362
views

asked 2 years ago

Hi , I have few queries regarding sagemaker. Are there any APIs that will provide- 1. DETAILED info on the running models (instances) deployed on Sagemaker endpoint ? 2. DETAILED info on the list of M...
1
answers
0
votes
318
views

asked 2 years ago

I followed AWS tutorials to deploy a model on Sagemaker, but when the server deployed takes several minutes to process a request, it runs the same processing multiple times for an invocation. I report...
2
answers
0
votes
411
views

asked 2 years ago

Hello, I am trying to serve a model using SageMaker Endpoint. I am using Triton Inference Server as a framework, I know that I can enable Triton's gRPC protocol communication by setting the `SAGEMA...
1
answers
0
votes
787
views

asked 2 years ago

I am curious about trying canvas, and I was wondering if I can use the models I build in canvas the same way I use the models I have built using automl. Specifically will I be able to use the sagemak...
1
answers
0
votes
337
views

asked 2 years ago

Hi, I created a postgres RDS database which I configured and was able to connect via pgAdmin. However when I try to connect to the db via code, I get timeout error For context my vpc security group r...
2
answers
0
votes
391
views

asked 2 years ago

  • 1
  • 2
  • 3
  • 4
  • 5
  • •••
  • 8
  • Page size
    12 / page