Skip to content

Questions tagged with Amazon SageMaker Deployment

Amazon SageMaker provides a broad selection of machine learning (ML) infrastructure and model deployment options to help meet your needs, whether real time or batch. Once you deploy a model, SageMaker creates persistent endpoints to integrate into your applications to make ML predictions (also known as inference). It supports the entire spectrum of inference, from low latency (a few milliseconds) and high throughput (hundreds of thousands of inference requests per second) to long-running inference for use cases such as natural language processing (NLP) and computer vision (CV). Whether you bring your own models and containers or use those provided by AWS, you can implement MLOps best practices using SageMaker to reduce the operational burden of managing ML models at scale.

Content language: English

Filter questions
Select tags to filter
Sort by
Sort by most recent
Filter Questions by:

Browse through the questions and answers listed below or filter and sort to narrow down your results.

93 results
**Context** * Base image (AWS DLC): 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-gpu-py312-cu124-ubuntu22.04 * This had been stable for months. ...
1
answers
0
votes
120
views

asked 6 months ago

I have deployed a model in AWS Sagemaker to generate embeddings. I used huggingface-tei image with version 1.8.2. It deployed well. Initially, It works fine and after sometimes it returns None in plac...
1
answers
0
votes
110
views

asked 7 months ago

Hey everyone, I am facing some issue in deploying my endpoint on Sagemaker (later to be invoked by a downstream service - ECS). Here's my code in deploying it: ``` import os, tarfile, uuid, boto3, s...
1
answers
0
votes
224
views

asked 9 months ago

Below is my docker file. I can run everything locally serving bentoml just fine however in cloud watch the Sagemaker endpoint never actually initializes it loops. I have tried multiple configurations ...
2
answers
0
votes
116
views

asked a year ago

I have created a serverless endpoint for inference using my own ECR image. However, creation fails with no CloudWatch logs... I do everything I can ...
1
answers
0
votes
196
views

asked a year ago

![Enter image description here](/media/postImages/original/IMiXyVYzfRQp6i7mAmRVTbfQ) The figure above illustrates the directory tree for model_data model.tar.gz, where inference.py has dependency on...
2
answers
0
votes
346
views

asked a year ago

Hi , I have few queries regarding sagemaker. Are there any APIs that will provide- 1. DETAILED info on the running models (instances) deployed on Sagemaker endpoint ? 2. DETAILED info on the list of M...
1
answers
0
votes
304
views

asked 2 years ago

I followed AWS tutorials to deploy a model on Sagemaker, but when the server deployed takes several minutes to process a request, it runs the same processing multiple times for an invocation. I report...
2
answers
0
votes
397
views

asked 2 years ago

Hello, I am trying to serve a model using SageMaker Endpoint. I am using Triton Inference Server as a framework, I know that I can enable Triton's gRPC protocol communication by setting the `SAGEMA...
1
answers
0
votes
762
views

asked 2 years ago

I am curious about trying canvas, and I was wondering if I can use the models I build in canvas the same way I use the models I have built using automl. Specifically will I be able to use the sagemak...
1
answers
0
votes
311
views

asked 2 years ago

Hi, I created a postgres RDS database which I configured and was able to connect via pgAdmin. However when I try to connect to the db via code, I get timeout error For context my vpc security group r...
2
answers
0
votes
375
views

asked 2 years ago

I am new to AWS and I do not understand a lot things here. I have a trained LSTM model in <model-name>.keras format. I am looking to deploy this model as a batch transformer. I have a custom script t...
1
answers
0
votes
660
views

asked 2 years ago

  • 1
  • 2
  • 3
  • 4
  • 5
  • •••
  • 8
  • Page size
    12 / page