SageMaker costs for AI model

Question

I recently deployed [anymodality/llava-v1.5-7b](https://huggingface.co/anymodality/llava-v1.5-7b) from Hugging face onto Amazon SageMaker for inference. I created a `model.tar.gz` file which was about 9 GB and uploaded to an S3 bucket to hold my model. The model was deployed for region west 2 with:

```
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.xlarge",
)
```

And now I have an endpoint that I can request to with an image url and text prompt to generate an AI image description. I'm totally new to AWS and SageMaker, I tried reading through the pricing but I dont quite understand it. Can someone explain how much I will be charged and how much I expect to be charged per month? If the model is just idle with no requests, will I be charged? If I continuously use the model for the full hour how much would I be charged? Also if I send a ton of requests at the same time to the server will it auto scale?

re:Post Agent · Accepted Answer

Thank you for providing details about your SageMaker deployment. I'll explain the pricing structure and address your questions.

For your deployment using an ml.g5.xlarge instance, you will be charged based on the time your endpoint is running, regardless of whether it's processing requests or idle. The pricing is per-second, with a one-minute minimum.

If your model is idle with no requests, you will still be charged for the running instance. This is because the instance is always available to serve requests, even when not actively processing.

For continuous use of the model for a full hour, you would be charged the hourly rate for the ml.g5.xlarge instance in the US West (Oregon) region. As of my last update, this was approximately $1.212 per hour, but please check the current pricing as it may have changed.

Regarding monthly costs, if you keep the endpoint running 24/7 for a month, you could expect to pay around $876 (30 days * 24 hours * $1.212/hour), plus any additional charges for data transfer or storage.

Your deployment configuration doesn't include auto-scaling. With the current setup (initial_instance_count=1), your endpoint will not automatically scale if you send a large number of simultaneous requests. It will try to process them sequentially, which might lead to increased response times or potential timeouts if the load is too high.

To enable auto-scaling, you would need to configure it explicitly using SageMaker's auto-scaling feature. This would allow your endpoint to handle increased loads by adding more instances when needed, but it would also increase your costs as you'd be charged for the additional instances.

Remember that you're also incurring some storage costs for your model in S3 and any attached EBS volumes, but these are typically much smaller than the instance costs.

To optimize costs, consider using the endpoint only when needed and shutting it down when not in use. You could also explore using smaller instance types or SageMaker's serverless inference if your workload allows it.

For the most accurate and up-to-date pricing information, I recommend checking the official AWS SageMaker pricing page or using the AWS Pricing Calculator.  
**Sources**  
[Machine Learning Service – Amazon SageMaker Pricing – AWS](https://aws.amazon.com/sagemaker-ai/pricing/)  
[Amazon SageMaker Studio pricing - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated-cost.html)

Shahed Kazi (@shajam) · Answer

You will be charged for the following items

* Storage in S3
* SageMaker instance if you are still running it. You can stop it if not needed.  The charge will depend on the instance size.
* SageMaker endpoint - this should be running all the time. It depends on how many endpoints you deploy.

Fill out these details in AWS pricing calculator, and it will tell you the costs. https://calculator.aws/#/

SageMaker costs for AI model

Add your answer

Relevant content