Sagemaker Inference pricing not pay-as-you-go

Question

We use Sagemaker inference only and I see in CostExplorer we pay every day around ~5.90USD/DAY but don't use Sagemaker serverless inference everyday, probably several times per month.

Can someone explain what is the reason of this pricing? I expect the pricing to be "pay-as-you-go" , not on-demand?

As per pricing example https://aws.amazon.com/sagemaker/pricing/

> With Serverless Inference, you only pay for the compute capacity used to process inference requests, billed by the millisecond, and the amount of data processed. The compute charge depends on the memory configuration you choose.

Answer

What type of endpoint are you using? Serverless Inference you are billed by the amount of time your provisioned infrastructure (memory size you have allocated) is up and running, after a certain amount of idle time these resources are automatically scaled down for you which is when you are not charged. For real-time endpoints as you have a dedicated instance behind the endpoint at all times you are billed on-demand.

Answer

Endpoint type - Real-time, instance type ml.m4.xlarge
Sagemaker is used very rarely, probably once per week or even less, but I see continuous costs per **day**
Do you think using SageMaker Serverless Inference should decrease the pricing? 
We need realtime response for most of models (less than 200ms), would SageMaker Serverless Inference meat such requirements?

Sagemaker Inference pricing not pay-as-you-go

Relevant content