Where can I find guidance for getting a customer started with SageMaker sizing and cost?
A customer wants to use SageMaker, but doesn't know how to get started with instance sizes or how to forecast the cost for it. I've looked at the SageMaker TCO PDF we have online, but that appears more marketing than helpful, i.e. more price comparison than guidance.
I know that the SageMaker cost is really the underlying EC2 and storage pieces, not SageMaker itself. However, I feel it is incorrect to say that they start with (say) t3.medium and see if that fits and scale up if they need more power behind it. As well, that doesn't help them to forecast either.
Any thoughts here?
- Newest
- Most votes
- Most comments
See the performance efficiency and cost optimization pillars in Machine Learning Lens.
Additionally this is an EC2 based right sizing best practices guide.
Overall, it's better to start small, then increase instance size as needed (as those that start large, never bother reduce the size), or apply auto scaling for SageMaker hosting.
Assuming a CPU ML predictions: When choosing ml.t2.medium instances the customer will need to keep an eye on the instance CPU credits. If they lack the knowledge, just start with M5.
Relevant content
- asked 8 months ago
- asked 10 months ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
Sorry for reviving an old question but where is it possible to view CPU credit usage when hosting Sagemaker endpoints on T instance types (specifically ml.t2.medium instances)? These instances don't appear in the EC2 console and the only CPU metric available in CloudWatch when clicking on the "View instance metrics" link under the "Monitor" section for an endpoint is CPUUtilization. Neither the CPUCreditUsage or CPUCreditBalance metrics seem to be available for Sagemaker endpoints even when using T instance types.