My site working slowly because EC2 can not load large machine learning model

0

Hello, I am trying to do some predictions with machine learning model which is 500mb, but when I just use load model function with pickle, it takes too much time and most of times it throw timeout error. I think size is the issue because with small sized model file it works perfect.

  • Hi Togrul.

    What type of EC2 instance are you using? Have you tried increasing the size within the same instance family to run a performance test?

    Thanks.

  • Hi Jose, I am using t2.micro. I have not tried it, actually, what I am trying to know is what is the best practice? If changing the instance solve the problem or I should use other services? I have checked the usage of the instance and the max CPU usage was 52% which I think is not critical.

1 Answer
2
Accepted Answer

After you train your machine learning model, you can deploy it using Amazon SageMaker to get predictions in any of the following ways, depending on your use case:

  • For persistent, real-time endpoints that make one prediction at a time, use SageMaker real-time hosting services. See Real-time inference.
  • Workloads that have idle periods between traffic spurts and can tolerate cold starts, use Serverless Inference. See Serverless Inference.
  • Requests with large payload sizes up to 1GB, long processing times, and near real-time latency requirements, use Amazon SageMaker Asynchronous Inference. See Asynchronous inference.
  • To get predictions for an entire dataset, use SageMaker batch transform. See Use Batch Transform.

You can also look into Bring your own model https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html#deploy-model-steps-byom

Not sure on your set up but if you building a flask application and load the pickled file for the prediction then AWS Compute Optimizer provides Amazon EC2 instance recommendations to help you improve performance, save money, or both. You can use these recommendations to decide whether to move to a new instance type.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-recommendations.html

profile pictureAWS
EXPERT
answered 10 months ago
profile picture
EXPERT
reviewed 10 months ago
profile pictureAWS
EXPERT
iBehr
reviewed 10 months ago
  • Many thanks, that will be new topic to learn. I will deep dive into Sagemaker. I am wondering whether I can connect it to EC2, My site is already public via EC2, can I make Sagemaker serve this site (I want to integrate them. Like easy parts to be done by EC2 server, while only this prediction part should be done by Sagemaker.)

    Mainly my product would let people value their real estate - so anybody can input the characteristics of their property and the model should give the approximate value of the home. I am not sure which service is the best for it Sagemaker or Compute Optimizer, obviously EC2 cannot handle it.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions