1 Answer
- Newest
- Most votes
- Most comments
0
If you're using a modest instance type to host multiple models, perhaps you might be struggling with CPU/memory resource contention as mentioned here in the docs? SageMaker recently launched configurable health-check time-outs, which may be interesting to you if your models or containers are particularly large / slow to set up (see the CreateEndpointConfig API)... But I'm not certain whether this configuration is supported for Multi-Container Endpoints at the moment.
Relevant content
asked 3 years ago
