Duration of SageMaker caching of multi-model endpoints

Question

SageMaker offers a flag to indicate whether to cache models for a multi-model endpoint, that is enabled by default.
https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_MultiModelConfig.html

For what duration does this get cached?

I tried searching "sagemaker cache duration" and "ModelCacheSetting" but did not find any hits.  The use case is GDPR compliance and endpoint deletion and cleanup.

Answer

Based on our [doc](https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html#multi-model-caching)
```
The cached models are unloaded and/or deleted from disk only when a container runs out of memory or disk space to accommodate a newly targeted model.
```

In other words, if possible, the model will remain in memory and/or disk.

Duration of SageMaker caching of multi-model endpoints

Relevant content