model_fn called multiple times(1 per GPU) during deployment.

I'm trying to deploy a quantized llama 3.1 70b model, wrote a custom model_fn to read it from disk, but after several errors and investigation I noticed that the loading function model_fn is called multiple times, 1 per GPU(I tried 2 instance sizes and always the number of calls is the same as the number of GPUs on the instance), so the first load works successfully but after the first one out of memory error occurs(because the model is attempted to be loaded to an already occupied GPU), I tried adding flags to determine if the model is already loading(and avoided loading it again) without success. Is this a bug? I was expecting model_fn to run just 1 time to load the model during deploy.