- Mais recentes
- Mais votos
- Mais comentários
I haven't got an up-to-date example, but tentatively believe this should be possible without having to build a custom container?
Generally in PyTorch framework, if your model.tar.gz
contains code/inference.py
(root of the tarball contains a code subfolder with an inference.py script), this should get picked up... So the approach would be to pack your inference scripts into your model tarballs.
Specifically with MME, I haven't tried on the most recent framework versions but last time I tried it out, you needed to also use TorchServe model archiver to package the model ready.
This sample creates a MultiDataModel in PyTorch with inference scripts, but is currently pinned at framework version <1.8.1
because of this issue. Hopefully it could still help you get started?
Refer to the following link to build your own Multi-Model Endpoint Container: https://docs.aws.amazon.com/sagemaker/latest/dg/build-multi-model-build-container.html
Conteúdo relevante
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há um ano
Error:
AttributeError Traceback (most recent call last) /tmp/ipykernel_8642/1469107258.py in <cell line: 47>() 45 46 print(type(mme)) ---> 47 predictor = mme.deploy(initial_instance_count=1, 48 instance_type='ml.m5.2xlarge', 49 endpoint_name=f'mme-pytorch-{current_time}')
~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/multidatamodel.py in deploy(self, initial_instance_count, instance_type, serializer, deserializer, accelerator_type, endpoint_name, tags, kms_key, wait, data_capture_config, **kwargs) 240 self.sagemaker_session = local.LocalSession() 241 --> 242 container_def = self.prepare_container_def(instance_type, accelerator_type=accelerator_type) 243 self.sagemaker_session.create_model( 244 self.name,
~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/multidatamodel.py in prepare_container_def(self, instance_type, accelerator_type, serverless_inference_config) 138 # copied over 139 if self.model: --> 140 container_definition = self.model.prepare_container_def(instance_type, accelerator_type) 141 image_uri = container_definition["Image"] 142 environment = container_definition["Environment"]
~/anaconda3/envs/pytorch_p38/lib/python3.8/site
~/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/sagemaker/pytorch/model.py in prepare_container_def(self, instance_type, accelerator_type, serverless_inference_config) 287 ) 288 --> 289 region_name = self.sagemaker_session.boto_session.region_name 290 deploy_image = self.serving_image_uri( 291 region_name,
AttributeError: 'NoneType' object has no attribute 'boto_session'
Hey @Alex_T I just tried as per your instruction but getting following error, so can you please tell me how we can fix this error? My script: