Deploy PipelineModel combining two "byom" PyTorchModels in Sagemaker Studio

0

I have two already trained ("bring your own model") pytorch models, which I want to run sequentially in a PipelineModel deployed as one endpoint. Each individual model can be loaded and deployed running something like:

timestamp_prefix = strftime("%Y-%m-%d-%H-%M-%S", gmtime())
endpoint_name = "model1-ep" + timestamp_prefix

model1 = PyTorchModel(
    model_data=model1_path,
    role=role,
    entry_point='./model1/code/detect_api.py',
    source_dir='./model1/code',
    image_uri="763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.13-cpu-py39",
    py_version='py3'
)

predictor = model1.deploy(instance_type='ml.c5.2xlarge', initial_instance_count=1, endpoint_name=endpoint_name)

(and the same for model2). Then the pipeline can be deployed running:

timestamp_prefix = strftime("%Y-%m-%d-%H-%M-%S", gmtime())
endpoint_name_pipeline = "pipeline-ep-" + timestamp_prefix

pipeline_model = PipelineModel(models=[model1, model2], role=role)
pipeline_model.deploy(initial_instance_count=1, instance_type="ml.g4dn.2xlarge", endpoint_name=endpoint_name_pipeline)

This is working. My question is: is there a way to deploy the PipelineModel without (re-)deploying the individual models each time? If I try to deploy the pipeline without running the "model1.deploy" and "model2.deploy" before this, I get an error:

AttributeError: 'NoneType' object has no attribute 'config' (based on the trace Sagenmaker is searching for self.sagemaker_session.config and specifically self.pipeline_container_def(instance_type), so where can I define these?).

A second question (if this is allowed) is: how should I choose the instance type for the pipeline instance (after having optimized the instance types for the individual models)?

asked 4 months ago84 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions