Load predefined SageMaker pipelines object to execute it.

Question

I have a notebook in which I create a SageMaker Pipeline:

```
from sagemaker.workflow.pipeline import Pipeline

pipeline = Pipeline(
    name = 'my-pipeline-name',
    parameters=pipeline_parameters,
    steps=[step_preprocess, step_lambda_compress_preprocessor_output, step_train, step_model_create, step_lambda_create_endpoint],
    pipeline_experiment_config = None
)

pipeline.upsert(role_arn=sagemaker_role)
```

I want to now load this pipeline object, **in another notebook/piece of API**, and execute it. Is this possible? The documentation https://sagemaker.readthedocs.io/en/v2.199.0/workflows/pipelines/sagemaker.workflow.pipelines.html#pipeline tells me I can create a new Pipeline object, but I cannot find how to load an already predefined one.

joe · Answer

The sagemaker API docs are useless on this point. I had to dig into github to find an example.  It's as simple as using your token to login to you account and use the name of the existing pipeline:

```
 pipeline = Pipeline(name='existing-pipeline-name')
```

rePost-User-9561832 · Answer

Hi,

Thanks for using AWS Sagemaker.

I understand that you would like to know how you can load predefined SageMaker pipelines object to execute it.

When you write pipeline object, are you talking about the python object within the notebook or the SageMaker Pipeline entity? For the second one, the SageMaker entity, you can find the pipeline in the Studio panel, or use list-pipelines with CLI or Boto3. For the python entity you can export the definition (pipeline.definition()) and write it to a json file.

To further understand the issue more in depth as I have limited visibility on your setup, I'd recommend you to reach to AWS Support by creating a support case[+] so that the engineer can investigate further and help you overcome the issue.

Reference:

-————-

[+] Open a support case with AWS using the link: https://console.aws.amazon.com/support/home?#/case/create

Load predefined SageMaker pipelines object to execute it.

Add your answer

Relevant content