SageMaker framework processor compatibility with sagemaker pipelines

Question

Hi all,

I am asking if it's possible to use `framework processor` inside a `sagemaker pipeline`.

I am asking because the to submit the source_dir for the framework processor, we have to do so when calling the .run() method, when wrapping the processor inside a `sagemaker.workflow.steps.ProcessingStep`, there isn't an available argument to specify the `source_dir`.

Thank you!
Best,
Ruoy

Answer

You can do this with the latest version of the sagemaker sdk 2.89.0

```
from sagemaker.workflow.pipeline_context import PipelineSession

session = PipelineSession()

inputs = [
    ProcessingInput(
    source="s3://my-bucket/sourcefile", 
    destination="/opt/ml/processing/inputs/",),
]

processor = FrameworkProcessor(...)

step_args = processor.run(inputs=inputs, source_dir="...")

step_sklearn = ProcessingStep(
    name="MyProcessingStep",
    step_args=step_args,
)
```

SageMaker framework processor compatibility with sagemaker pipelines

Relevant questions

Is it possible to test locally SageMaker Inference Pipelines?

Without ECR being enabled in AWS account at Organization Level, what's the impact to SageMaker Studio?

How to configure a serverless sagemaker endpoint?

SageMaker framework processor compatibility with sagemaker pipelines

Can Sagemaker-trained models be deployed to non-Sagemaker endpoints?

Specify a subnet for SageMaker endpoints

Using R model in SageMaker ML pipelines

Auto rollback with Guardrails if model accuracy is not good

[Feature Request] Serverless Inference with VPC Config

Unable to create endpoint