SageMaker framework processor compatibility with sagemaker pipelines

0

Hi all,

I am asking if it's possible to use framework processor inside a sagemaker pipeline.

I am asking because the to submit the source_dir for the framework processor, we have to do so when calling the .run() method, when wrapping the processor inside a sagemaker.workflow.steps.ProcessingStep, there isn't an available argument to specify the source_dir.

Thank you! Best, Ruoy

asked 2 years ago791 views
1 Answer
0
Accepted Answer

You can do this with the latest version of the sagemaker sdk 2.89.0

from sagemaker.workflow.pipeline_context import PipelineSession

session = PipelineSession()

inputs = [
    ProcessingInput(
    source="s3://my-bucket/sourcefile", 
    destination="/opt/ml/processing/inputs/",),
]

processor = FrameworkProcessor(...)

step_args = processor.run(inputs=inputs, source_dir="...")

step_sklearn = ProcessingStep(
    name="MyProcessingStep",
    step_args=step_args,
)
AWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions