How to use sagemaker.processing.Processor run method

0

This is sagemaker docs. What is the purpose of sagemaker.processing.Processor as its run method does not have input for code or script? then how can I use it?

Of course, I can use FrameworkProcessor, ScriptProcessor, SklearnProcessor because I can provide my processing.py. But for the sagemaker.processing.Processor, how can I use it?

hai
asked 2 years ago299 views
2 Answers
1

sagemaker.processing.Processor is the base class which is extended by ScriptProcessor, SKLearnProcessor etc... Its not recommended to directly use the base Processsor class. However you can still have a custom docker image and Run it using the base Processor module.

AWS
answered 2 years ago
AWS
EXPERT
Alex_T
reviewed 2 years ago
  • +1: Since base Processor doesn't support passing in scripts at run-time, the main use case for it would be where you have a container image which already bakes your script in.

0

Thank you, I understand the Processor is a base class. Then, find out how to provide my script to Processor via the entrypoint and run it.

container_base_path = "/opt/ml/processing"

processor = Processor(
    role=os.environ['ROLE'],
    image_uri=image_url,
    instance_count=1,
    instance_type='ml.m4.xlarge',
    entrypoint=["python", f"{container_base_path}/input/process-data.py"]
)

processor.run(
    job_name=f'processor-{strftime("%Y-%m-%d-%H-%M-%S")}',
    inputs=[
        ProcessingInput(
            source=data_input_path,
            destination=f"{container_base_path}/data/"
        ),
        ProcessingInput(
            source=code_input_path,
            destination=f"{container_base_path}/input/"
        )
    ],
...

I feel that SageMaker SDK is not consistent for developers because each processor has a different way to pass my code into it.

  • Processor: entrypoint
  • ScriptProcessor: command
  • SklearnProcessor='my_script.py' in the run method
hai
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions