How to use sagemaker.processing.Processor run method

Question

This is sagemaker [docs](shttps://sagemaker.readthedocs.io/en/stable/api/training/processing.html). What is the purpose of sagemaker.processing.Processor as its run method does not have input for code or script? then how can I use it?

Of course, I can use FrameworkProcessor, ScriptProcessor, SklearnProcessor because I can provide my processing.py. But for the sagemaker.processing.Processor, how can I use it?

Answer

sagemaker.processing.Processor is the base class which is extended by ScriptProcessor, SKLearnProcessor etc... Its not recommended to directly use the base Processsor class. However you can still have a custom docker image and Run it using the base Processor module.

Answer

Thank you, I understand the Processor is a base class. Then, find out how to provide my script to Processor via the entrypoint and run it.

```
container_base_path = "/opt/ml/processing"

processor = Processor(
    role=os.environ['ROLE'],
    image_uri=image_url,
    instance_count=1,
    instance_type='ml.m4.xlarge',
    entrypoint=["python", f"{container_base_path}/input/process-data.py"]
)

processor.run(
    job_name=f'processor-{strftime("%Y-%m-%d-%H-%M-%S")}',
    inputs=[
        ProcessingInput(
            source=data_input_path,
            destination=f"{container_base_path}/data/"
        ),
        ProcessingInput(
            source=code_input_path,
            destination=f"{container_base_path}/input/"
        )
    ],
...
```

I feel that SageMaker SDK is not consistent for developers because each processor has a different way to pass my code into it. 
* Processor: entrypoint
* ScriptProcessor: command
* SklearnProcessor='my_script.py' in the run method

How to use sagemaker.processing.Processor run method

相关内容