2개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
1
sagemaker.processing.Processor is the base class which is extended by ScriptProcessor, SKLearnProcessor etc... Its not recommended to directly use the base Processsor class. However you can still have a custom docker image and Run it using the base Processor module.
답변함 2년 전
0
Thank you, I understand the Processor is a base class. Then, find out how to provide my script to Processor via the entrypoint and run it.
container_base_path = "/opt/ml/processing"
processor = Processor(
role=os.environ['ROLE'],
image_uri=image_url,
instance_count=1,
instance_type='ml.m4.xlarge',
entrypoint=["python", f"{container_base_path}/input/process-data.py"]
)
processor.run(
job_name=f'processor-{strftime("%Y-%m-%d-%H-%M-%S")}',
inputs=[
ProcessingInput(
source=data_input_path,
destination=f"{container_base_path}/data/"
),
ProcessingInput(
source=code_input_path,
destination=f"{container_base_path}/input/"
)
],
...
I feel that SageMaker SDK is not consistent for developers because each processor has a different way to pass my code into it.
- Processor: entrypoint
- ScriptProcessor: command
- SklearnProcessor='my_script.py' in the run method
답변함 2년 전
+1: Since base Processor doesn't support passing in scripts at run-time, the main use case for it would be where you have a container image which already bakes your script in.