I'm using sagemaker pipeline to create processing and training steps, which works. next, for my training step, i wanted to use my own image , hosted in our own private container registry. I provide subnets, vpc config, but the error message i get is =>
'clienterror : Failed to invoke sagemaker: CreateTrainingJob. Error Details: TrainingImageConfig with TrainingRepositoryAccessMode set to VPC must be provided when using a training image from a private docker registry..."
when looking at the sagemaker pipeline documentation , i don't see this configuration in the docs. how can i provide this , when using sagamaker pipeline. ???
pipeline_project
src
processing.py
train.py
utils
helper.py
from sagemaker.processing import ScriptProcessor, ProcessingInput, ProcessingOutput
script_processor = ScriptProcessor(command=['python3'],
image_uri='image_uri',
role='role_arn',
instance_count=1,
instance_type='ml.m5.xlarge',
)
step_process = ProcessingStep(
name="ProcessStep",
processor=script_processor,
code = 'src/processing.py'
input = [
ProcessingInput(
input_name="utils"
source="src/utils",
destination="/opt/ml/processing/input/src/utils",
)
)
security_group = ["somegroupid"]
my_subnets = ["subnet-1", subnet-2"]
network_config = NetworkConfig(enable_network_isolation=False,
security_group_ids = security_group ,
subnets= my_subnets)
from sagemaker.estimator import Estimator
#training step
estimate = Estimator (
entry_point="src/train.py",
....
network_config= network_config,
....
image_uri='private_docker_registry/some_image',
...
)