how to use custom docker images from private repo for training in sagemaker pipelines?

0

I'm using sagemaker pipeline to create processing and training steps, which works. next, for my training step, i wanted to use my own image , hosted in our own private container registry. I provide subnets, vpc config, but the error message i get is => 'clienterror : Failed to invoke sagemaker: CreateTrainingJob. Error Details: TrainingImageConfig with TrainingRepositoryAccessMode set to VPC must be provided when using a training image from a private docker registry..."

when looking at the sagemaker pipeline documentation , i don't see this configuration in the docs. how can i provide this , when using sagamaker pipeline. ???

pipeline_project
      src
          processing.py
          train.py
      utils
           helper.py
from sagemaker.processing import ScriptProcessor, ProcessingInput, ProcessingOutput

script_processor = ScriptProcessor(command=['python3'],
                image_uri='image_uri',
                role='role_arn',
                instance_count=1,
                instance_type='ml.m5.xlarge', 
)

step_process = ProcessingStep(
        name="ProcessStep",
        processor=script_processor, 
        code = 'src/processing.py'
        input = [ 
                ProcessingInput(
                      input_name="utils"
                      source="src/utils", 
                      destination="/opt/ml/processing/input/src/utils",
                )
)

security_group = ["somegroupid"]
my_subnets = ["subnet-1", subnet-2"]
network_config = NetworkConfig(enable_network_isolation=False, 
                               security_group_ids = security_group , 
                               subnets= my_subnets)

from sagemaker.estimator import Estimator
#training step 
estimate = Estimator (
  entry_point="src/train.py", 
  ....
  network_config= network_config, 
  ....
  image_uri='private_docker_registry/some_image', 
  ...
 )
asked 7 months ago431 views
1 Answer
0

Please refer the below document to use private registry with SageMaker training - https://docs.aws.amazon.com/sagemaker/latest/dg/docker-containers-adapt-your-own-private-registry.html

AWS
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions