SageMaker Pipelines and CI/CD with GitLab Multiaccount

0

Hi,

I have a scenario where I am building different models for multiple different use cases. For each use case I have a SageMaker Pipeline for it ( with preprocessing, training ....deployment etc). Thus I have multiple independent pipelines.

Note : Using workflows and steps

How to do Sagemaker pipelines CI/CD across multiple account using GitLab?

Limitations/ Things which I cannot use/allow due to compliance/policy :

  • Cannot use MLOps template directly given by AWS : As sagemaker starts creating repo.
  • dev, test and prod all are in 3 different account
  • cannot use aws CodeCommit, CodePipeline ( can only use GitLab)

Note : Few use case requires continuous training and batch transforms.

Any suggestions would be appreciated.

Thanks in advance.

2 Answers
0

The main steps to deploy models in several accounts would be --> Create a model --> register model in model registry creating a model group then attach two policies, one for the S3 bucket containing the model.tar.gz and the second policy for the model group. In the target account you create a model pointing to the above model package group. And finally you simply deploy the model with the model you have just created in the target account. If you want to see the process in detail please refer to the blog below:

https://aws.amazon.com/blogs/machine-learning/multi-account-model-deployment-with-amazon-sagemaker-pipelines/

AWS
EXPERT
answered 2 years ago
0

I'm assuming you want to run all the SageMaker Pipelines in one account and deploy the models to a different account. In that case you need to do a few things:

  1. Allow the target accounts to read the models in the S3 bucket of the main account;
  2. Attach a policy to the ModelPackageGroup (SageMaker Pipelines) that allows the target accounts to read it;
  3. Create an EventBridge rule that detects events (Approval) in the ModelPackageGroup and invoke GItLab to start the CI/CD process that will:
  • Create a new Model in the SageMaker Models Catalog of the target accounts that points to the Version you just registered in the ModelPackageGroup of the main account
  • Finally deploy the model to the target account by using the Model you just created in each target account.

If you need to run each SageMaker pipeline in a different account, I would just use GitLab to, kick-off the pipelines using SageMaker Python Library or Boto3.

Regarding the template, you can also create a custom SageMaker Project template the way you need, instead of using the MLOps templates: https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-projects-templates-custom.html

In this blog post you have an example of how to create the policies for S3 and ModelGroup. You can this blog as an inspiration: https://aws.amazon.com/blogs/machine-learning/multi-account-model-deployment-with-amazon-sagemaker-pipelines/

AWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions