Azure Repos to Sagemaker pipeline integration

0

Hi, we use Azure Repos as our Version Control System. We would like to train ML models on our local env . Whenever ML code is pushed, we want to use Azure DevOps pipelines to integrate with SageMaker pipelines. We have couple of questions?

  1. Can we track the experimentation done on local env using SageMaker API? If yes, can you share any relevant articles as I have come across mostly outdated code snippets online for that purpose.
  2. What is the recommended method to integrate Azure Repos to SageMaker pipelines through Azure DevOps? As code gets merged, Azure DevOps pipeline gets triggered then the SageMaker pipeline for training and deployment gets executed. That's the intended approach. Appreciate any articles or references.

Thanks

1 Answer
1

Yes, you can track experimentation done on a local environment using the SageMaker API. SageMaker provides a number of APIs that you can use to create, track, and manage machine learning experiments. You can use the SageMaker Python SDK to create and manage experiments in your local environment, and then track those experiments in SageMaker using the Experiment and Trial objects.

Here are some resources that can help you get started with tracking experiments in SageMaker:

SageMaker Python SDK documentation: https://sagemaker.readthedocs.io/en/stable/

SageMaker Experiments documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/experiments.html

Tutorial on using SageMaker Experiments with the SageMaker Python SDK: https://aws.amazon.com/blogs/machine-learning/streamline-machine-learning-experimentation-and-tracking-using-amazon-sagemaker-experiments-and-the-python-sdk/

To integrate Azure Repos with SageMaker pipelines through Azure DevOps, you can use Azure Pipelines. Azure Pipelines is a continuous integration and continuous deployment (CI/CD) service that can be used to automate the building, testing, and deployment of applications. You can use Azure Pipelines to set up a pipeline that triggers when code is pushed to Azure Repos, and then use that pipeline to execute the SageMaker pipeline for training and deployment.

Here are some resources that can help you get started with setting up an Azure Pipelines pipeline to integrate Azure Repos with SageMaker pipelines:

Azure Pipelines documentation: https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops

Tutorial on using Azure Pipelines to deploy a machine learning model to SageMaker: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-and-where#azure-pipelines

I hope this helps

AWS_Guy
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions