Tips for managing several ML project with similar framework

0

Hi,

I'm working on an end-to-end ml project which, for the moment, goes from training (it takes already processed train/val/test data from an S3 bucket) to deploy, passing through hyperparameter tunning. This project has been developed on SageMaker Studio and in the beginning, I decided to keep track on the project with a github's repository.

So, my work has certain degree of maturity: I was able to successfully train, tune, deploy and infer over a dataset. But the troubles came when I try to replicate this work for another dataset (a new project with a similar framework).

Let's say that I had a client "A" for which I developed this project and now I have a new client "B" with similar requirements than client A. I'm looking for the best way of "copy & paste" the project considering the following:

a) I would like to keep working on the repository. The idea here is the repo been like a project's template ir order to clone the repository, make some few corrections (changing model's name, working bucket, etc) and then execute tuning, evaluation and deploy. b) There's a lot of changes and improvements that I should make in the future. So, I'd like those changes been reflected on both projects.

If anyone could give me some tips, guidelines, share his experience with something like this I would be very grateful.

Regards! :D

1 Answer
1

SageMaker Projects gives you the ability to quickly setup and standardize your environment for repeatability. Using SageMaker projects, you can define templates that bootstrap version control (Github in your case), automated ML pipelines, and other codes to quickly iterate over using different data sets. SageMaker projects are provisioned using AWS Service Catalog products. Please refer this blog for more insights : https://aws.amazon.com/blogs/machine-learning/building-automating-managing-and-scaling-ml-workflows-using-amazon-sagemaker-pipelines/

AWS
marancs
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions