By using AWS re:Post, you agree to the Terms of Use

Hyperparameter tuning for pipeline model

1

We are developing a machine learning algorithm that creates additional features. These features should then be fed into eg a classification algorithm (SVM, XGboost,...). We are using a pipeline to put the two units together, and now want to do hyperparameter tuning for the whole pipeline. Basically what we want to do is:

model = PipelineModel(..., models = [ our_model, xgb_model ])
deploy = Estimator(image_uri = model, ...)
...
tuner = HyperparameterTuner(deply, .... tune_parameters, ....)
tuner.fit(...)

but there are always errors somewhere along the line.

Is there any documentation or example how to achieve something like the above?

Thanks

2 Answers
0

Hello Norbert,

I believe you are referring here the SageMaker pipelines and how that can be incorporated with the Hyper parameter turning. (Please correct me otherwise.)

Following notebook provides and example of how a Hyperparameter Tuning Job can be run as a step in a SageMaker Pipeline.

https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-pipelines/tabular/tuning-step/sagemaker-pipelines-tuning-step.ipynb

answered 7 months ago
  • Hi thanks for your comment and the link! Yes, we want to combine our own feature extraction algorithm (container based) with a standard classification system, plug them together via a sagemaker pipeline, and the do hyperparameter tuning across the whole.

    We will look into the provided sagemaker notebook.

    Thanks again

    Norbert

0

@Shamika-AWS

Again, thanks for your answer, but after study it turns out that you are speaking about "Sagemaker Workflow Pipelines", while I need hyper-parameter tuning for models created via sagemaker.pipeline.ModelPipeline

Do you have any suggestion concerning hyperparameter tuning of modelpipelines?

answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions