Hyperparameter tuning for pipeline model

Question

We are developing a machine learning algorithm that creates additional features. These features should then be fed into eg a classification algorithm (SVM, XGboost,...). We are using a pipeline to put the two units together, and now want to do hyperparameter tuning for the whole pipeline. Basically what we want to do is:

```
model = PipelineModel(..., models = [ our_model, xgb_model ])
deploy = Estimator(image_uri = model, ...)
...
tuner = HyperparameterTuner(deply, .... tune_parameters, ....)
tuner.fit(...)
```
but there are always errors somewhere along the line.

Is there any documentation or example how to achieve something like the above?

Thanks

Answer

Hello Norbert,

I believe you are referring here the SageMaker pipelines and how that can be incorporated with the Hyper parameter turning. (Please correct me otherwise.)

Following notebook provides and example of how a Hyperparameter Tuning Job can be run as a step in a SageMaker Pipeline.

https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-pipelines/tabular/tuning-step/sagemaker-pipelines-tuning-step.ipynb

Answer

@Shamika-AWS

Again, thanks for your answer, but after study it turns out that you are speaking about "Sagemaker Workflow Pipelines", while I need hyper-parameter tuning for models created via sagemaker.pipeline.ModelPipeline

Do you have any suggestion concerning hyperparameter tuning of modelpipelines?

Hyperparameter tuning for pipeline model

Relevant content