With a single click, automatic Hyperparameter Optimization ensures that your model is precisely tuned for optimal predictive results.
A hyperparameter is a parameter whose value is used to control the learning process. Machine learning models require different constraints, weights or learning rates to generalize different data patterns. These measures are called hyperparameters and must be tuned so that the model can optimally solve the machine learning problem.
By default, Kraken uses a predefined set of hyperparameter values for each algorithm used in model training. These values are the standard values generally accepted by the data science community.
However, in some cases, good may not be good enough... that's where Kraken's automatic Hyperparameter Optimization (HPO) comes in. If you have HPO enabled as part of your Kraken subscription, it is accessible when you click the pink "+" sign in the upper right of the Data Pipeline.
Important Notes about Hyperparameter Optimization (HPO)
Don't use HPO the first time you train a model. HPO is designed to be used AFTER you have trained your model and are content with the results; often that process requires repeated refinement and retraining. Once that process is complete, that's when HPO should be used.
HPO adds considerable time to the model training process. Kraken runs up to 100 models per algorithm while analyzing your data. If the training process for your model takes five minutes using the standard predefined hyperparameter values, training that same model with HPO enabled could take hours.
HPO is limited to specific algorithms and model types. HPO is specifically designed to work with these model types using the following algorithms:
- Binary Classification models
- Regression models
Kraken automatically chooses the best combination of hyperparameter values. Once model training with HPO enabled is complete you can review the training results by clicking the "Model Metrics" button on the Analysis Overview page. Kraken will display the total number of models trained at the top of the page, and will indicate with a green checkmark which algorithm has been automatically selected as performing the best. Clicking on the blank space to the right of the algorithm name - or the arrow icon on the right of the screen - will expand the results to show you various combinations of hyperparameter values that Kraken analyzed, sorted in descending order by model score.
You can override the algorithm that Kraken has automatically selected. If you prefer a different algorithm than Kraken has selected, click the pink "SELECT" text. Note that Kraken will automatically select the top-performing combination of hyperparameter values for your chosen algorithm.
Use enhanced Model Metrics to dig into the details. You can click the underlined name of any top performing model to dive into the Model Metrics screen, which will display data science metrics for your model, along with a list of the specific hyperparameter values used for this particular model training. For binary classification models, the Model Metrics screen will also include the Automatic Threshold Tuning value used for this model.
HPO is not available in all Kraken subscriptions. You may not have access to HPO, depending on your company's subscription plan. If you have questions or would like to discuss getting HPO enabled, contact your friendly neighborhood Big Squid Customer Success Representative.