As a follow-on to our August 3 release, we've got some more Kraken goodness for you. We have expanded SHAP Importance with a new card for your Prediction data. We have improved the Permutation and SHAP Importance Analysis cards to include chart scales for easier data interpretation as well as more prescriptive help text and documentation. You can now name your Analysis version before you refine your Analysis. And when selecting your Apply dataset, Kraken now reminds you of the details of your training dataset.
SHAP Importance on Predictions
After generating Predictions using algorithms for which SHAP Importance values are created, Kraken will include a SHAP Importance card in the Predictions section of the Project Overview. This card is identical to the SHAP Importance card that is created on your training data, except that the values are aggregated on your prediction data. This card is an aggregation of SHAP values (formerly known as Prediction Influencers) on your prediction data representing how a feature influences the prediction of a single row relative to the other features in that row and to the average outcome in the dataset. When aggregated, SHAP importance provides a general indication of relative influence among the features in the prediction dataset, and includes a dynamic scale to help quickly interpret the data values.
NOTE: SHAP Importance is calculated when your Predictions are generated for algorithms on which Kraken supports SHAP Importance on datasets up to 100,000 rows. If you don't see a SHAP Importance card in the Predictions section, SHAP Importance values are not available for your Predictions.
Feature Importance card group improvements
After running an Analysis, you have access to an improved Feature Importance card group that provides additional insight beyond what the former Driver Influence card provided. The charts are accessible in both thumbnail view on the Project Overview screen as well as full screen view by clicking the appropriate card name:
- Permutation Importance (formerly known as Driver Influence), which indicates how much a feature impacts the prediction. It is a measure of how sensitive the prediction is to changes in the value of that feature. The higher the sensitivity, the greater the impact. The chart now includes a fixed scale to help quickly interpret the data values.
- SHAP Importance, which is an aggregation of SHAP values (formerly known as Prediction Influencers) on your training data represents how a feature influences the prediction of a single row relative to the other features in that row and to the average outcome in the dataset. When aggregated, SHAP importance provides a general indication of relative influence among the features in the training dataset. The chart now includes a dynamic scale to help quickly interpret the range of data values.
REMINDER: SHAP Importance is calculated when you create a new Analysis or refine an existing Analysis. For your current Analyses, SHAP Importance will not be available until you refine/retrain your Analysis. Kraken will visually indicate that you need to retrain your model (by refining your Analysis) so that SHAP Importance can be generated on your new Analysis version.
TIP: After you retrain an existing Analysis to generate SHAP Importance values and then leave that Analysis, the next time you access the Analysis you may not see the SHAP Importance values. Ensure that you select and publish a version of your Analysis for which SHAP Importance values were created (it will be the most recent version if you only retrained once) via the DEPLOY tab from the Project overview screen.
Name your Analysis version before refining
Kraken now provides you with the ability to name your Analysis version before you refine your Analysis. No longer do you have to navigate to the DEPLOY tab to name your version, so you can quickly give your Analysis version a name that indicates why this version exists, such as "Removed ID field" or "renamed Territory to State". Of course, you can still access and edit Analysis version names from the DEPLOY tab at any time.
TIP: You can hover over the Analysis version number on the Project Overview screen to see the name of your existing Analysis version.
Reminder of your training data when selecting your apply dataset
Kraken now reminds you of the target column, training dataset and provider when selecting your apply dataset to generate Predictions. This can be helpful if you've built several Analyses and may have forgotten which training data was used when you go to choose your Apply dataset.
We've also implemented several bug fixes and performance improvements.
Reminder: If Kraken is misbehaving, check KIKI first.
All Kraken users have access to KIKI, a.k.a. "Kraken Important Known Issues". If you're experiencing an issue with Kraken, KIKI is your first stop to determine if we're aware of the issue. Where possible, we provide suggested workarounds while we work to eliminate issues. KIKI will be updated as we eliminate issues through Kraken product updates, as well as when new issues are identified.
Reminder: SONAR© Guide content is just a click away!
As a Kraken user, you have access to SONAR© Guide, the proven training methodology from Big Squid and your key to learning everything you need to know to use Kraken effectively and become a citizen data scientist hero in the process. A link to the SONAR© Guide content library is accessible whenever you access Kraken help screens; just look for the SONAR© Guide logo on the right-hand side. It's also available in the SONAR© Guide on the Big Squid Support site.