In Kraken, Feature Importance is comprised of two distinct visualizations: Permutation Importance and SHAP Importance.
Feature importance is measured a little bit differently depending on the model type used, but is surfacing a rankable measure of how much impact each variable has on the target variable. The simplistic interpretation of feature importance is that changing the most feature important variable will change the target variable more than by changing any other variable. Changing two of the most feature important variables will likely have a greater impact than only changing one, but the principle is that something with very low feature importance probably doesn’t have very much predictive power and that controlling or changing it might not make a difference.
Feature importance can be very helpful in identifying problems with the data being used to train the model. For example, if I was trying to predict a grade number for students and I forgot to remove the column that contains the letter grade that the student achieved, that would probably be the most predictive and most feature important column, causing the model to perform better than it would in real life, because if I am trying to predict the numeric grade score, it means that I probably don’t know the letter grade yet.
Feature importance can also be helpful in finding the next steps in making a model iteratively better. The most feature important values can sometimes be good things to segment on or sometimes there are more features that can be captured or engineered to better represent what the more feature important variables are trying to depict without adding a lot of redundancy. As an example of segmenting, maybe an autopay flag is very feature important, two models might be able to do a better job if one was trained only on customers that were set up for autopay and the other one was only trained for customers that are not set up on autopay. In an example of engineering, maybe a very feature important variable is the product family that the business is producing. Perhaps breaking that out into a few more descriptive features about the products might be more significant.