What is “overfitting”? Why is it generally better to use a smaller number of Drivers in my model?

“Overfitting” means that a model is overly complex and as a result, is unreliable for predicting new data. Overfitting tends to happen when there are too many Drivers relative to the number of data points available. For example: you may only have 50 rows of data and 100 Drivers (columns) in the dataset.

The predictive model can use all of the Drivers to come up with a series of complicated rules that perform well against the data used to train the mode, when in reality the predicted Metric may be influenced by only one or two predictors.

As a rule of thumb, simpler is better. The more Drivers introduced to a model, the more error exists that can potentially cover up the true underlying relationship that you want to uncover.