Scoring Regression Models

• R2 - R-Squared - This is a metric that ranges from negative infinity to 1. It is a measurement that explains how much variance in the y value can be determined by x variables. The closer to 1, the more variance in y can be explained by x. Or in other words the closer to 1, the more likely it is that you have feature important variables that would lead to accurate predictions on y because the features explain variance or changes in y.
• RMSE = Root Mean Squared Error = The root mean squared error is the square root of the mean squared error and can be interpreted as the average +/- difference expected between a predicted value and the actual value.
• MSE = Mean Squared Error = The mean squared error is the average squared error between the actual value and the predicted value and can be interpreted as the squared +/- difference that would expected to see on average from a predicted value and the actual value.

Ultimately, the value of these metrics is the ability to see both unitless and unit metrics on how well the model is doing. For example, if the target to predict is contract value, and R2 = .63 RMSE = 1250, and MSE = 15625000:

• R2 is a unitless measure of correlation of the features to the target.
• MSE is the average +/- difference in the units squared or in other words it is saying that the model is off by +/- 1,562,500 $2. Note that it is in dollars squared.
• RMSE is the average +/- difference that the model is predicting from what actually happened. So, it is clearly saying that on average the predicted value is +/- $1,250 from the actual value.