On Variable Importance in Linear Regression

The paper examines in detail one particular measure of variable importance for linear regression that was theoretically justified by Pratt (1987), but which has since been criticized by Bring (1996) for producing “counterintuitive” results in certain situations, and by other authors for failing to guarantee that importance be non-negative. In the article, the “counterintuitive” result is explored and shown to be a defensible characteristic of an importance measure. It is also shown that negative importance of large magnitude can only occur in the presence of multicollinearity of the explanatory variables, and methods for applying Pratt's measure in such cases are described. The objective of the article is to explain and to clarify the characteristics of Pratt's measure, and thus to assist practitioners who have to choose from among the many methods available for assessing variable importance in linear regression.