Lemuel Shattuck,Statist Founder of the American Statistical Association

Polynomial regression of degree p in one independent variable χ is considered. Numerically large sample correlations between χ^α and χ^β, α < β, a, β = 1, ···, p, may cause ill-conditioning in the matrix to be inverted in application of the method of least squares. These sample correlations are investigated. It is confirmed that centering of the independent variable to have zero sample mean removes nonessential ill-conditioning. If the sample values of χ are placed symmetrically about their mean, the sample correlation between χ^α and χ^β is reduced to zero by centering when α + β is odd, but may remain large when α + β is even. Some examples and recommendations are given.     

A Tale of Two Matrix Factorizations

In statistical practice, rectangular tables of numeric data are commonplace, and are often analyzed using dimension-reduction methods like the singular value decomposition and its close cousin, principal component analysis (PCA). This analysis produces score and loading matrices representing the rows and the columns of the original table and these matrices may be used for both prediction purposes and to gain structural understanding of the data. In some tables, the data entries are necessarily nonnegative (apart, perhaps, from some small random noise), and so the matrix factors meant to represent them should arguably also contain only nonnegative elements. This thinking, and the desire for parsimony, underlies such techniques as rotating factors in a search for “simple structure.” These attempts to transform score or loading matrices of mixed sign into nonnegative, parsimonious forms are, however, indirect and at best imperfect. The recent development of nonnegative matrix factorization, or NMF, is an attractive alternative. Rather than attempt to transform a loading or score matrix of mixed signs into one with only nonnegative elements, it directly seeks matrix factors containing only nonnegative elements. The resulting factorization often leads to substantial improvements in interpretability of the factors. We illustrate this potential by synthetic examples and a real dataset. The question of exactly when NMF is effective is not fully resolved, but some indicators of its domain of success are given. It is pointed out that the NMF factors can be used in much the same way as those coming from PCA for such tasks as ordination, clustering, and prediction. Supplementary materials for this article are available online.