A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

Another Cautionary Note about R 2: Its Use in Weighted Least-Squares Regression Analysis

A recent article in this journal presented a variety of expressions for the coefficient of determination (R ²) and demonstrated that these expressions were generally not equivalent. The article discussed potential pitfalls in interpreting the R ² statistic in ordinary least-squares regression analysis. The current article extends this discussion to the case in which regression models are fit by weighted least squares and points out an additional pitfall that awaits the unwary data analyst. We show that unthinking reliance on the R ² statistic can lead to an overly optimistic interpretation of the proportion of variance accounted for in the regression. We propose a modification of the estimator and demonstrate its utility by example.     

Estimators of the multiple correlation coefficient: Local robustness and confidence intervals

Many robust regression estimators are defined by minimizing a measure of spread of the residuals. An accompanying R ²-measure, or multiple correlation coefficient, is then easily obtained. In this paper, local robustness properties of these robust R ²-coefficients are investigated. It is also shown how confidence intervals for the population multiple correlation coefficient can be constructed in the case of multivariate normality.     

The Prediction Sum of Squares as a General Measure for Regression Diagnostics

Statistics that usually accompany the regression model do not provide insight into the quality of the data or the potential influence of the individual observations on the estimates. In this study, the Q² statistic is used as a criterion for detecting influential observations or outliers. The statistic is derived from the jackknifed residuals, the squared sum of which is generally known as the prediction sum of squares or PRESS. This article compares R ² with Q² and suggests that the latter be used as part of the data-quality check. It is shown, for two separate data sets obtained from regional cost of living and U.S. food industry studies, that in the presence of outliers the Q² statistic can be negative, because it is sensitive to the choice of regressors and the inclusion of influential observations. Once the outliers are dropped from the sample, the discrepancy between Q² and R ² values is negligible.     

Frequency of Selecting Noise Variables in Subset Regression Analysis: A Simulation Study

This article presents the results of a simulation study of variable selection in a multiple regression context that evaluates the frequency of selecting noise variables and the bias of the adjusted R ² of the selected variables when some of the candidate variables are authentic. It is demonstrated that for most samples a large percentage of the selected variables is noise, particularly when the number of candidate variables is large relative to the number of observations. The adjusted R ² of the selected variables is highly inflated.     

Comparison of Goodness-of-Fit Measures in Probit Regression Model

This article examines several goodness-of-fit measures in the binary probit regression model. Existing pseudo-R ² measures are reviewed, two modified and one new pseudo-R ² measure are proposed. For the probit regression model, empirical comparisons are made for different goodness-of-fit measures with the squared sample correlation coefficient of the observed response and the predicted probabilities. As an illustration, the goodness-of-fit measures are applied to a “paid labor force” data set.     

Variability explained by covariates in linear mixed‐effect models for longitudinal data

Variability explained by covariates or explained variance is a well‐known concept in assessing the importance of covariates for dependent outcomes. In this paper we study R² statistics of explained variance pertinent to longitudinal data under linear mixed‐effect models, where the R² statistics are computed at two different levels to measure, respectively, within‐ and between‐subject variabilities explained by the covariates. By deriving the limits of R² statistics, we find that the interpretation of explained variance for the existing R² statistics is clear only in the case where the covariance matrix of the outcome vector is compound symmetric. Two new R² statistics are proposed to address the effect of time‐dependent covariate means. In the general case where the outcome covariance matrix is not compound symmetric, we introduce the concept of compound symmetry projection and use it to define level‐one and level‐two R² statistics. Numerical results are provided to support the theoretical findings and demonstrate the performance of the R² statistics. The Canadian Journal of Statistics 38: 352–368; 2010 © 2010 Statistical Society of Canada     

Diagnosis of Multivariate Control Chart Signal Based on Dummy Variable Regression Technique

Abstract

It is common to monitor several correlated quality characteristics using the Hotelling's T ² statistic. However, T ² confounds the location shift with scale shift and consequently it is often difficult to determine the factors responsible for out of control signal in terms of the process mean vector and/or process covariance matrix. In this paper, we propose a diagnostic procedure called ‘D-technique’ to detect the nature of shift. For this purpose, two sets of regression equations, each consisting of regression of a variable on the remaining variables, are used to characterize the ‘structure’ of the ‘in control’ process and that of ‘current’ process. To determine the sources responsible for an out of control state, it is shown that it is enough to compare these two structures using the dummy variable multiple regression equation. The proposed method is operationally simpler and computationally advantageous over existing diagnostic tools. The technique is illustrated with various examples.     

THE MULTIPLE CORRELATION COEFFICIENT AND FISHER'S A STATISTIC1

Fisher's A statistic, often called the adjusted R² statistic, is shown to be a close approximation to the maximum likelihood estimate of the multiple correlation coefficient, p², based on the marginal distribution of R². Expansions for the estimate are obtained. The same methods lead to maximum marginal likelihood estimators for the noncentrality parameters for noncentral X² and F.     

The estimation of R 2 and adjusted R 2 in incomplete data sets using multiple imputation

The coefficient of determination, known also as the R ², is a common measure in regression analysis. Many scientists use the R ² and the adjusted R ² on a regular basis. In most cases, the researchers treat the coefficient of determination as an index of ‘usefulness’ or ‘goodness of fit,’ and in some cases, they even treat it as a model selection tool. In cases in which the data is incomplete, most researchers and common statistical software will use complete case analysis in order to estimate the R ², a procedure that might lead to biased results. In this paper, I introduce the use of multiple imputation for the estimation of R ² and adjusted R ² in incomplete data sets. I illustrate my methodology using a biomedical example.     

An Asymptotic Characterization of Finite Degree U-statistics With Sample Size-Dependent Kernels: Applications to Nonparametric Estimators and Test Statistics

We provide a simple result on the H-decomposition of a U-statistics that allows for easy determination of its magnitude when the statistic’s kernel depends on the sample size n. The result provides a direct and convenient method to characterize the asymptotic magnitude of semiparametric and nonparametric estimators or test statistics involving high dimensional sums. We illustrate the use of our result in previously studied estimators/test statistics and in a novel nonparametric R² test for overall significance of a nonparametric regression model.     

The Target Parameter of Adjusted R-Squared in Fixed-Design Experiments

R-squared (R²) and adjusted R-squared (R²_Adj) are sometimes viewed as statistics detached from any target parameter, and sometimes as estimators for the population multiple correlation. The latter interpretation is meaningful only if the explanatory variables are random. This article proposes an alternative perspective for the case where the x’s are fixed. A new parameter is defined, in a similar fashion to the construction of R², but relying on the true parameters rather than their estimates. (The parameter definition includes also the fixed x values.) This parameter is referred to as the “parametric” coefficient of determination, and denoted by ρ²_*. The proposed ρ²_* remains stable when irrelevant variables are removed (or added), unlike the unadjusted R², which always goes up when variables, either relevant or not, are added to the model (and goes down when they are removed). The value of the traditional R²_Adj may go up or down with added (or removed) variables, either relevant or not. It is shown that the unadjusted R² overestimates ρ²_*, while the traditional R²_Adj underestimates it. It is also shown that for simple linear regression the magnitude of the bias of R²_Adj can be as high as the bias of the unadjusted R² (while their signs are opposite). Asymptotic convergence in probability of R²_Adj to ρ²_* is demonstrated. The effects of model parameters on the bias of R² and R²_Adj are characterized analytically and numerically. An alternative bi-adjusted estimator is presented and evaluated.     

On the coefficient of determination for mixed regression models

For mixed regression models, we define a variance decomposition including three terms, explained individual variance, unexplained individual variance and noise variance. In contrast to traditional variance decomposition, it is thus the unexplained  , not the explained, variance that is split. It gives rise to a coefficient of individual determination (CID) defined as the estimated fraction of explained individual variance. We argue that in many applications CID is a valuable complement to R²$R^2$, since it excludes noise variance (which can never be explained) and thus has one as a natural upper bound.     

Using a Truncated C p Statistic for Variable Selection in Multiple Linear Regression

In multiple linear regression analysis each lower-dimensional subspace L of a known linear subspace M of ? ⁿ corresponds to a non empty subset of the columns of the regressor matrix. For a fixed subspace L, the C _p statistic is an unbiased estimator of the mean square error if the projection of the response vector onto L is used to estimate the expected response. In this article, we consider two truncated versions of the C _p statistic that can also be used to estimate this mean square error. The C _p statistic and its truncated versions are compared in two example data sets, illustrating that use of the truncated versions may result in models different from those selected by standard C _p .     

Catanova for multidimensional contingency tables: Nominal-scale response

This paper extends an analysis of variance for categorical data (CATANOVA) procedure to multidimensional contingency tables involving several factors and a response variable measured on a nominal scale. Using an appropriate measure of total variation for multinomial data, partial and multiple association measures are developed as R² quantities which parallel the analogous statistics in multiple linear regression for quantitative data. In addition, test statistics are derived in terms of these R² criteria. Finally, this CATANOVA approach is illustrated within the context of 2 three-way contingency table from a multicenter clinicaltrial.     

Cautionary Note about R 2

The coefficient of determination (R ²) is perhaps the single most extensively used measure of goodness of fit for regression models. It is also widely misused. The primary source of the problem is that except for linear models with an intercept term, the several alternative R ² statistics are not generally equivalent. This article discusses various considerations and potential pitfalls in using the R ²'s. Specific points are exemplified by means of empirical data. A new resistant statistic is also introduced.     

Some issues in logistic regression

Much research has been performed in the area of multiple linear regression, with the resuit that the field is well-developed. This is not true of logistic regression, however. The latter presents special problems because the response is not continuous. Some of these problems are: the difficulty of developing a suitable R² statistic, possibly poor results produced by the method of maximum likelihood, and the challenge to develop suitable graphical techniques. We describe recent work in some of these directions, and discuss the need for additional research.     

A Markov-chain-based regression model with random effects for the analysis of 18O-labelled mass spectra

The enzymatic ¹⁸O-labelling is a useful technique for reducing the influence of the between-spectra variability on the results of mass-spectrometry experiments. A difficulty in applying the technique lies in the quantification of the corresponding peptides due to the possibility of an incomplete labelling, which may result in biased estimates of the relative peptide abundance. To address the problem, Zhu et al. [A Markov-chain-based heteroscedastic regression model for the analysis of high-resolution enzymatically ¹⁸O-labeled mass spectra, J. Proteome Res. 9(5) (2010), pp. 2669–2677] proposed a Markov-chain-based regression model with heteroscedastic residual variance, which corrects for the possible bias. In this paper, we extend the model by allowing for the estimation of the technical and/or biological variability for the mass spectra data. To this aim, we use a mixed-effects version of the model. The performance of the model is evaluated based on results of an application to real-life mass spectra data and a simulation study.     

Identifying Variables Contributing to Outliers in Phase I

When a process is monitored with a T ² control chart in a Phase II setting, the MYT decomposition is a valuable diagnostic tool for interpreting signals in terms of the process variables. The decomposition splits a signaling T ² statistic into independent components that can be associated with either individual variables or groups of variables. Since these components are T ² statistics with known distributions, they can be used to determine which of the process variable(s) contribute to the signal. However, this procedure cannot be applied directly to Phase I since the distributions of the individual components are unknown. In this article, we develop the MYT decomposition procedure for a Phase I operation, when monitoring a random sample of individual observations and identifying outliers. We use a relationship between the T ² statistic in Phase I with the corresponding T ² statistic resulting when an observation is omitted from this sample to derive the distributions of these components and demonstrate the Phase I application of the MYT decomposition.     

On hsu's model in regression analysis

The paper examplifies with Hsu’s model a general pattern as how to derive results of variance component estimation from well known results on mean estimation, as far as linear model theory is concerned. This ’ dispersion-mean-correspondence‘provides new and short proofs for various theorems from the literature, concerning unbiased invariant quadratic estimators with minimum BAYES risk or minimum variance. For pure variance component models, unbiased non-negative quadratic estimability is characterized in terms of the design matrices.