Bootstrapping regression models with BLUS residuals

To bootstrap a regression problem, pairs of response and explanatory variables or residuals can be resam‐pled, according to whether we believe that the explanatory variables are random or fixed. In the latter case, different residuals have been proposed in the literature, including the ordinary residuals (Efron 1979), standardized residuals (Bickel & Freedman 1983) and Studentized residuals (Weber 1984). Freedman (1981) has shown that the bootstrap from ordinary residuals is asymptotically valid when the number of cases increases and the number of variables is fixed. Bickel & Freedman (1983) have shown the asymptotic validity for ordinary residuals when the number of variables and the number of cases both increase, provided that the ratio of the two converges to zero at an appropriate rate. In this paper, the authors introduce the use of BLUS (Best Linear Unbiased with Scalar covariance matrix) residuals in bootstrapping regression models. The main advantage of the BLUS residuals, introduced in Theil (1965), is that they are uncorrelated. The main disadvantage is that only n —p residuals can be computed for a regression problem with n cases and p variables. The asymptotic results of Freedman (1981) and Bickel & Freedman (1983) for the ordinary (and standardized) residuals are generalized to the BLUS residuals. A small simulation study shows that even though only n — p residuals are available, in small samples bootstrapping BLUS residuals can be as good as, and sometimes better than, bootstrapping from standardized or Studentized residuals.     

On a Relation between Principal Components and Regression Analysis

A regression approach to principal component analysis is presented in this note. We provide an alternative interpretation of principal components that illustrates the relation between the extra sum of squares in regression analysis and the eigenvalues associated with the principal components.     

Bias correction and improved residuals for non-exponential family nonlinear models

In this paper we derive general formulae for the biases to order n ^?1 of the parameter estimates in a general class of nonlinear regression models, where n is the sample size. The formulae are related to those of Cordeiro and McCullagh (1991) and Paula (1992) and may be viewed as extensions of their results, Correction factors are derived for the score and deviance component residuals in these models. The practical use of such corrections is illustrated for the log-gamma model.     

Market Models and Heteroscedasticity of Residual Security Returns

Bickel's tests provide attractive robust alternatives to traditional tests of heteroscedasticity. NYSE securities have homoscedastic residuals once nonnormality of security returns is allowed. Some portfolio designs may generate some heteroscedastic residuals, but this heteroscedasticity is reduced by the Kraus-Litzenberger quadratic market model. This result suggests a misspecification in the traditional market model.     

Econometric Applications of the Forward Search in Regression: Robustness,Diagnostics, and Graphics

The article illustrates the use of the forward search to provide robust analyses of econometric data. The emphasis is on informative plots that reveal the inferential importance of each observation. The division of observations into “good” and “bad” leverage points is shown to be potentially misleading.     

Extended Glivenko–Cantelli Theorem in Nonparametric Regression

In this paper, we consider the uniform strong consistency of the cumulative distribution function estimator in nonparametric regression. We obtain the extended Glivenko–Cantelli theorem for the residual-based empirical distribution function.     

Robust estimation for finite populations based on a working model

A common scenario in finite population inference is that it is possible to find a working superpopulation model which explains the main features of the population but which may not capture all the fine details. In addition, there are often outliers in the population which do not follow the assumed superpopulation model. In situations like these, it is still advantageous to make use of the working model to estimate finite population quantities, provided that we do it in a robust manner. The approach that we suggest is first to fit the working model to the sample and then to fine-tune for departures from the model assumed by estimating the conditional distribution of the residuals as a function of the auxiliary variable. This is a more direct approach to handling outliers and model misspecification than the Huber approach that is currently being used. Two simple methods, stratification and nearest neighbour smoothing, are used to estimate the conditional distributions of the residuals, which result in two modifications to the standard model-based estimator of the population distribution function. The estimators suggested perform very well in simulation studies involving two types of model departure and have small variances due to their model-based construction as well as acceptable bias. The potential advantage of the proposed robustified model-based approach over direct nonparametric regression is also demonstrated.     

Residual plots for linear regression models with censored outcome data: A refined method for visualizing residual uncertainty

Residual plots are a standard tool for assessing model fit. When some outcome data are censored, standard residual plots become less appropriate. Here, we develop a new procedure for producing residual plots for linear regression models where some or all of the outcome data are censored. We implement two approaches for incorporating parameter uncertainty. We illustrate our methodology by examining the model fit for an analysis of bacterial load data from a trial for chronic obstructive pulmonary disease. Simulated datasets show that the method can be used when the outcome data consist of a variety of types of censoring.     

Comments on “bootstrapping time series models”

We take issue with the main suggestion in Li and Maddala (LiMa) that bootstrapping residuals is always the preferred approach and question some of their guidelines. We show that it can be potentially misleading to mimic the autocovariance structure of residuals, since it can be very different from that of true errors. We emphasize that the residuals are sensitive to model misspecification and generally not a part of the information set. We make constructive suggestions and propose a semiparametric method.     

Beta Regression for Modelling Rates and Proportions

This paper proposes a regression model where the response is beta distributed using a parameterization of the beta law that is indexed by mean and dispersion parameters. The proposed model is useful for situations where the variable of interest is continuous and restricted to the interval (0, 1) and is related to other variables through a regression structure. The regression parameters of the beta regression model are interpretable in terms of the mean of the response and, when the logit link is used, of an odds ratio, unlike the parameters of a linear regression that employs a transformed response. Estimation is performed by maximum likelihood. We provide closed-form expressions for the score function, for Fisher's information matrix and its inverse. Hypothesis testing is performed using approximations obtained from the asymptotic normality of the maximum likelihood estimator. Some diagnostic measures are introduced. Finally, practical applications that employ real data are presented and discussed.