Some properties of,and relationships among,several uncorrelated and homoscedastic residual vectors

In this paper we examine the properties of four types of residual vectors, arising from fitting a linear regression model to a set of data by least squares. The four types of residuals are (i) the Stepwise residuals (Hedayat and Robson, 1970), (ii) the Recursive residuals (Brown, Durbin, and Evans, 1975), (iii) the Sequentially Adjusted residuals (to be defined herein), and (iv) the BLUS residuals (Theil, 1965, 1971). We also study the relationships among the four residual vectors. It is found that, for any given sequence of observations, (i) the first three sets of residuals are identical, (ii) each of the first three sets, being identical, is a member of Thei’rs (1965, 1971) family of residuals; specifically, they are Linear Unbiased with a Scalar covariance matrix (LUS) but not Best Linear Unbiased with a Scalar covariance matrix (BLUS). We find the explicit form of the transformation matrix and show that the first three sets of residual vectors can be written as an orthogonal transformation of the BLUS residual vector. These and other properties may prove to be useful in the statistical analysis of residuals.     

An evaluation of bootstrap methods for outlier detection in least squares regression

Outlier detection is a critical part of data analysis, and the use of Studentized residuals from regression models fit using least squares is a very common approach to identifying discordant observations in linear regression problems. In this paper we propose a bootstrap approach to constructing critical points for use in outlier detection in the context of least-squares Studentized residuals, and find that this approach allows naturally for mild departures in model assumptions such as non-Normal error distributions. We illustrate our methodology through both a real data example and simulated data.     

Bootstrapping stochastic regression models under homoskedasticity: wild bootstrap vs. pairs bootstrap

We investigate by simulation how the wild bootstrap and pairs bootstrap perform in t and F tests of regression parameters in the stochastic regression model, where explanatory variables are stochastic and not given and there exists no heteroskedasticity. The wild bootstrap procedure due to Davidson and Flachaire [The wild bootstrap, tamed at last, Working paper, IER#1000, Queen's University, 2001] with restricted residuals works best but its dominance is not strong compared to the result of Flachaire [Bootstrapping heteroskedastic regression models: wild bootstrap vs. pairs bootstrap, Comput. Statist. Data Anal. 49 (2005), pp. 361–376] in the fixed regression model where explanatory variables are fixed and there exists heteroskedasticity.     

FAST DOUBLE BOOTSTRAP TESTS OF NONNESTED LINEAR REGRESSION MODELS

ABSTRACT

It has been shown in previous work that bootstrapping the J test for nonnested linear regression models dramatically improves its finite-sample performance. We provide evidence that a more sophisticated bootstrap procedure, which we call the fast double bootstrap, produces a very substantial further improvement in cases where the ordinary bootstrap does not work as well as it might. This FDB procedure is only about twice as expensive as the usual single bootstrap.     

The extreme residuals in logistic regression models

Goodness-of-fit tests for logistic regression models using extreme residuals are considered. Approximations to the moments of the Pearson residuals are given for model fits made by maximum likelihood, minimum chi-square and weighted least squares and used to define modified residuals. Approximations to the critical values of the extreme statistics based on the ordinary and modified Pearson residuals are developed and assessed for the case of a single explanatory variable.     

Wild Bootstrapping in Finite Populations with Auxiliary Information

Consider a finite population u , which can be viewed as a realization of a super-population model. A simple ratio model (linear regression, without intercept) with heteroscedastic errors is supposed to have generated u . A random sample is drawn without replacement from u . In this set-up a two-stage wild bootstrap resampling scheme as well as several other useful forms of bootstrapping in finite populations will be considered. Some asymptotic results for various bootstrap approximations for normalized and Studentized versions of the well-known ratio and regression estimator are given. Bootstrap based confidence interval s for the population total and for the regression parameter of the underlying ratio model are also discussed     

A Non‐parametric ANOVA‐type Test for Regression Curves Based on Characteristic Functions

This article studies a new procedure to test for the equality of k regression curves in a fully non‐parametric context. The test is based on the comparison of empirical estimators of the characteristic functions of the regression residuals in each population. The asymptotic behaviour of the test statistic is studied in detail. It is shown that under the null hypothesis, the distribution of the test statistic converges to a finite combination of independent chi‐squared random variables with one degree of freedom. The coefficients in this linear combination can be consistently estimated. The proposed test is able to detect contiguous alternatives converging to the null at the rate n ^{? 1 ∕ 2}. The practical performance of the test based on the asymptotic null distribution is investigated by means of simulations.     

Re-colouring the Intensity-Based Bootstrap for Point Processes

Abstract

A method for obtaining bootstrapping replicates for one-dimensional point processes is presented. The method involves estimating the conditional intensity of the process and computing residuals. The residuals are bootstrapped using a block bootstrap and used, together with the conditional intensity, to define the bootstrap realizations. The method is applied to the estimation of the cross-intensity function for data arising from a reaction time experiment.     

Heteroskedasticity-consistent covariance matrix estimation:white's estimator and the bootstrap ∗

This paper considers the issue of estimating the covariance matrix of ordinary least squares estimates in a linear regression model when heteroskedasticity is suspected. We perform Monte Carlo simulation on the White estimator, which is commonly used in.

empirical research, and also on some alternatives based on different bootstrapping schemes. Our results reveal that the White estimator can be considerably biased when the sample size is not very large, that bias correction via bootstrap does not work well, and that the weighted bootstrap estimators tend to display smaller biases than the White estimator and its variants, under both homoskedasticity and heteroskedasticity. Our results also reveal that the presence of (potentially) influential observations in the design matrix plays an important role in the finite-sample performance of the heteroskedasticity-consistent estimators.     

A simple test for comparing regression curves versus one-sided alternatives

In this article we present a simple procedure to test for the null hypothesis of equality of two regression curves versus one-sided alternatives in a general nonparametric and heteroscedastic setup. The test is based on the comparison of the sample averages of the estimated residuals in each regression model under the null hypothesis. The test statistic has asymptotic normal distribution and can detect any local alternative of rate n^-1/2. Some simulations and an application to a data set are included.     

Testing the equality of the variances of two linear models

Testing the equality of variances of two linear models with common β-parameter is considered. A test based on least squares residuals (ASR test) is proposed, and it is shown that this test is invariant under the group of scale and translation changes. For some special cases, it is also proved that this test has a monotone power function. Finding the exact critical values of this test is not easy; an approximation is given to facilitate the computation of these. The powers of the BLUS test, the F-test and the new test are computed for various alternatives and compared in a particular case. The proposed test seems to be locally more powerful than the alternative tests.     

Sufficient m-out-of-n (m/n) bootstrap

Traditional resampling methods for estimating sampling distributions sometimes fail, and alternative approaches are then needed. For example, if the classical central limit theorem does not hold and the naïve bootstrap fails, the m/n bootstrap, based on smaller-sized resamples, may be used as an alternative. An alternative to the naïve bootstrap, the sufficient bootstrap, which uses only the distinct observations in a bootstrap sample, is another recently proposed bootstrap approach that has been suggested to reduce the computational burden associated with bootstrapping. It works as long as naïve bootstrap does. However, if the naïve bootstrap fails, so will the sufficient bootstrap. In this paper, we propose combining the sufficient bootstrap with the m/n bootstrap in order to both regain consistent estimation of sampling distributions and to reduce the computational burden of the bootstrap. We obtain necessary and sufficient conditions for asymptotic normality of the proposed method, and propose new values for the resample size m. We compare the proposed method with the naïve bootstrap, the sufficient bootstrap, and the m/n bootstrap by simulation.     

Exact bootstrap confidence intervals for regression coefficients in small samples

ABSTRACT

Regression analysis is one of the important tools in statistics to investigate the relationships among variables. When the sample size is small, however, the assumptions for regression analysis can be violated. This research focuses on using the exact bootstrap to construct confidence intervals for regression parameters in small samples. The comparison of the exact bootstrap method with the basic bootstrap method was carried out by a simulation study. It was found that on a very small sample (n ≈ 5) under Laplace distribution with the independent variable treated as random, the exact bootstrap was more effective than the standard bootstrap confidence interval.     

Standardized Martingale Residuals Applied to Grouped Left Truncated Observations of Dementia Cases

The use of martingale residuals have been proposed for modelchecking and also to get a non-parametric estimate of the effectof an explanatory variable. We apply this approach to an epidemiologicalproblem which presents two characteristics: the data are lefttruncated due to delayed entry in the cohort; the data are groupedinto geographical units (parishes). This grouping suggests anatural way of smoothing the graph of residuals which is to computethe sum of the residuals for each parish. It is also naturalto present a graph with standardized residuals. We derive thevariances of the estimated residuals for left truncated datawhich allows computing the standardized residuals. This methodis applied to the study of dementia in a cohort of old people,and to the possible effect of the concentration of aluminum andsilica in drinking water on the risk of developing dementia.     

Comparison of PLS algorithms when number of objects is much larger than number of variables

NIPALS and SIMPLS algorithms are the most commonly used algorithms for partial least squares analysis. When the number of objects, N, is much larger than the number of explanatory, K, and/or response variables, M, the NIPALS algorithm can be time consuming. Even though the SIMPLS is not as time consuming as the NIPALS and can be preferred over the NIPALS, there are kernel algorithms developed especially for the cases where N is much larger than number of variables. In this study, the NIPALS, SIMPLS and some kernel algorithms have been used to built partial least squares regression model. Their performances have been compared in terms of the total CPU time spent for the calculations of latent variables, leave-one-out cross validation and bootstrap methods. According to the numerical results, one of the kernel algorithms suggested by Dayal and MacGregor (J Chemom 11:73–85, 1997) is the fastest algorithm.     

On the Edgeworth expansion and bootstrap approximation for the cox regression model under random censorship

We establish the one-term Edgeworth expansion for various statistics related to Cox semipara-metric regression model when the covariate is one-dimensional and the observations are i.i.d. We show that the bootstrap approximation method is second-order correct. The second-order-correct estimates of the sampling distribution can be obtained without Monte Carlo simulation. We pay special attention to the Studentized version of the statistics and show that their distributions are different from those of the original statistics to order n^-½     

Bootstrapping time series models

This paper surveys recent development in bootstrap methods and the modifications needed for their applicability in time series models. The paper discusses some guidelines for empirical researchers in econometric analysis of time series. Different sampling schemes for bootstrap data generation and different forms of bootstrap test statistics are discussed. The paper also discusses the applicability of direct bootstrapping of data in dynamic models and cointegrating regression models. It is argued that bootstrapping residuals is the preferable approach. The bootstrap procedures covered include the recursive bootstrap, the moving block bootstrap and the stationary bootstrap.     

Selection of Predictors in Distance-Based Regression

Distance-based regression is a prediction method consisting of two steps: from distances between observations we obtain latent variables which, in turn, are the regressors in an ordinary least squares linear model. Distances are computed from actually observed predictors by means of a suitable dissimilarity function. Being generally nonlinearly related with the response, their selection by the usual F tests is unavailable. In this article, we propose a solution to this predictor selection problem by defining generalized test statistics and adapting a nonparametric bootstrap method to estimate their p-values. We include a numerical example with automobile insurance data.