A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.

     

Performance of asymmetric links and correction methods for imbalanced data in binary regression

In binary regression, imbalanced data result from the presence of values equal to zero (or one) in a proportion that is significantly greater than the corresponding real values of one (or zero). In this work, we evaluate two methods developed to deal with imbalanced data and compare them to the use of asymmetric links. The results based on simulation study show, that correction methods do not adequately correct bias in the estimation of regression coefficients and that the models with power links and reverse power considered produce better results for certain types of imbalanced data. Additionally, we present an application for imbalanced data, identifying the best model among the various ones proposed. The parameters are estimated using a Bayesian approach, considering the Hamiltonian Monte-Carlo method, utilizing the No-U-Turn Sampler algorithm and the comparisons of models were developed using different criteria for model comparison, predictive evaluation and quantile residuals.     

A CONSISTENT MODEL SPECIFICATION TEST BASED ON THE KERNEL SUM OF SQUARES OF RESIDUALS

This paper constructs a consistent model specification test based on the difference between the nonparametric kernel sum of squares of residuals and the sum of squares of residuals from a parametric null model. We establish the asymptotic normality of the proposed test statistic under the null hypothesis of correct parametric specification and show that the wild bootstrap method can be used to approximate the null distribution of the test statistic. Results from a small simulation study are reported to examine the finite sample performance of the proposed tests.     

Change-Point Tests for the Error Distribution in Non-parametric Regression

Abstract.  Several testing procedures are proposed that can detect change-points in the error distribution of non-parametric regression models. Different settings are considered where the change-point either occurs at some time point or at some value of the covariate. Fixed as well as random covariates are considered. Weak convergence of the suggested difference of sequential empirical processes based on non-parametrically estimated residuals to a Gaussian process is proved under the null hypothesis of no change-point. In the case of testing for a change in the error distribution that occurs with increasing time in a model with random covariates the test statistic is asymptotically distribution free and the asymptotic quantiles can be used for the test. This special test statistic can also detect a change in the regression function. In all other cases the asymptotic distribution depends on unknown features of the data-generating process and a bootstrap procedure is proposed in these cases. The small sample performances of the proposed tests are investigated by means of a simulation study and the tests are applied to a data example.     

Superiority comparisons between mixed regression estimators

This paper gives necessary and sufficient conditions for a mixed regression estimator to be superior to another mixed estimator. The comparisons are based on the mean square error matrices of the estimators. Both estimators are allowed to be biased.     

Some properties of,and relationships among,several uncorrelated and homoscedastic residual vectors

In this paper we examine the properties of four types of residual vectors, arising from fitting a linear regression model to a set of data by least squares. The four types of residuals are (i) the Stepwise residuals (Hedayat and Robson, 1970), (ii) the Recursive residuals (Brown, Durbin, and Evans, 1975), (iii) the Sequentially Adjusted residuals (to be defined herein), and (iv) the BLUS residuals (Theil, 1965, 1971). We also study the relationships among the four residual vectors. It is found that, for any given sequence of observations, (i) the first three sets of residuals are identical, (ii) each of the first three sets, being identical, is a member of Thei’rs (1965, 1971) family of residuals; specifically, they are Linear Unbiased with a Scalar covariance matrix (LUS) but not Best Linear Unbiased with a Scalar covariance matrix (BLUS). We find the explicit form of the transformation matrix and show that the first three sets of residual vectors can be written as an orthogonal transformation of the BLUS residual vector. These and other properties may prove to be useful in the statistical analysis of residuals.     

Influence diagnostics in the Gamma regression model with adjusted deviance residuals

In this study, we develop the adjusted deviance residuals for the gamma regression model (GRM) by following Cordeiro's (2004) method. These adjusted deviance residuals under the GRM are used for influence diagnostics. A comparative analysis has been sorted out between our proposed method of the adjusted deviance residuals and an existing method for influence diagnostics. These results are illustrated by a simulation study and using a real data set. They are presented for different values of dispersion and sample sizes and indicate the significant role of the GRM inferences.     

Estimation of the error distribution in a varying coefficient regression model

This paper deals with the estimation of the error distribution function in a varying coefficient regression model. We propose two estimators and study their asymptotic properties by obtaining uniform stochastic expansions. The first estimator is a residual-based empirical distribution function. We study this estimator when the varying coefficients are estimated by under-smoothed local quadratic smoothers. Our second estimator which exploits the fact that the error distribution has mean zero is a weighted residual-based empirical distribution whose weights are chosen to achieve the mean zero property using empirical likelihood methods. The second estimator improves on the first estimator. Bootstrap confidence bands based on the two estimators are also discussed.     

Application of 'delete = replace' to deletion diagnostics for variance component estimation in the linear mixed model

Summary.  'Delete = replace' is a powerful and intuitive modelling identity. This paper extends previous work by stating and proving the identity in more general terms and extending its application to deletion diagnostics for estimates of variance components obtained by restricted maximum likelihood estimation for the linear mixed model. We present a new, fast, transparent and approximate computational procedure, arising as a by-product of the fitting process. We illustrate the effect of the deletion of individual observations, of 'subjects' and of arbitrary subsets. Central to the identity and its application is the conditional residual.     

The Performance of Randomization Tests that Use Permutations of Independent Variables

ABSTRACT

In this article we evaluate the performance of a randomization test for a subset of regression coefficients in a linear model. This randomization test is based on random permutations of the independent variables. It is shown that the method maintains its level of significance, except for extreme situations, and has power that approximates the power of another randomization test, which is based on the permutation of residuals from the reduced model. We also show, via an example, that the method of permuting independent variables is more valuable than other randomization methods because it can be used in connection with the downweighting of outliers.     

Bootstrapping regression models with BLUS residuals

To bootstrap a regression problem, pairs of response and explanatory variables or residuals can be resam‐pled, according to whether we believe that the explanatory variables are random or fixed. In the latter case, different residuals have been proposed in the literature, including the ordinary residuals (Efron 1979), standardized residuals (Bickel & Freedman 1983) and Studentized residuals (Weber 1984). Freedman (1981) has shown that the bootstrap from ordinary residuals is asymptotically valid when the number of cases increases and the number of variables is fixed. Bickel & Freedman (1983) have shown the asymptotic validity for ordinary residuals when the number of variables and the number of cases both increase, provided that the ratio of the two converges to zero at an appropriate rate. In this paper, the authors introduce the use of BLUS (Best Linear Unbiased with Scalar covariance matrix) residuals in bootstrapping regression models. The main advantage of the BLUS residuals, introduced in Theil (1965), is that they are uncorrelated. The main disadvantage is that only n —p residuals can be computed for a regression problem with n cases and p variables. The asymptotic results of Freedman (1981) and Bickel & Freedman (1983) for the ordinary (and standardized) residuals are generalized to the BLUS residuals. A small simulation study shows that even though only n — p residuals are available, in small samples bootstrapping BLUS residuals can be as good as, and sometimes better than, bootstrapping from standardized or Studentized residuals.     

A robust estimator for wall following

An early goal in autonomous navigation research is to build a research vehicle which can travel through office areas and factory floors, A simple strategy for directing the robot's movement in a hallway is to maintain a fixed distance from the wall. The problem is complicated by the fact that there are many factors in the environment, such as opened doors, pillars or other temporary objects, that can introduce 'noise' into the distance measure. To maintain a proper path with minimum interruption, the robot should have the ability to make decisions, based on measurements, and adjust its course only when it is deemed necessary. This report describes a new algorithm which enables the robot to move along and maintain a fixed distance from a reference object. The method, based on a robust estimator of the location, combines information from earlier measurements with current observations from range sensors to effectively produce an estimate of the distance between the robot and the object. A simulation study, showing the trajectories generated using this algorithm with different parameters for different environments, is presented.     

Bootstrap misspecification tests for ARCH based on the empirical process of squared residuals

We propose and study by means of simulations and graphical tools a class of goodness-of-fit tests for ARCH models. The tests are based on the empirical distribution function of squared residuals and smooth (parametric) bootstrap. We examine empirical size and power by means of a simulation study. While the tests have overall correct size, their power strongly depends on the type of alternative and is particularly high when the assumption of Gaussian innovations is violated. As an example, the tests are applied to returns on Foreign Exchange rates.     

A simple test for comparing regression curves versus one-sided alternatives

In this article we present a simple procedure to test for the null hypothesis of equality of two regression curves versus one-sided alternatives in a general nonparametric and heteroscedastic setup. The test is based on the comparison of the sample averages of the estimated residuals in each regression model under the null hypothesis. The test statistic has asymptotic normal distribution and can detect any local alternative of rate n^-1/2. Some simulations and an application to a data set are included.     

APPLICATIONS OF ROBUST ESTIMATION FOR VARIANCE COMPONENTS MODELS:THE DETECTION OF MAJOR GENE EFFECTS IN FINGER RIDGE COUNTS IN NORMAL AND FRAGILE X FAMILIES

A diagnostic technique is proposed to detect major gene effects and other systematic departures from a model for the trait means in the presence of outliers. The technique is based on the examination of residuals from fitting variance components models to quantitative pedigree data using robust statistical procedures. The approach is demonstrated using the total ridge count and ridge count of the middle finger from 54 extended families affected with the Fragile X syndrome, and a sample of 217 normal pedigrees.     

Identification of Multiple Outliers in Logistic Regression

The use of logistic regression modeling has seen a great deal of attention in the literature in recent years. This includes all aspects of the logistic regression model including the identification of outliers. A variety of methods for the identification of outliers, such as the standardized Pearson residuals, are now available in the literature. These methods, however, are successful only if the data contain a single outlier. In the presence of multiple outliers in the data, which is often the case in practice, these methods fail to detect the outliers. This is due to the well-known problems of masking (false negative) and swamping (false positive) effects. In this article, we propose a new method for the identification of multiple outliers in logistic regression. We develop a generalized version of standardized Pearson residuals based on group deletion and then propose a technique for identifying multiple outliers. The performance of the proposed method is then investigated through several examples.     

Testing the equality of the variances of two linear models

Testing the equality of variances of two linear models with common β-parameter is considered. A test based on least squares residuals (ASR test) is proposed, and it is shown that this test is invariant under the group of scale and translation changes. For some special cases, it is also proved that this test has a monotone power function. Finding the exact critical values of this test is not easy; an approximation is given to facilitate the computation of these. The powers of the BLUS test, the F-test and the new test are computed for various alternatives and compared in a particular case. The proposed test seems to be locally more powerful than the alternative tests.     

On the sensitivity to non-normality of a test for outliers in linear models

In this note we use the class of exponential power distributions to assess the robustness to non-normality of the test for outliers based on the maximum absolute studentized residual. We find that the significance levels can be quite markedly affected by even moderate departures from normality of the error distribution in a regression model when the sample size is moderately large.     

Checking the Short‐Term and Long‐Term Hazard Ratio Model for Survival Data

Abstract. The short‐term and long‐term hazard ratio model includes the proportional hazards model and the proportional odds model as submodels, and allows a wider range of hazard ratio patterns compared with some of the more traditional models. We propose two omnibus tests for checking this model, based, respectively, on the martingale residuals and the contrast between the non‐parametric and model‐based estimators of the survival function. These tests are shown to be consistent against any departure from the model. The empirical behaviours of the tests are studied in simulations, and the tests are illustrated with some real data examples.