Using Heteroscedasticity-Consistent Standard Errors for the Linear Regression Model with Correlated Regressors

The use of heteroscedasticity-consistent covariance matrix (HCCM) estimators is very common in practice to draw correct inference for the coefficients of a linear regression model with heteroscedastic errors. However, in addition to the problem of heteroscedasticity, linear regression models may also be plagued with some considerable degree of collinearity among the regressors when two or more regressors are considered. This situation causes many adverse effects on the least squares measures and alternatively, the ordinary ridge regression method is used as a common practice. But in the available literature, the problems of multicollinearity and heteroscedasticity have not been discussed as a combined issue especially, for the inference of the regression coefficients. The present article addresses the inference about the regression coefficients taking both the issues of multicollinearity and heteroscedasticity into account and suggests the use of HCCM estimators for the ridge regression. This article proposes t- and F-tests, based on these HCCM estimators, that perform adequately well in the numerical evaluation of the Monte Carlo simulations.     

Exact bootstrap confidence intervals for regression coefficients in small samples

ABSTRACT

Regression analysis is one of the important tools in statistics to investigate the relationships among variables. When the sample size is small, however, the assumptions for regression analysis can be violated. This research focuses on using the exact bootstrap to construct confidence intervals for regression parameters in small samples. The comparison of the exact bootstrap method with the basic bootstrap method was carried out by a simulation study. It was found that on a very small sample (n ≈ 5) under Laplace distribution with the independent variable treated as random, the exact bootstrap was more effective than the standard bootstrap confidence interval.     

Maximum likelihood estimation in a Weibull regression model with type-1 censoring: a Monte Carlo study

Results of the Monte Carlo study of the performance of a maximum likelihood estimation in a Weibull parametric regression model with two explanatory variables are presented. One simulation run contained 1000 samples censored on the average by the amount of 0-30%. Each simulatedsample was generated in a form of two-factor two-level balanced experiment. The confidence intervals were computed using the large-sample normal approximation via the matrix of observed information. For small sample sizes the estimates of the scale parameter b of the loglifetime were significantly negatively biased, which resulted in a poor quality of confidence intervals for b and the low-level quantiles. All estimators improved their quality when the nominal value of b decreased. A moderate amount of censoring improved the quality of point and confidence estimation. The reparametrization b 7 produced rather accurate confidence intervals. Exact confidence intervals for b in case of non-censoring were obtained using the pivotal quantity b/b.     

POOLED SLICING METHODS VERSUS SLICING METHODS

We consider the semiparametric regression model introduced by [1] Duan, N. and Li, K. C. 1991. Slicing regression: a link-free regression method. The Annals of Statistics, 19: 505–530. [Crossref], [Web of Science ®] , [Google Scholar]. The dependent variable y is linked to the index x′ β through an unknown link function. [1] Duan, N. and Li, K. C. 1991. Slicing regression: a link-free regression method. The Annals of Statistics, 19: 505–530. [Crossref], [Web of Science ®] , [Google Scholar] and [2] Li, K. C. 1991. Sliced inverse regression for dimension reduction, with discussions. Journal of the American Statistical Association, 86: 316–342. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar] present Slicing methods (the Sliced Inverse Regression methods SIR-I, SIR-II and SIR_α) in order to estimate the direction of the unknown slope parameter β. These methods are computationally simple and fast but depend on the choice of an arbitrary slicing fixed by the user. When the sample size is small, the number and the position of slices have an influence on the estimated direction. In this paper, we suggest to use the corresponding Pooled Slicing methods: PSIR-I (proposed by [3] Aragon, Y. and Saracco, J. 1997. Sliced Inverse Regression (SIR): an appraisal of small sample alternatives to slicing. Computational Statistics, 12: 109–130. [Web of Science ®] , [Google Scholar]), PSIR-II and PSIR_α. These methods combine the results from a number of slicings. We compare the sample behaviour of Slicing and Pooled Slicing methods on simulations. We also propose a practical choice of α in SIR_α and PSIR_α methods.     

Some Modifications for Choosing Ridge Parameters

Standard least square regression can produce estimates having a large mean squares error (MSE) when predictor variables are highly correlated or multicollinear. In this article, we propose four modifications to choose the ridge parameter (K) when multicollinearity exists among the columns of the design matrix. The proposed new estimators are extended versions of that suggested by Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameter for regression problems . Commun. Statist. A 34 : 1177 – 1182 . [CSA] [Taylor & Francis Online] , [Google Scholar]). The properties of these estimators are compared with those of Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. . 12 : 55 – 67 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the OLS using the MSE criterion. All estimators under consideration are evaluated using simulation techniques under certain conditions where a number of factors that may affect their properties have been varied. In addition, it is shown that at least one of the proposed estimators either has a smaller MSE than the others or is the next best otherwise.     

Use of the Correlation Coefficient with Normal Probability Plots

The use of the correlation coefficient is suggested as a technique for summarizing and objectively evaluating the information contained in probability plots. Goodness-of-fit tests are constructed using this technique for several commonly used plotting positions for the normal distribution. Empirical sampling methods are used to construct the null distribution for these tests, which are then compared on the basis of power against certain nonnormal alternatives. Commonly used regression tests of fit are also included in the comparisons. The results indicate that use of the plotting position p_i = (i - .375)/(n + .25) yields a competitive regression test of fit for normality.     

Book reviews

Donald J. Koosis: Statistics. A Self-Teaching Guide, Fourth Edition. Wiley 1997, ISBN 0-471-14688-9

Christopher C. Heyde: Quasi-Likelihood and Its Applications, Springer Series in Statistics, 1997, pp. 235, [ISBN 0-387-98225-61

Jeffrey S. Simonoff: Smoothing Methods in Statistics, Springer Series in Statistics, 1996, pp. 338

Andr´e I. Khuri, Thomas Mathew and Birmal K. Sinha: Statistical Tests for Mixed Linear Models, Wiley Series in Probability and Statistics, 1998, pp. 352, PSBN 0-471-1 5653-11

Ronald Christensen: Log-Linear Models and Logistic Regression, Springer Texts in Statistics, 1997, pp. 483, [ISBN 0-387-98247-71

E. L. Lehmann: Testing Statistical Hypotheses, Springer Texts in Statistics, 1997, pp. 600, [ISBN 0-387-94919-41

S. R. Searle, G. Casella and Ch. E. McCulloch: Variance Components, Wiley and Sons, 1992, pp. 496, [ISBN 0-471-62162-51

B. J. T. Morgan: Analysis of Quantal Response Data, Chapman & Hall, 1992, pp. 51 1

Carl D. Huberty: Applied Discriminant Analysis, Wiley and Sons, 1994, pp. 490

C. R. Rao and H. Toutenburg: Linear Models. Least Squares and Alternatives, Springer Series in Statistics, 1995, 188 pp., [ISBN 0-387-94562-81     

Nonparametric comparison of curves with dependent errors

Suppose that data {(x _l,i,n , y _l,i,n ): l?=?1, …, k; i?=?1, …, n} are observed from the regression models: Y _l,i,n ?=?m _l (x _l,i,n )?+?? _l,i,n , l?=?1, …, k, where the regression functions {m _l } _l=1 ^k are unknown and the random errors {? _l,i,n } are dependent, following an MA(∞) structure. A new test is proposed for testing the hypothesis H ₀: m ₁?=?·?·?·?=?m _k , without assuming that {m _l } _l=1 ^k are in a parametric family. The criterion of the test derives from a Crámer-von-Mises-type functional based on different distances between {[mcirc]} _l and {[mcirc]} _s , l?≠?s, l, s?=?1, …, k, where {[mcirc] _l } _l=1 ^k are nonparametric Gasser–Müller estimators of {m _l } _l=1 ^k . A generalization of the test to the case of unequal design points, with different sample sizes {n _l } _l=1 ^k and different design densities {f _l } _l=1 ^k , is also considered. The asymptotic normality of the test statistic is obtained under general conditions. Finally, a simulation study and an analysis with real data show a good behavior of the proposed test.     

A Comparison of Multiple Outlier Detection Methods for Regression Data

The problem of outliers in statistical data has attracted many researchers for a long time. Consequently, numerous outlier detection methods have been proposed in the statistical literature. However, no consensus has emerged as to which method is uniformly better than the others or which one is recommended for use in practical situations. In this article, we perform an extensive comparative Monte Carlo simulation study to assess the performance of the multiple outlier detection methods that are either recently proposed or frequently cited in the outlier detection literature. Our simulation experiments include a wide variety of realistic and challenging regression scenarios. We give recommendations on which method is superior to others under what conditions.