Robust regression through robust covariances

This paper discusses the estimation of regression parameters after summarizing the data by a covariance matrix of the concatenated vector of explanatory variables and response variable. A robust estimate of the covariance matrix leads to a robust regression estimator. An M-estimator at the covariance estimation step is studied in the paper, and the resulting regression estimator is compared to a few previously proposed robust regression estimators.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Robust estimation of a high-dimensional integrated covariance matrix

In this article, we consider a robust method of estimating a realized covariance matrix calculated as the sum of cross products of intraday high-frequency returns. According to recent articles in financial econometrics, the realized covariance matrix is essentially contaminated with market microstructure noise. Although techniques for removing noise from the matrix have been studied since the early 2000s, they have primarily investigated a low-dimensional covariance matrix with statistically significant sample sizes. We focus on noise-robust covariance estimation under converse circumstances, that is, a high-dimensional covariance matrix possibly with a small sample size. For the estimation, we utilize a statistical hypothesis test based on the characteristic that the largest eigenvalue of the covariance matrix asymptotically follows a Tracy–Widom distribution. The null hypothesis assumes that log returns are not pure noises. If a sample eigenvalue is larger than the relevant critical value, then we fail to reject the null hypothesis. The simulation results show that the estimator studied here performs better than others as measured by mean squared error. The empirical analysis shows that our proposed estimator can be adopted to forecast future covariance matrices using real data.     

Robust estimation of variance components

New robust estimates for variance components are introduced. Two simple models are considered: the balanced one-way classification model with a random factor and the balanced mixed model with one random factor and one fixed factor. However, the method of estimation proposed can be extended to more complex models. The new method of estimation we propose is based on the relationship between the variance components and the coefficients of the least-mean-squared-error predictor between two observations of the same group. This relationship enables us to transform the problem of estimating the variance components into the problem of estimating the coefficients of a simple linear regression model. The variance-component estimators derived from the least-squares regression estimates are shown to coincide with the maximum-likelihood estimates. Robust estimates of the variance components can be obtained by replacing the least-squares estimates by robust regression estimates. In particular, a Monte Carlo study shows that for outlier-contaminated normal samples, the estimates of variance components derived from GM regression estimates and the derived test outperform other robust procedures.     

Estimation of partially specified spatial panel data models with random-effects and spatially correlated error components

This paper studies estimation of a partially specified spatial panel data linear regression with random-effects and spatially correlated error components. Under the assumption of exogenous spatial weighting matrix and exogenous regressors, the unknown parameter is estimated by applying the instrumental variable estimation. Under some sufficient conditions, the proposed estimator for the finite dimensional parameters is shown to be root-N consistent and asymptotically normally distributed; the proposed estimator for the unknown function is shown to be consistent and asymptotically distributed as well, though at a rate slower than root-N. Consistent estimators for the asymptotic variance–covariance matrices of both the parametric and unknown components are provided. The Monte Carlo simulation results suggest that the approach has some practical value.     

Estimation of the Length Distribution of Marine Populations in the Gaussian-multinomial Setting using the Method of Moments

In this paper, the problem of estimation of the length distribution of marine populations in the Gaussian-multinomial model is considered. For the purpose of the mean and covariance parameter estimation, the method of moments estimators are developed. That is, minimum variance linear unbiased estimator for the mean frequency vector is derived and a consistent estimator for the covariance matrix of the length observations is presented. The usefulness of the proposed estimators is illustrated with an analysis of real cod length measurement data.     

Robust estimation for zero-inflated poisson autoregressive models based on density power divergence

In this study, we consider a robust estimation for zero-inflated Poisson autoregressive models using the minimum density power divergence estimator designed by Basu et al. [Robust and efficient estimation by minimising a density power divergence. Biometrika. 1998;85:549–559]. We show that under some regularity conditions, the proposed estimator is strongly consistent and asymptotically normal. The performance of the estimator is evaluated through Monte Carlo simulations. A real data analysis using New South Wales crime data is also provided for illustration.     

The quantile-based flattened logistic distribution: Some properties and applications

In this paper, the quantile-based flattened logistic distribution has been studied. Some classical and quantile-based properties of the distribution have been obtained. Closed form expressions of L-moments, L-moment ratios and expectation of order statistics of the distribution have been obtained. A quantile-based analysis concerning the method of matching L-moments estimation is employed to estimate the parameters of the proposed model. We further derive the asymptotic variance–covariance matrix of the matching L-Moments estimators of the proposed model. Finally, we apply the proposed model to simulated as well as two real life datasets and compare the fit with the logistic distribution.     

Outlier detection and robust estimation in linear regression models with fixed group effects

This work studies outlier detection and robust estimation with data that are naturally distributed into groups and which follow approximately a linear regression model with fixed group effects. For this, several methods are considered. First, the robust fitting method of Peña and Yohai [A fast procedure for outlier diagnostics in large regression problems. J Am Stat Assoc. 1999;94:434–445], called principal sensitivity components (PSC) method, is adapted to the grouped data structure and the mentioned model. The robust methods RDL₁ of Hubert and Rousseeuw [Robust regression with both continuous and binary regressors. J Stat Plan Inference. 1997;57:153–163] and M-S of Maronna and Yohai [Robust regression with both continuous and categorical predictors. Journal of Statistical Planning and Inference 2000;89:197–214] are also considered. These three methods are compared in terms of their effectiveness in outlier detection and their robustness through simulations, considering several contamination scenarios and growing contamination levels. Results indicate that the adapted PSC procedure is able to detect a high percentage of true outliers and a small number of false outliers. It is appropriate when the contamination is in the error term or in the covariates, detecting also possibly masked high leverage points. Moreover, in simulations the final robust regression estimator preserved good efficiency under Normality while keeping good robustness properties.     

Robust estimation for order of hidden Markov models based on density power divergences

In this paper, we study the robust estimation for the order of hidden Markov model (HMM) based on a penalized minimum density power divergence estimator, which is obtained by utilizing the finite mixture marginal distribution of HMM. For this task, we adopt the locally conic parametrization method used in [D. Dacunha-Castelle and E. Gassiate, Testing in locally conic models and application to mixture models. ESAIM Probab. Stat. (1997), pp. 285–317; D. Dacunha-Castelle and E. Gassiate, Testing the order of a model using locally conic parametrization: population mixtures and stationary arma processes, Ann. Statist. 27 (1999), pp. 1178–1209; T. Lee and S. Lee, Robust and consistent estimation of the order of finite mixture models based on the minimizing a density power divergence estimator, Metrika 68 (2008), pp. 365–390] to avoid the difficulties that arise in handling mixture marginal models, such as the non-identifiability of the parameter space and the singularity problem with the asymptotic variance. We verify that the estimated order is consistent and simulation results are provided for illustration.     

A Wald-type test statistic based on robust modified median estimator in logistic regression models

In this paper a new robust estimator, modified median estimator, is introduced and studied for the logistic regression model. This estimator is based on the median estimator considered in Hobza et al. [Robust median estimator in logistic regression. J Stat Plan Inference. 2008;138:3822–3840]. Its asymptotic distribution is obtained. Using the modified median estimator, we also consider a Wald-type test statistic for testing linear hypotheses in the logistic regression model and we obtain its asymptotic distribution under the assumption of random regressors. An extensive simulation study is presented in order to analyse the efficiency as well as the robustness of the modified median estimator and Wald-type test based on it.     

Regularized covariance matrix estimation under the common principal components model

The common principal components (CPC) model provides a way to model the population covariance matrices of several groups by assuming a common eigenvector structure. When appropriate, this model can provide covariance matrix estimators of which the elements have smaller standard errors than when using either the pooled covariance matrix or the per group unbiased sample covariance matrix estimators. In this article, a regularized CPC estimator under the assumption of a common (or partially common) eigenvector structure in the populations is proposed. After estimation of the common eigenvectors using the Flury–Gautschi (or other) algorithm, the off-diagonal elements of the nearly diagonalized covariance matrices are shrunk towards zero and multiplied with the orthogonal common eigenvector matrix to obtain the regularized CPC covariance matrix estimates. The optimal shrinkage intensity per group can be estimated using cross-validation. The efficiency of these estimators compared to the pooled and unbiased estimators is investigated in a Monte Carlo simulation study, and the regularized CPC estimator is applied to a real dataset to demonstrate the utility of the method.     

Robust estimation of mean and covariance for longitudinal data with dropouts

In this paper, we study estimation of linear models in the framework of longitudinal data with dropouts. Under the assumptions that random errors follow an elliptical distribution and all the subjects share the same within-subject covariance matrix which does not depend on covariates, we develop a robust method for simultaneous estimation of mean and covariance. The proposed method is robust against outliers, and does not require to model the covariance and missing data process. Theoretical properties of the proposed estimator are established and simulation studies show its good performance. In the end, the proposed method is applied to a real data analysis for illustration.     

A consistent simulation-based estimator in generalized linear mixed models

We propose a strongly root-n consistent simulation-based estimator for the generalized linear mixed models. This estimator is constructed based on the first two marginal moments of the response variables, and it allows the random effects to have any parametric distribution (not necessarily normal). Consistency and asymptotic normality for the proposed estimator are derived under fairly general regularity conditions. We also demonstrate that this estimator has a bounded influence function and that it is robust against data outliers. A bias correction technique is proposed to reduce the finite sample bias in the estimation of variance components. The methodology is illustrated through an application to the famed seizure count data and some simulation studies.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

Accounting for Uncertainty in Heteroscedasticity in Nonlinear Regression

Toxicologists and pharmacologists often describe toxicity of a chemical using parameters of a nonlinear regression model. Thus estimation of parameters of a nonlinear regression model is an important problem. The estimates of the parameters and their uncertainty estimates depend upon the underlying error variance structure in the model. Typically, a priori the researcher would not know if the error variances are homoscedastic (i.e., constant across dose) or if they are heteroscedastic (i.e., the variance is a function of dose). Motivated by this concern, in this paper we introduce an estimation procedure based on preliminary test which selects an appropriate estimation procedure accounting for the underlying error variance structure. Since outliers and influential observations are common in toxicological data, the proposed methodology uses M-estimators. The asymptotic properties of the preliminary test estimator are investigated; in particular its asymptotic covariance matrix is derived. The performance of the proposed estimator is compared with several standard estimators using simulation studies. The proposed methodology is also illustrated using a data set obtained from the National Toxicology Program.     

A new heteroskedasticity-consistent covariance matrix estimator for the linear regression model

The assumption that all random errors in the linear regression model share the same variance (homoskedasticity) is often violated in practice. The ordinary least squares estimator of the vector of regression parameters remains unbiased, consistent and asymptotically normal under unequal error variances. Many practitioners then choose to base their inferences on such an estimator. The usual practice is to couple it with an asymptotically valid estimation of its covariance matrix, and then carry out hypothesis tests that are valid under heteroskedasticity of unknown form. We use numerical integration methods to compute the exact null distributions of some quasi-t test statistics, and propose a new covariance matrix estimator. The numerical results favor testing inference based on the estimator we propose.     

稳健高维协方差矩阵估计及其投资组合应用——基于中心正则化算法

高维协方差矩阵的估计问题现已成为大数据统计分析中的基本问题,传统方法要求数据满足正态分布假定且未考虑异常值影响,当前已无法满足应用需要,更加稳健的估计方法亟待被提出。针对高维协方差矩阵,一种稳健的基于子样本分组的均值-中位数估计方法被提出且简单易行,然而此方法估计的矩阵并不具备正定稀疏特性。基于此问题,本文引进一种中心正则化算法,弥补了原始方法的缺陷,通过在求解过程中对估计矩阵的非对角元素施加L1范数惩罚,使估计的矩阵具备正定稀疏的特性,显著提高了其应用价值。在数值模拟中,本文所提出的中心正则稳健估计有着更高的估计精度,同时更加贴近真实设定矩阵的稀疏结构。在后续的投资组合实证分析中,与传统样本协方差矩阵估计方法、均值-中位数估计方法和RA-LASSO方法相比,基于中心正则稳健估计构造的最小方差投资组合收益率有着更低的波动表现。     

Adaptive robust estimation in joint mean–covariance regression model for bivariate longitudinal data

The estimation of the covariance matrix is important in the analysis of bivariate longitudinal data. A good estimator for the covariance matrix can improve the efficiency of the estimators of the mean regression coefficients. Furthermore, the covariance estimation itself is also of interest, but it is a challenging job to model the covariance matrix of bivariate longitudinal data due to the complex structure and positive definite constraint. In addition, most of existing approaches are based on the maximum likelihood, which is very sensitive to outliers or heavy-tail error distributions. In this article, an adaptive robust estimation method is proposed for bivariate longitudinal data. Unlike the existing likelihood-based methods, the proposed method can adapt to different error distributions. Specifically, at first, we utilize the modified Cholesky block decomposition to parameterize the covariance matrices. Secondly, we apply the bounded Huber's score function to develop a set of robust generalized estimating equations to estimate the parameters both in the mean and the covariance models simultaneously. A data-driven approach is presented to select the parameter c in the Huber's score function, which can ensure that the proposed method is robust and efficient. A simulation study and a real data analysis are conducted to illustrate the robustness and efficiency of the proposed approach.     

Choosing a robustness tuning parameter

A novel method is proposed for choosing the tuning parameter associated with a family of robust estimators. It consists of minimising estimated mean squared error, an approach that requires pilot estimation of model parameters. The method is explored for the family of minimum distance estimators proposed by [Basu, A., Harris, I.R., Hjort, N.L. and Jones, M.C., 1998, Robust and efficient estimation by minimising a density power divergence. Biometrika, 85, 549–559.] Our preference in that context is for a version of the method using the L ₂ distance estimator [Scott, D.W., 2001, Parametric statistical modeling by minimum integrated squared error. Technometrics, 43, 274–285.] as pilot estimator.