Improvements in the small sample efficiency of the minimum S-divergence estimators under discrete models

This paper considers the problem of inliers and empty cells and the resulting issue of relative inefficiency in estimation under pure samples from a discrete population when the sample size is small. Many minimum divergence estimators in the S-divergence family, although possessing very strong outlier stability properties, often have very poor small sample efficiency in the presence of inliers and some are not even defined in the presence of a single empty cell; this limits the practical applicability of these estimators, in spite of their otherwise sound robustness properties and high asymptotic efficiency. Here, we study a penalized version of the S-divergences such that the resulting minimum divergence estimators are free from these issues, without altering their robustness properties and asymptotic efficiencies. We present a general proof for the asymptotic properties of these minimum penalized S-divergence estimators. This provides a significant addition to the literature, as the asymptotics of penalized divergences which are not finitely defined are currently unavailable in the literature. The small sample advantages of the minimum penalized S-divergence estimators are examined through an extensive simulation study and some empirical suggestions regarding the choice of the relevant underlying tuning parameters are also provided.     

Asymptotic properties of one-step M-estimators

We study the asymptotic behavior of one-step M-estimators based on not necessarily independent identically distributed observations. In particular, we find conditions for asymptotic normality of these estimators. Asymptotic normality of one-step M-estimators is proven under a wide spectrum of constraints on the exactness of initial estimators. We discuss the question of minimal restrictions on the exactness of initial estimators. We also discuss the asymptotic behavior of the solution to an M-equation closest to the parameter under consideration. As an application, we consider some examples of one-step approximation of quasi-likelihood estimators in nonlinear regression.     

Discussion of “The power of monitoring: how to make the most of a contaminated multivariate sample” by Andrea Cerioli,Marco Riani,Anthony C. Atkinson and Aldo Corbellini

This paper discusses the contribution of Cerioli et al. (Stat Methods Appl, 2018), where robust monitoring based on high breakdown point estimators is proposed for multivariate data. The results follow years of development in robust diagnostic techniques. We discuss the issues of extending data monitoring to other models with complex structure, e.g. factor analysis, mixed linear models for which S and MM-estimators exist or deviating data cells. We emphasise the importance of robust testing that is often overlooked despite robust tests being readily available once S and MM-estimators have been defined. We mention open questions like out-of-sample inference or big data issues that would benefit from monitoring.     

Hypothesis Testing in Functional Comparative Calibration Models

The purpose of this article is to investigate hypothesis testing in functional comparative calibration models. Wald type statistics are considered which are asymptotically distributed according to the chi-square distribution. The statistics are based on maximum likelihood, corrected score approach, and method of moment estimators of the model parameters, which are shown to be consistent and asymptotically normally distributed. Results of analytical and simulation studies seem to indicate that the Wald statistics based on the method of moment estimators and the corrected score estimators are, as expected, less efficient than the Wald type statistic based on the maximum likelihood estimators for small n. Wald statistic based on moment estimators are simpler to compute than the other Wald statistics tests and their performance improves significantly as n increases. Comparisons with an alternative F statistics proposed in the literature are also reported.     

Some small-sample properties of some recently proposed multivariate outlier detection techniques

Recently, several new robust multivariate estimators of location and scatter have been proposed that provide new and improved methods for detecting multivariate outliers. But for small sample sizes, there are no results on how these new multivariate outlier detection techniques compare in terms of p _n , their outside rate per observation (the expected proportion of points declared outliers) under normality. And there are no results comparing their ability to detect truly unusual points based on the model that generated the data. Moreover, there are no results comparing these methods to two fairly new techniques that do not rely on some robust covariance matrix. It is found that for an approach based on the orthogonal Gnanadesikan–Kettenring estimator, p _n can be very unsatisfactory with small sample sizes, but a simple modification gives much more satisfactory results. Similar problems were found when using the median ball algorithm, but a modification proved to be unsatisfactory. The translated-biweights (TBS) estimator generally performs well with a sample size of n≥20 and when dealing with p-variate data where p≤5. But with p=8 it can be unsatisfactory, even with n=200. A projection method as well the minimum generalized variance method generally perform best, but with p≤5 conditions where the TBS method is preferable are described. In terms of detecting truly unusual points, the methods can differ substantially depending on where the outliers happen to be, the number of outliers present, and the correlations among the variables.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Redescending M-estimators

In finite sample studies redescending M-estimators outperform bounded M-estimators (see for example, Andrews et al. [1972. Robust Estimates of Location. Princeton University Press, Princeton]). Even though redescenders arise naturally out of the maximum likelihood approach if one uses very heavy-tailed models, the commonly used redescenders have been derived from purely heuristic considerations. Using a recent approach proposed by Shurygin, we study the optimality of redescending M-estimators. We show that redescending M-estimator can be designed by applying a global minimax criterion to locally robust estimators, namely maximizing over a class of densities the minimum variance sensitivity over a class of estimators. As a particular result, we prove that Smith's estimator, which is a compromise between Huber's skipped mean and Tukey's biweight, provides a guaranteed level of an estimator's variance sensitivity over the class of densities with a bounded variance.     

Recursive kernel estimate of the conditional quantile for functional ergodic data

ABSTRACT

In this article, we study the recursive kernel estimator of the conditional quantile of a scalar response variable Y given a random variable (rv) X taking values in a semi-metric space. Two estimators are considered. While the first one is given by inverting the double-kernel estimate of the conditional distribution function, the second estimator is obtained by using the robust approach. We establish the almost complete consistency of these estimates when the observations are sampled from a functional ergodic process. Finally, a simulation study is carried out to illustrate the finite sample performance of these estimators.     

Bias-robust L-estimators of a scale parameter

We derive optimal bias-robust L-estimators of a scale parameter σ based on random samples from F(( ·?θ/σ), where θ and σ are unknown and F is an unknown member of a ε-contaminated neighborhood of a fixed symmetric error distribution F ₀. Within a very general class S of L-estimators which are Fisher-consistent at F, we solve for: (i) the estimator with minimax asymptotic bias over the ε-contamination neighborhood; and (ii) the estimator with minimum gross error sensitivity at F ₀ [the limiting case of (i) as ε → 0]. The solutions to problems (i) and (ii) are shown, using a generalized method of moment spaces, to be mixtures of at most two interquantile ranges. A graphical method is presented for finding the optimal bias-robust solutions, and examples are given.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Comparison of the estimators of the intra-cluster correlation for the nested error regression model

The intra-cluster correlation is insisted on nested error regression model that, in practice, is rarely known. This article demonstrates the size in generalized least squares (GLS) F-test using Fuller–Battese transformation and modification F-test. For the balanced case, the former using strictly positive, analysis of covariance (ANCOVA) and analysis of variance (ANOVA) estimators of intra-cluster correlation can control the size for moderate intra-cluster correlations. For small intra-cluster correlation, they perform well when the numbers of cluster are large. The latter using the ANOVA estimator performs well except for small numbers of cluster. When intra-cluster correlation is large, it cannot control the size. For the unbalanced case, the GLS F-test using the Fuller–Battese transformation and the modification F-test using the strictly positive, the ANCOVA and the ANOVA estimators maintain the significance level for small total sample size and small intra-cluster correlations when there is a large variation in cluster sizes, but they perform well in controlling the size for large total sample size and small different variation in cluster sizes. Besides, Henderson’s method 3 estimator maintains the significance level for a few situations.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Asymptotic Normality of M-Estimators for Varying Coefficient Models with Longitudinal Data

This article considers a nonparametric varying coefficient regression model with longitudinal observations. The relationship between the dependent variable and the covariates is assumed to be linear at a specific time point, but the coefficients are allowed to change over time. A general formulation is used to treat mean regression, median regression, quantile regression, and robust mean regression in one setting. The local M-estimators of the unknown coefficient functions are obtained by local linear method. The asymptotic distributions of M-estimators of unknown coefficient functions at both interior and boundary points are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are derived. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

M-Estimator of a Generalized Linear Model with Measurement Errors

In this article, we propose a generalized linear model and estimate the unknown parameters using robust M-estimator. Under suitable conditions and by the strong law of large numbers and central limits theorem, the proposed M-estimators are proved to be consistent and asymptotically normal. We also evaluate the finite sample performance of our estimator through a Monte Carlo study.     

Outlier detection for high dimensional data using the Comedian approach

The process of detection of outliers is an interesting and important aspect in the analysis of data, as it could impact the inference. There are various methods available in the literature for detection of outliers in multivariate data [V. Barnett and T. Lewis, Outliers in Statistical Data, John Wiley & Sons, Chichester, 1994] using the Mahalanobis distance measure. An attempt is made to propose an alternate method of outlier detection based on the comedian introduced by Falk [On MAD and Comedians, Ann. Inst. Statist. Math. 49 (1997), pp. 615–644]. The proposed method is computationally efficient with high breakdown value and low computation time. Further, important properties, namely, success rates (SR) and false detection rates (FDR) are studied and compared with some of the well-known outlier detection methods through a simulation study. The Comedian method has high SR and low FDR for all combination of parameters. On removal of the detected outliers or down weighing, the same, highly robust and approximately affine equivariant estimators of multivariate location and scatter can be obtained. Finally, the method is applied to well-known real data sets to evaluate its performance.     

Robust kernel estimators for additive models with dependent observations

Robust nonparametric estimators for additive regression or autoregression models under an α-mixing condition are proposed. They are based on local M-estimators or local medians with kernel weights, and their asymptotic behaviour is studied. Moreover, diese local M-estimators achieve the same univariate rate of convergence as their linear relatives.     

Robust M-estimators of scale: Minimax bias versus maximal variance

In the location-scale estimation problem, we study robustness properties of M-estimators of the scale parameter under unknown ?-contamination of a fixed symmetric unimodal error distribution F₀. Within a general class of M-estimators, the estimator with minimax asymptotic bias is shown to lie within the subclass of α-interquantile ranges of the empirical distribution symmetrized about the sample median. Our main result is that as ? → 0, the limiting minimax asymptotic bias estimator is sometimes (e.g., when F_o is Cauchy), but not always, the median absolute deviation about the median. It is also shown that contamination in the neighbourhood of a discontinuity of the influence function of a minimax bias estimator can sometimes inflate the asymptotic variance beyond that achieved by placing all the ?-contamination at infinity. This effect is quantified by a new notion of asymptotic efficiency that takes into account the effect of infinitesimal contamination of the parametric model for the error distribution.     

Robust test for means when population variances are unequal

We consider the problem of testing the equality of two population means when the population variances are not necessarily equal. We propose a Welch-type statistic, say T^* _c, based on Tiku!s ‘1967, 1980’ modified maximum likelihood estimators, and show that this statistic is robust to symmetric and moderately skew distributions. We investigate the power properties of the statistic T^* _c; T^* _c clearly seems to be more powerful than Yuen's ‘1974’ Welch-type robust statistic based on the trimmed sample means and the matching sample variances. We show that the analogous statistics based on the ‘adaptive’ robust estimators give misleading Type I errors. We generalize the results to testing linear contrasts among k population means     

Evaluating the performance of simple estimators for probit models with two dummy endogenous regressors

This study considers the small sample performance of approximate but simple two-stage estimators for probit models with two endogenous binary covariates. Monte Carlo simulations show that all the considered estimators, including the simulated maximum-likelihood (SML) estimation, of the trivariate probit model are biased in very small samples (N=100). With moderately small samples (N=500), some of the approximations perform as well as the SML estimator when the degree of endogeneity is not very large. Some of the approximations seem robust with higher correlations and are also promising for testing the exogeneity of binary covariates. The methods are used to estimate the impact of employment-based health insurance and health care (HC) on HC use, where the approximations seem to work at least as well as the SML and in some cases better.     

Robust Hotelling's T2 control charts under non-normality: the case of t-Student distribution

Traditional multivariate control charts are based upon the assumption that the observations follow a multivariate normal distribution. In many practical applications, however, this supposition may be difficult to verify. In this paper, we use control charts based on robust estimators of location and scale to improve the capability of detection observations out of control under non-normality in the presence of multiple outliers. Concretely, we use a simulation process to analyse the behaviour of the robust alternatives to Hotelling's T ², which use minimum volume ellipsoidal (MVE) and minimum covariance determinant (MCD) in the presence of observations with a Student's t-distribution. The results show that these robust control charts are good alternatives for small deviations from normality due to the fact that the percentage of out-of-control observations detected for these charts in the Phase II are higher.