M-Estimators Based on Inverse Probability Weighted Estimating Equations with Response Missing at Random

Asymptotic properties of M-estimators with complete data are investigated extensively. In the presence of missing data, however, the standard inference procedures for complete data cannot be applied directly. In this article, the inverse probability weighted method is applied to missing response problem to define M-estimators. The existence of M-estimators is established under very general regularity conditions. Consistency and asymptotic normality of the M-estimators are proved, respectively. An iterative algorithm is applied to calculating the M-estimators. It is shown that one step iteration suffices and the resulting one-step M-estimate has the same limit distribution as in the fully iterated M-estimators.     

Robust M-estimators of multivariate location and scatter in the presence of asymmetry

Robust estimation of location vectors and scatter matrices is studied under the assumption that the unknown error distribution is spherically symmetric in a central region and completely unknown in the tail region. A precise formulation of the model is given, an analysis of the identifiable parameters in the model is presented, and consistent initial estimators of the identifiable parameters are constructed. Consistent and asymptotically normal M-estimators are constructed (solved iteratively beginning with the initial estimates) based on “influence functions” which vanish outside specified compact sets. Finally M-estimators which are asymptotically minimax (in the sense of Huber) are derived.     

L 1-estimation for varying coefficient models

The varying coefficient model is a useful extension of linear models and has many advantages in practical use. To estimate the unknown functions in the model, the kernel type with local linear least-squares (L ₂) estimation methods has been proposed by several authors. When the data contain outliers or come from population with heavy-tailed distributions, L ₁-estimation should yield better estimators. In this article, we present the local linear L ₁-estimation method and derive the asymptotic distributions of the L ₁-estimators. The simulation results for two examples, with outliers and heavy-tailed distribution, respectively, show that the L ₁-estimators outperform the L ₂-estimators.     

Local Linear Estimation for Spatiotemporal Models Based on Least Absolute Deviation

When the data contain outliers or come from population with heavy-tailed distributions, which appear very often in spatiotemporal data, the estimation methods based on least-squares (L₂) method will not perform well. More robust estimation methods are required. In this article, we propose the local linear estimation for spatiotemporal models based on least absolute deviation (L₁) and drive the asymptotic distributions of the L₁-estimators under some mild conditions imposed on the spatiotemporal process. The simulation results for two examples, with outliers and heavy-tailed distribution, respectively, show that the L₁-estimators perform better than the L₂-estimators.     

Asymptotic Normality of M-Estimators for Varying Coefficient Models with Longitudinal Data

This article considers a nonparametric varying coefficient regression model with longitudinal observations. The relationship between the dependent variable and the covariates is assumed to be linear at a specific time point, but the coefficients are allowed to change over time. A general formulation is used to treat mean regression, median regression, quantile regression, and robust mean regression in one setting. The local M-estimators of the unknown coefficient functions are obtained by local linear method. The asymptotic distributions of M-estimators of unknown coefficient functions at both interior and boundary points are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are derived. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

Multivariate τ-estimators for location and scatter

We discuss the robustness and asymptotic behaviour of τ-estimators for multivariate location and scatter. We show that τ-estimators correspond to multivariate M-estimators defined by a weighted average of redescending ψ-functions, where the weights are adaptive. We prove consistency and asymptotic normality under weak assumptions on the underlying distribution, show that τ-estimators have a high breakdown point, and obtain the influence function at general distributions. In the special case of a location-scatter family, τ-estimators are asymptotically equivalent to multivariate S-estimators defined by means of a weighted ψ-function. This enables us to combine a high breakdown point and bounded influence with good asymptotic efficiency for the location and covariance estimator.     

M-estimation in regression models for censored data

In this paper, we study M-estimators of regression parameters in semiparametric linear models for censored data. A class of consistent and asymptotically normal M-estimators is constructed. A resampling method is developed for the estimation of the asymptotic covariance matrix of the estimators.     

Robust M-estimators of scale: Minimax bias versus maximal variance

In the location-scale estimation problem, we study robustness properties of M-estimators of the scale parameter under unknown ?-contamination of a fixed symmetric unimodal error distribution F₀. Within a general class of M-estimators, the estimator with minimax asymptotic bias is shown to lie within the subclass of α-interquantile ranges of the empirical distribution symmetrized about the sample median. Our main result is that as ? → 0, the limiting minimax asymptotic bias estimator is sometimes (e.g., when F_o is Cauchy), but not always, the median absolute deviation about the median. It is also shown that contamination in the neighbourhood of a discontinuity of the influence function of a minimax bias estimator can sometimes inflate the asymptotic variance beyond that achieved by placing all the ?-contamination at infinity. This effect is quantified by a new notion of asymptotic efficiency that takes into account the effect of infinitesimal contamination of the parametric model for the error distribution.     

Asymptotic properties of one-step M-estimators

We study the asymptotic behavior of one-step M-estimators based on not necessarily independent identically distributed observations. In particular, we find conditions for asymptotic normality of these estimators. Asymptotic normality of one-step M-estimators is proven under a wide spectrum of constraints on the exactness of initial estimators. We discuss the question of minimal restrictions on the exactness of initial estimators. We also discuss the asymptotic behavior of the solution to an M-equation closest to the parameter under consideration. As an application, we consider some examples of one-step approximation of quasi-likelihood estimators in nonlinear regression.     

Comparisons of asymptotic biases and variances of m-estimators of scale under asymmetric contamination

We study robustness properties of two types of M-estimators of scale when both location and scale parameters are unknown: (i) the scale estimator arising from simultaneous M-estimation of location and scale; and (ii) its symmetrization about the sample median. The robustness criteria considered are maximal asymptotic bias and maximal asymptotic variance when the known symmetric unimodal error distribution is subject to unknown, possibly asymmetric, £-con-tamination. Influence functions and asymptotic variance functionals are derived, and computations of asymptotic biases and variances, under the normal distribution with ε-contamination at oo, are presented for the special subclass arising from Huber's Proposal 2 and its symmetrized version. Symmetrization is seen to reduce both asymptotic bias and variance. Some complementary theoretical results are obtained, and the tradeoff between asymptotic bias and variance is discussed.     

Minimax Asymptotic Mean-Squared-Error of L-Estimators of Scale Parameter

We derive the AMSE (maximal asymptotic mean-squared-error) of the general class of L-estimators of scale that are location-scale equivariant and Fisher consistent. For non-normal error distributions, we determined estimators that have minimum AMSE over the subclass of (i) α-interquantile ranges and (ii) mixtures of at most two α-interquantile ranges. Finally, the L-estimators of scale symmetrized about the median were found to have the same AMSE as their nonsymmetrized counterparts, thus yielding the same results as in the symmetrized case.     

Consistency and Normality of M-Estimators in Partly Linear Models with Stochastic Adapted Errors

The robust M-estimators for the partly linear model under stochastic adapted errors are considered. It is shown that the M-estimator of parameter is asymptotically normal and the M-estimator of the nonparametric function achieves the optimal rate of convergence for nonparametric regression. Some known results are improved and generalized. Some simulations and a real data example are conducted to illustrate the proposed method.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

On the relation between Hodges-Lehmann efficiency and Pitman efficiency

The relation between Hodges–Lehmann efficiency and Pitman efficiency is studied in the context of testing one-sided hypotheses about a real-valued parameter. It is first shown that for tests based on sums of independently and identically distributed observations local Hodges-Lehmann efficiency is equivalent to Pitman efficiency. Then, it is proved that this equivalence also carries over to tests based on two broad classes of M -estimators for the location problem. In all cases considered explicit formulas of the Hodges-Lehmann efficacies are obtained.     

M-Estimator of a Generalized Linear Model with Measurement Errors

In this article, we propose a generalized linear model and estimate the unknown parameters using robust M-estimator. Under suitable conditions and by the strong law of large numbers and central limits theorem, the proposed M-estimators are proved to be consistent and asymptotically normal. We also evaluate the finite sample performance of our estimator through a Monte Carlo study.     

De l'unicité des estimateurs robustes en régression lorsque le paramètre d'échelle et le paramètre de la régression sont estimés simultanément

Several methods have been suggested to calculate robust M- and G-M -estimators of the regression parameter β and of the error scale parameter σ in a linear model. This paper shows that, for some data sets well known in robust statistics, the nonlinear systems of equations for the simultaneous estimation of β, with an M-estimate with a redescending ψ-function, and σ, with the residual median absolute deviation (MAD), have many solutions. This multiplicity is not caused by the possible lack of uniqueness, for redescending ψ-functions, of the solutions of the system defining β with known σ; rather, the simultaneous estimation of β and σ together creates the problem. A way to avoid these multiple solutions is to proceed in two steps. First take σ as the median absolute deviation of the residuals for a uniquely defined robust M-estimate such as Huber's Proposal 2 or the L₁-estimate. Then solve the nonlinear system for the M-estimate with σ equal to the value obtained at the first step to get the estimate of β. Analytical conditions for the uniqueness of M and G-M-estimates are also given.     

Redescending M-estimators

In finite sample studies redescending M-estimators outperform bounded M-estimators (see for example, Andrews et al. [1972. Robust Estimates of Location. Princeton University Press, Princeton]). Even though redescenders arise naturally out of the maximum likelihood approach if one uses very heavy-tailed models, the commonly used redescenders have been derived from purely heuristic considerations. Using a recent approach proposed by Shurygin, we study the optimality of redescending M-estimators. We show that redescending M-estimator can be designed by applying a global minimax criterion to locally robust estimators, namely maximizing over a class of densities the minimum variance sensitivity over a class of estimators. As a particular result, we prove that Smith's estimator, which is a compromise between Huber's skipped mean and Tukey's biweight, provides a guaranteed level of an estimator's variance sensitivity over the class of densities with a bounded variance.     

On the trimmed mean and minimax-variance L-estimation in Kolmogorov neighbourhoods

We consider the properties of the trimmed mean, as regards minimax-variance L-estimation of a location parameter in a Kolmogorov neighbourhood K() of the normal distribution: We first review some results on the search for an L-minimax estimator in this neighbourhood, i.e. a linear combination of order statistics whose maximum variance in K_t() is a minimum in the class of L-estimators. The natural candidate – the L-estimate which is efficient for that member of K_t,() with minimum Fisher information – is known not to be a saddlepoint solution to the minimax problem. We show here that it is not a solution at all. We do this by showing that a smaller maximum variance is attained by an appropriately trimmed mean. We argue that this trimmed mean, as well as being computationally simple – much simpler than the efficient L-estimate referred to above, and simpler than the minimax M- and R-estimators – is at least “nearly” minimax.     

Robust inference for the stress–strength reliability

We address the problem of robust inference about the stress–strength reliability parameter R = P(X < Y), where X and Y are taken to be independent random variables. Indeed, although classical likelihood based procedures for inference on R are available, it is well-known that they can be badly affected by mild departures from model assumptions, regarding both stress and strength data. The proposed robust method relies on the theory of bounded influence M-estimators. We obtain large-sample test statistics with the standard asymptotic distribution by means of delta-method asymptotics. The finite sample behavior of these tests is investigated by some numerical studies, when both X and Y are independent exponential or normal random variables. An illustrative application in a regression setting is also discussed.