On minimum Hellinger distance estimation

Efficiency and robustness are two fundamental concepts in parametric estimation problems. It was long thought that there was an inherent contradiction between the aims of achieving robustness and efficiency; that is, a robust estimator could not be efficient and vice versa. It is now known that the minimum Hellinger distance approached introduced by Beran [R. Beran, Annals of Statistics 1977;5:445–463] is one way of reconciling the conflicting concepts of efficiency and robustness. For parametric models, it has been shown that minimum Hellinger estimators achieve efficiency at the model density and simultaneously have excellent robustness properties. In this article, we examine the application of this approach in two semiparametric models. In particular, we consider a two‐component mixture model and a two‐sample semiparametric model. In each case, we investigate minimum Hellinger distance estimators of finite‐dimensional Euclidean parameters of particular interest and study their basic asymptotic properties. Small sample properties of the proposed estimators are examined using a Monte Carlo study. The results can be extended to semiparametric models of general form as well. The Canadian Journal of Statistics 37: 514–533; 2009 © 2009 Statistical Society of Canada     

Minimum disparity estimation based on combined disparities: Asymptotic results

The maximum likelihood estimator (MLE) is asymptotically efficient for most parametric models under standard regularity conditions, but it has very poor robustness properties. On the other hand some of the minimum disparity estimators like the minimum Hellinger distance estimator (MHDE) have strong robustness features but their small sample efficiency at the model turns out to be very poor compared to the MLE. Methods based on the minimization of some combined disparities can substantially improve their small sample performances without affecting their robustness properties (Park et al., 1995). All studies involving the combined disparity have so far been empirical, and there are no results on the asymptotic properties of these estimators. In view of the usefulness of these procedures this is a major gap in theory, which we try to fill through the present work. Some illustrations of the performance of the estimators and the corresponding tests are also provided.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.     

Robust minimum distance inference based on combined distances

The minimum disparity estimators proposed by Lindsay (1994) for discrete models form an attractive subclass of minimum distance estimators which achieve their robustness without sacrificing first order efficiency at the model. Similarly, disparity test statistics are useful robust alternatives to the likelihood ratio test for testing of hypotheses in parametric models; they are asymptotically equivalent to the likelihood ratio test statistics under the null hypothesis and contiguous alternatives. Despite their asymptotic optimality properties, the small sample performance of many of the minimum disparity estimators and disparity tests can be considerably worse compared to the maximum likelihood estimator and the likelihood ratio test respectively. In this paper we focus on the class of blended weight Hellinger distances, a general subfamily of disparities, and study the effects of combining two different distances within this class to generate the family of “combined” blended weight Hellinger distances, and identify the members of this family which generally perform well. More generally, we investigate the class of "combined and penal-ized" blended weight Hellinger distances; the penalty is based on reweighting the empty cells, following Harris and Basu (1994). It is shown that some members of the combined and penalized family have rather attractive properties     

Minimum Hellinger distance estimation for randomized play the winner design

Response-adaptive designs in clinical trials incorporate information from prior patient responses in order to assign better performing treatments to the future patients of a clinical study. An example of a response adaptive design that has received much attention in recent years is the randomized play the winner design (RPWD). Beran [1977. Minimum Hellinger distance estimates for parametric models. Ann. Statist. 5, 445–463] investigated the problem of minimum Hellinger distance procedure (MHDP) for continuous data and showed that minimum Hellinger distance estimator (MHDE) of a finite dimensional parameter is as efficient as the MLE (maximum likelihood estimator) under a true model assumption. This paper develops minimum Hellinger distance methodology for data generated using RPWD. A new algorithm using the Monte Carlo approximation to the estimating equation is proposed. Consistency and asymptotic normality of the estimators are established and the robustness and small sample performance of the estimators are illustrated using simulations. The methodology when applied to the clinical trial data conducted by Eli-Lilly and Company, brings out the treatment effect in one of the strata using the frequentist techniques compared to the Bayesian argument of Tamura et al [1994. A case study of an adaptive clinical trialin the treatment of out-patients with depressive disorder. J. Amer. Statist. Assoc. 89, 768–776].     

Semiparametric estimation in copula models

The author recalls the limiting behaviour of the empirical copula process and applies it to prove some asymptotic properties of a minimum distance estimator for a Euclidean parameter in a copula model. The estimator in question is semiparametric in that no knowledge of the marginal distributions is necessary. The author also proposes another semiparametric estimator which he calls “rank approximate Z‐estimator” and whose asymptotic normality he derives. He further presents Monte Carlo simulation results for the comparison of various estimators in four well‐known bivariate copula models.     

Hellinger distance as a penalized log likelihood

The present paper studies the minimum Hellinger distance estimator by recasting it as the maximum likelihood estimator in a data driven modification of the model density. In the process, the Hellinger distance itself is expressed as a penalized log likelihood function. The penalty is the sum of the model probabilities over the non-observed values of the sample space. A comparison of the modified model density with the original data provides insights into the robustness of the minimum Hellinger distance estimator. Adjustments of the amount of penalty leads to a class of minimum penalized Hellinger distance estimators, some members of which perform substantially better than the minimum Hellinger distance estimator at the model for small samples, without compromising the robustness properties of the latter.     

A note on the asymptotic relative efficiency of estimators from the additive hazards model

In this note, the asymptotic variance formulas are explicitly derived and compared between the parametric and semiparametric estimators of a regression parameter and survival probability under the additive hazards model. To obtain explicit formulas, it is assumed that the covariate term including a regression coefficient follows a gamma distribution and the baseline hazard function is constant. The results show that the semiparametric estimator of the regression coefficient parameter is fully efficient relative to the parametric counterpart when the survival time and a covariate are independent, as in the proportional hazards model. Relative to a more realistic case of the parametric additive hazards model with a Weibull baseline, the loss of efficiency of the semiparametric estimator of survival probability is moderate.     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

Semiparametric lower bounds for tail index estimation

We consider estimation of the tail index parameter from i.i.d. observations in Pareto and Weibull type models, using a local and asymptotic approach. The slowly varying function describing the non-tail behavior of the distribution is considered as an infinite dimensional nuisance parameter. Without further regularity conditions, we derive a local asymptotic normality (LAN) result for suitably chosen parametric submodels of the full semiparametric model. From this result, we immediately obtain the optimal rate of convergence of tail index parameter estimators for more specific models previously studied. On top of the optimal rate of convergence, our LAN result also gives the minimal limiting variance of estimators (regular for our parametric model) through the convolution theorem. We show that the classical Hill estimator is regular for the submodels introduced with limiting variance equal to the induced convolution theorem bound. We also discuss the Weibull model in this respect.     

Fixed-width confidence interval based on a minimum Hellinger distance estimator

In the context of discrete data, a sequential fixed-width confidence interval for an unknown parameter in a parametric model is constructed using a minimum Hellinger distance estimator (MHD) as the center of the interval. It is shown that our sequential procedure is asymptotically consistent and efficient, when the assumed parametric model is correct. These results, in addition to being exactly same as those obtained by Khan [1969, A general method of determining fixed-width confidence intervals. Ann. Math. Statist. 40, 704–709] and Yu [1989, On fixed-width confidence intervals associated with maximum likelihood estimation. J. Theoret. Probab. 2, 193–199] using a maximum likelihood estimator (MLE), offer an alternative which has several in-built robustness properties. Monte Carlo simulations show that the performance of our sequential procedure based on MHD, measured in terms of average sample size and the coverage probability, are as good as those based on MLE, when the assumed Poisson model is correct. However, when the samples come from a gross-error contaminated Poisson model, our numerical results show that the deviation from the Poisson model assumption severely affects the performance of the sequential procedure based on MLE, while the procedure based on MHD continues to perform well, thus exhibiting robustness of MHD against gross-error contaminations even for random sample sizes.     

Statistical inference for varying-coefficient partially linear errors-in-variables models with missing data

Abstract

The purpose of this paper is twofold. First, we investigate estimations in varying-coefficient partially linear errors-in-variables models with covariates missing at random. However, the estimators are often biased due to the existence of measurement errors, the bias-corrected profile least-squares estimator and local liner estimators for unknown parametric and coefficient functions are obtained based on inverse probability weighted method. The asymptotic properties of the proposed estimators both for the parameter and nonparametric parts are established. Second, we study asymptotic distributions of an empirical log-likelihood ratio statistic and maximum empirical likelihood estimator for the unknown parameter. Based on this, more accurate confidence regions of the unknown parameter can be constructed. The methods are examined through simulation studies and illustrated by a real data analysis.     

Robust Estimation for Zero‐Inflated Poisson Regression

Abstract. The zero‐inflated Poisson regression model is a special case of finite mixture models that is useful for count data containing many zeros. Typically, maximum likelihood (ML) estimation is used for fitting such models. However, it is well known that the ML estimator is highly sensitive to the presence of outliers and can become unstable when mixture components are poorly separated. In this paper, we propose an alternative robust estimation approach, robust expectation‐solution (RES) estimation. We compare the RES approach with an existing robust approach, minimum Hellinger distance (MHD) estimation. Simulation results indicate that both methods improve on ML when outliers are present and/or when the mixture components are poorly separated. However, the RES approach is more efficient in all the scenarios we considered. In addition, the RES method is shown to yield consistent and asymptotically normal estimators and, in contrast to MHD, can be applied quite generally.     

Semiparametric Estimation of Risk–Return Relationships

This article proposes semiparametric generalized least-squares estimation of parametric restrictions between the conditional mean and the conditional variance of excess returns given a set of parametric factors. A distinctive feature of our estimator is that it does not require a fully parametric model for the conditional mean and variance. We establish consistency and asymptotic normality of the estimates. The theory is nonstandard due to the presence of estimated factors. We provide sufficient conditions for the estimated factors not to have an impact in the asymptotic standard error of estimators. A simulation study investigates the finite sample performance of the estimates. Finally, an application to the CRSP value-weighted excess returns highlights the merits of our approach. In contrast to most previous studies using nonparametric estimates, we find a positive and significant price of risk in our semiparametric setting.     

样本选择模型的一个简单半参数估计量

本文发展了一个针对样本选择模型的两阶段半参数估计量,其首先在第一阶段基于对数欧几里得分布差异测度估计离散选择概率,进而在第二阶段利用非参数sieve方法估计一个包含参数和非参数部分的部分线性模型以得到模型参数的估计。相对于文献中已有的半参数估计量,该估计量的计算更加简便,且计算负担相对较小。我们说明了该半参数估计量的一致性和渐近正态性,同时给出了其渐近方差的计算公式。蒙特卡洛模拟结果符合我们的理论结论。     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

Minimum local distance density estimation

We present a local density estimator based on first-order statistics. To estimate the density at a point, x, the original sample is divided into subsets and the average minimum sample distance to x over all such subsets is used to define the density estimate at x. The tuning parameter is thus the number of subsets instead of the typical bandwidth of kernel or histogram-based density estimators. The proposed method is similar to nearest-neighbor density estimators but it provides smoother estimates. We derive the asymptotic distribution of this minimum sample distance statistic to study globally optimal values for the number and size of the subsets. Simulations are used to illustrate and compare the convergence properties of the estimator. The results show that the method provides good estimates of a wide variety of densities without changes of the tuning parameter, and that it offers competitive convergence performance.     

Seemingly Unrelated Ridge Regression in Semiparametric Models

This article is concerned with the problem of multicollinearity in the linear part of a seemingly unrelated semiparametric (SUS) model. It is also suspected that some additional non stochastic linear constraints hold on the whole parameter space. In the sequel, we propose semiparametric ridge and non ridge type estimators combining the restricted least squares methods in the model under study. For practical aspects, it is assumed that the covariance matrix of error terms is unknown and thus feasible estimators are proposed and their asymptotic distributional properties are derived. Also, necessary and sufficient conditions for the superiority of the ridge-type estimator over the non ridge type estimator for selecting the ridge parameter K are derived. Lastly, a Monte Carlo simulation study is conducted to estimate the parametric and nonparametric parts. In this regard, kernel smoothing and cross validation methods for estimating the nonparametric function are used.     

Efficient Estimation in Heteroscedastic Partially Linear Varying Coefficient Models

This article considers statistical inference for the heteroscedastic partially linear varying coefficient models. We construct an efficient estimator for the parametric component by applying the weighted profile least-squares approach, and show that it is semiparametrically efficient in the sense that the inverse of the asymptotic variance of the estimator reaches the semiparametric efficiency bound. Simulation studies are conducted to illustrate the performance of the proposed method.     

Minimum Hellinger distance estimators for multivariate distributions from the Johnson system

In this paper, we study minimum Hellinger distance estimators (MHDEs) for multivariate distributions from the Johnson system. We prove some properties of these estimators, such as consistency and asymptotic normality, and show that they represent a robust alternative for other existing estimators.