Profile Hellinger distance estimation

The successful application of the Hellinger distance approach to fully parametric models is well known. The corresponding optimal estimators, known as minimum Hellinger distance (MHD) estimators, are efficient and have excellent robustness properties [Beran R. Minimum Hellinger distance estimators for parametric models. Ann Statist. 1977;5:445–463]. This combination of efficiency and robustness makes MHD estimators appealing in practice. However, their application to semiparametric statistical models, which have a nuisance parameter (typically of infinite dimension), has not been fully studied. In this paper, we investigate a methodology to extend the MHD approach to general semiparametric models. We introduce the profile Hellinger distance and use it to construct a minimum profile Hellinger distance estimator of the finite-dimensional parameter of interest. This approach is analogous in some sense to the profile likelihood approach. We investigate the asymptotic properties such as the asymptotic normality, efficiency, and adaptivity of the proposed estimator. We also investigate its robustness properties. We present its small-sample properties using a Monte Carlo study.     

Robust minimum distance inference based on combined distances

The minimum disparity estimators proposed by Lindsay (1994) for discrete models form an attractive subclass of minimum distance estimators which achieve their robustness without sacrificing first order efficiency at the model. Similarly, disparity test statistics are useful robust alternatives to the likelihood ratio test for testing of hypotheses in parametric models; they are asymptotically equivalent to the likelihood ratio test statistics under the null hypothesis and contiguous alternatives. Despite their asymptotic optimality properties, the small sample performance of many of the minimum disparity estimators and disparity tests can be considerably worse compared to the maximum likelihood estimator and the likelihood ratio test respectively. In this paper we focus on the class of blended weight Hellinger distances, a general subfamily of disparities, and study the effects of combining two different distances within this class to generate the family of “combined” blended weight Hellinger distances, and identify the members of this family which generally perform well. More generally, we investigate the class of "combined and penal-ized" blended weight Hellinger distances; the penalty is based on reweighting the empty cells, following Harris and Basu (1994). It is shown that some members of the combined and penalized family have rather attractive properties     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

On minimum Hellinger distance estimation

Efficiency and robustness are two fundamental concepts in parametric estimation problems. It was long thought that there was an inherent contradiction between the aims of achieving robustness and efficiency; that is, a robust estimator could not be efficient and vice versa. It is now known that the minimum Hellinger distance approached introduced by Beran [R. Beran, Annals of Statistics 1977;5:445–463] is one way of reconciling the conflicting concepts of efficiency and robustness. For parametric models, it has been shown that minimum Hellinger estimators achieve efficiency at the model density and simultaneously have excellent robustness properties. In this article, we examine the application of this approach in two semiparametric models. In particular, we consider a two‐component mixture model and a two‐sample semiparametric model. In each case, we investigate minimum Hellinger distance estimators of finite‐dimensional Euclidean parameters of particular interest and study their basic asymptotic properties. Small sample properties of the proposed estimators are examined using a Monte Carlo study. The results can be extended to semiparametric models of general form as well. The Canadian Journal of Statistics 37: 514–533; 2009 © 2009 Statistical Society of Canada     

Minimum Hellinger distance estimation for randomized play the winner design

Response-adaptive designs in clinical trials incorporate information from prior patient responses in order to assign better performing treatments to the future patients of a clinical study. An example of a response adaptive design that has received much attention in recent years is the randomized play the winner design (RPWD). Beran [1977. Minimum Hellinger distance estimates for parametric models. Ann. Statist. 5, 445–463] investigated the problem of minimum Hellinger distance procedure (MHDP) for continuous data and showed that minimum Hellinger distance estimator (MHDE) of a finite dimensional parameter is as efficient as the MLE (maximum likelihood estimator) under a true model assumption. This paper develops minimum Hellinger distance methodology for data generated using RPWD. A new algorithm using the Monte Carlo approximation to the estimating equation is proposed. Consistency and asymptotic normality of the estimators are established and the robustness and small sample performance of the estimators are illustrated using simulations. The methodology when applied to the clinical trial data conducted by Eli-Lilly and Company, brings out the treatment effect in one of the strata using the frequentist techniques compared to the Bayesian argument of Tamura et al [1994. A case study of an adaptive clinical trialin the treatment of out-patients with depressive disorder. J. Amer. Statist. Assoc. 89, 768–776].     

ROBUST INFERENCE IN PARAMETRIC MODELS USING THE FAMILY OF GENERALIZED NEGATIVE EXPONENTIAL DISPARITIES

We examine robust estimators and tests using the family of generalized negative exponential disparities, which contains the Pearson's chi‐square and the ordinary negative exponential disparity as special cases. The influence function and α‐influence function of the proposed estimators are discussed and their breakdown points derived. Under the model, the estimators are asymptotically efficient, and are shown to have an asymptotic breakdown point of 50%. The proposed tests are shown to be equivalent to the likelihood ratio test under the null hypothesis, and their breakdown points are obtained. The competitive performance of the proposed estimators and tests relative to those based on the Hellinger distance is illustrated through examples and simulation results. Unlike the Hellinger distance, several members of this family of generalized negative exponential disparities generate estimators which also possess excellent inlier‐controlling capability. The corresponding tests of hypothesis are shown to have better power breakdown than the Hellinger deviance test in the cases examined.     

A general lower bound of minimax risk for absolute-error loss

A general lower bound of minimax risk for absolute-error loss is given in terms of the Hellinger modulus of the estimation problem. The main results are applicable to various parametric, semi-parametric and nonparametric problems. Two examples of parametric estimation problems and two examples of density estimation problems are given. In all of these examples, the general lower bound achieves the convergence rates of minimax risk.     

The shapes of things to come: probability density quantiles

For every discrete or continuous location-scale family having a square-integrable density, there is a unique continuous probability distribution on the unit interval that is determined by the density-quantile composition introduced by Parzen in 1979. These probability density quantiles (pdQs) only differ in shape, and can be usefully compared with the Hellinger distance or Kullback–Leibler divergences. Convergent empirical estimates of these pdQs are provided, which leads to a robust global fitting procedure of shape families to data. Asymmetry can be measured in terms of distance or divergence of pdQs from the symmetric class. Further, a precise classification of shapes by tail behaviour can be defined simply in terms of pdQ boundary derivatives.     

Hellinger distances and α-entropy in a one-parameter class of density functions

Hellinger distances are considered as measures of distance between density functions, and an inequality concerning different Hellinger distances is proved. Distance measures based on the α-entropy are proposed, and their relationship to a Hellinger distance is shown. Furthermore explicit expressions for the distance measures examined are derived in a one—parameter class of density functions, including Weibull, Gamma, and Maxwell distributions.     

Minimum disparity estimation based on combined disparities: Asymptotic results

The maximum likelihood estimator (MLE) is asymptotically efficient for most parametric models under standard regularity conditions, but it has very poor robustness properties. On the other hand some of the minimum disparity estimators like the minimum Hellinger distance estimator (MHDE) have strong robustness features but their small sample efficiency at the model turns out to be very poor compared to the MLE. Methods based on the minimization of some combined disparities can substantially improve their small sample performances without affecting their robustness properties (Park et al., 1995). All studies involving the combined disparity have so far been empirical, and there are no results on the asymptotic properties of these estimators. In view of the usefulness of these procedures this is a major gap in theory, which we try to fill through the present work. Some illustrations of the performance of the estimators and the corresponding tests are also provided.     

On the behaviour of tests based on sample spacings for moderatesamples

We revisit the question about optimal performance of goodness-of-fit tests based on sample spacings. We reveal the importance of centering of the test-statistic and of the sample size when choosing a suitable test-statistic from a family of statistics based on power transformations of sample spacings. In particular, we find that a test-statistic based on empirical estimation of the Hellinger distance between hypothetical and data-supported distribution does possess some optimality properties for moderate sample sizes. These findings confirm earlier statements about the robust behaviour of the test-statistic based on the Hellinger distance and are in contrast to findings about the asymptotic (when sample size approaches infinity) of statistics such as Moran's and/or Greenwood's statistic. We include simulation results that support our findings.     

Asymptotic expansion of the risk of maximum likelihood estimator with respect to α-divergence

For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to α-divergence, which includes the special cases of Kullback–Leibler divergence, the Hellinger distance, and essentially χ²-divergence. The asymptotic expansion of the risk is given with respect to sample sizes up to order n^{? 2}. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model.     

Distance-based variable generation with applications to the FACT experiment

We introduce a new way to construct variables for classification in a setting of astronomy. The newly constructed variables complement the currently used Hillas parameters and are specifically designed to improve the classification. They are based on fitting elliptic or skewed bivariate distributions to images gathered by imaging atmospheric Cherenkov telescopes and evaluating the distance between the observed and the fitted distribution. As distance measures we use the Chi-square distance, the Kullback–Leibler divergence and the Hellinger distance. The new variables lead to an improved classification in terms of misclassification errors.     

Hellinger distance as a penalized log likelihood

The present paper studies the minimum Hellinger distance estimator by recasting it as the maximum likelihood estimator in a data driven modification of the model density. In the process, the Hellinger distance itself is expressed as a penalized log likelihood function. The penalty is the sum of the model probabilities over the non-observed values of the sample space. A comparison of the modified model density with the original data provides insights into the robustness of the minimum Hellinger distance estimator. Adjustments of the amount of penalty leads to a class of minimum penalized Hellinger distance estimators, some members of which perform substantially better than the minimum Hellinger distance estimator at the model for small samples, without compromising the robustness properties of the latter.     

Fixed-width confidence interval based on a minimum Hellinger distance estimator

In the context of discrete data, a sequential fixed-width confidence interval for an unknown parameter in a parametric model is constructed using a minimum Hellinger distance estimator (MHD) as the center of the interval. It is shown that our sequential procedure is asymptotically consistent and efficient, when the assumed parametric model is correct. These results, in addition to being exactly same as those obtained by Khan [1969, A general method of determining fixed-width confidence intervals. Ann. Math. Statist. 40, 704–709] and Yu [1989, On fixed-width confidence intervals associated with maximum likelihood estimation. J. Theoret. Probab. 2, 193–199] using a maximum likelihood estimator (MLE), offer an alternative which has several in-built robustness properties. Monte Carlo simulations show that the performance of our sequential procedure based on MHD, measured in terms of average sample size and the coverage probability, are as good as those based on MLE, when the assumed Poisson model is correct. However, when the samples come from a gross-error contaminated Poisson model, our numerical results show that the deviation from the Poisson model assumption severely affects the performance of the sequential procedure based on MLE, while the procedure based on MHD continues to perform well, thus exhibiting robustness of MHD against gross-error contaminations even for random sample sizes.     

A Hellinger distance approach to MCMC diagnostics

Bayesian analysis often requires the researcher to employ Markov Chain Monte Carlo (MCMC) techniques to draw samples from a posterior distribution which in turn is used to make inferences. Currently, several approaches to determine convergence of the chain as well as sensitivities of the resulting inferences have been developed. This work develops a Hellinger distance approach to MCMC diagnostics. An approximation to the Hellinger distance between two distributions f and g based on sampling is introduced. This approximation is studied via simulation to determine the accuracy. A criterion for using this Hellinger distance for determining chain convergence is proposed as well as a criterion for sensitivity studies. These criteria are illustrated using a dataset concerning the Anguilla australis, an eel native to New Zealand.     

Density estimation under the koziol-green model of censoring by penalized likelihood methods

We propose two density estimators of the survival distribution in the setting of the Koziol-Green random-censoring model. The estimators are obtained by maximum-penalized-likelihood methods, and we provide an algorithm for their numerical evaluation. We establish the strong consistency of the estimators in the Hellinger metric, the L_p-norms, p= 1,2, ∞, and a Sobolev norm, under mild conditions on the underlying survival density and the censoring distribution.     

Bayes and maximum likelihood for $$L^1$$-Wasserstein deconvolution of Laplace mixtures

We consider the problem of recovering a distribution function on the real line from observations additively contaminated with errors following the standard Laplace distribution. Assuming that the latent distribution is completely unknown leads to a nonparametric deconvolution problem. We begin by studying the rates of convergence relative to the \(L^2\)-norm and the Hellinger metric for the direct problem of estimating the sampling density, which is a mixture of Laplace densities with a possibly unbounded set of locations: the rate of convergence for the Bayes’ density estimator corresponding to a Dirichlet process prior over the space of all mixing distributions on the real line matches, up to a logarithmic factor, with the \(n^{-3/8}\log ^{1/8}n\) rate for the maximum likelihood estimator. Then, appealing to an inversion inequality translating the \(L^2\)-norm and the Hellinger distance between general kernel mixtures, with a kernel density having polynomially decaying Fourier transform, into any \(L^p\)-Wasserstein distance, \(p\ge 1\), between the corresponding mixing distributions, provided their Laplace transforms are finite in some neighborhood of zero, we derive the rates of convergence in the \(L^1\)-Wasserstein metric for the Bayes’ and maximum likelihood estimators of the mixing distribution. Merging in the \(L^1\)-Wasserstein distance between Bayes and maximum likelihood follows as a by-product, along with an assessment on the stochastic order of the discrepancy between the two estimation procedures.     

Estimation for u-shaped beta distributions: minimum hellinger distance and related methods

We compare minimum Hellinger distance and minimum Heiiinger disparity estimates for U-shaped beta distributions. Given suitable density estimates, both methods are known to be asymptotically efficient when the data come from the assumed model family, and robust to small perturbations from the model family. Most implementations use kernel density estimates, which may not be appropriate for U-shaped distributions. We compare fixed binwidth histograms, percentile mesh histograms, and averaged shifted histograms. Minimum disparity estimates are less sensitive to the choice of density estimate than are minimum distance estimates, and the percentile mesh histogram gives the best results for both minimum distance and minimum disparity estimates. Minimum distance estimates are biased and a bias-corrected method is proposed. Minimum disparity estimates and bias-corrected minimum distance estimates are comparable to maximum likelihood estimates when the model holds, and give better results than either method of moments or maximum likelihood when the data are discretized or contaminated, Although our re¬sults are for the beta density, the implementations are easily modified for other U-shaped distributions such as the Dirkhlet or normal generated distribution.     

Hellinger distance estimation of general bilinear time series models

In the present paper, minimum Hellinger distance estimates for parameters of a bilinear time series model are presented. The probabilistic properties such as stationarity, existence of moments of the stationary distribution and strong mixing property of the model are well known (see for instance [J. Liu, A note on causality and invertibility of a general bilinear time series model, Adv. Appl. Probab. 22 (1990) 247–250; J. Liu, P.J. Brockwell, On the general bilinear time series model, J. Appl. Probab. 25 (1988) 553–564; D.T. Pham, The mixing property of bilinear and generalised random coefficients autoregressive models, Stoch. Process Appl. 23 (1986) 291–300]). We establish, under some mild conditions, the consistency and the asymptotic normality of the minimum Hellinger distance estimates of the parameters of the model.