Robust M-estimators of scale: Minimax bias versus maximal variance

In the location-scale estimation problem, we study robustness properties of M-estimators of the scale parameter under unknown ?-contamination of a fixed symmetric unimodal error distribution F₀. Within a general class of M-estimators, the estimator with minimax asymptotic bias is shown to lie within the subclass of α-interquantile ranges of the empirical distribution symmetrized about the sample median. Our main result is that as ? → 0, the limiting minimax asymptotic bias estimator is sometimes (e.g., when F_o is Cauchy), but not always, the median absolute deviation about the median. It is also shown that contamination in the neighbourhood of a discontinuity of the influence function of a minimax bias estimator can sometimes inflate the asymptotic variance beyond that achieved by placing all the ?-contamination at infinity. This effect is quantified by a new notion of asymptotic efficiency that takes into account the effect of infinitesimal contamination of the parametric model for the error distribution.     

An asymptotic expansion of Wishart distribution when the population eigenvalues are infinitely dispersed

Takemura and Sheena [A. Takemura, Y. Sheena, Distribution of eigenvalues and eigenvectors of Wishart matrix when the population eigenvalues are infinitely dispersed and its application to minimax estimation of covariance matrix, J. Multivariate Anal. 94 (2005) 271–299] derived the asymptotic joint distribution of the eigenvalues and the eigenvectors of a Wishart matrix when the population eigenvalues become infinitely dispersed. They also showed necessary conditions for an estimator of the population covariance matrix to be tail minimax for typical loss functions by calculating the asymptotic risk of the estimator. In this paper, we further examine those distributions and risks by means of an asymptotic expansion. We obtain the asymptotic expansion of the distribution function of relevant elements of the sample eigenvalues and eigenvectors. We also derive the asymptotic expansion of the risk function of a scale and orthogonally equivariant estimator with respect to Stein’s loss. As an application, we prove non-minimaxity of Stein’s and Haff’s estimators, which has been an open problem for a long time.     

Efficiency of a Class of Unbiased Estimators for the Invariant Distribution Function of a Diffusion Process

We consider the problem of the estimation of the invariant distribution function of an ergodic diffusion process when the drift coefficient is unknown. The empirical distribution function is a natural estimator which is unbiased, uniformly consistent and efficient in different metrics. Here we study the properties of optimality for another kind of estimator recently proposed. We consider a class of unbiased estimators and we show that they are also efficient in the sense that their asymptotic risk, defined as the integrated mean square error, attains the same asymptotic minimax lower bound of the empirical distribution function.     

Asymptotic Minimax Risk for the White Noise Model on the Sphere

Estimation of an unknown function on the unit sphere of the Euclidean space is considered. The function is observed in Gaussian continuous time white noise. Uniform norm is chosen as a loss function and exact asymptotic minimax risk is derived extending the result of Korostelev (1993). The exact asymptotic minimax risk is also given for the L ₂-loss, applying the result of Pinsker (1980).     

On MISE of a Non linear Wavelet Estimator of the Regression Function Based on Biased Data under Strong Mixing

In this paper, we consider the adaptation of the non linear wavelet-based estimator of the regression function for the biased data setup under strong mixing. We provide an asymptotic sharp bound for the mean integrated squared error (MISE) of the estimator, that is nearly optimal in the minimax sense over a large range of Besov function classes.     

Compromise Pitman estimators

In this paper, we investigate the construction of compromise estimators of location and scale, by averaging over several models selected among a specified large set of possible models. The weight given to each distribution is based on the profile likelihood, which leads to a notion of distance between distributions as we study the asymptotic behaviour of such estimators. The selection of the models is made in a minimax way, in order to choose distributions that are close to any possible distribution. We also present simulation results of such compromise estimators based on contaminated Gaussian and Student's t distributions.     

Minimax-variance L- and R-estimators of location

We consider the problem of minimax-variance, robust estimation of a location parameter, through the use of L- and R-estimators. We derive an easily checked necessary condition for L-estimation to be minimax, and a related sufficient condition for R-estimation to be minimax. Those cases in the literature in which L-estimation is known not to be minimax, and those in which R-estimation is minimax, are derived as consequences of these conditions. New classes of examples are given in each case. As well, we answer a question of Scholz (1974), who showed essentially that the asymptotic variance of an R-estimator never exceeds that of an L-estimator, if both are efficient at the same strongly unimodal distribution. Scholz raised the question of whether or not the assumption of strong unimodality could be dropped. We answer this question in the negative, theoretically and by examples. In the examples, the minimax property fails both for L-estimation and for R-estimation, but the variance of the L-estimator, as the distribution of the observation varies over the given neighbourhood, remains unbounded. That of the R-estimator is unbounded.     

speed of covergence to the extreme value distributions on their probability ploting parers

The speed of convergence of the distribution of the normalized maximum, of a sample of independent and identically distributed random variables, to its asymptotic distribution is considered in this article. Assuming that the cumulative distribution function of the random variables is known, the error committed replacing the actual distribution of the normalized maximum by its asympotic distribution is studied. Instead of using the arithmetical scale of probabilities, we measure the difference between the actual and asympotic distribution in terms of the double-log scale used for building the probability plotting paper for the the latter. We demonstrate that the difference between the double-log values corresponding to two probabilities in the upper tail is almost exactly equal to the logarithm of the distribution may not be uniform in this double-log scale and that the difference between the actual and asymptotic distributions, on the probebility plotting paper, may be a logarithmic, power, or even exponential function in the upper tail when the latter distribution is of Fisher-Tippett type I, but that difference is at most logarithmic in the upper tail for type II and III distributions. This fact is exploited to obtain transformed variables that converge tothe asymptotic distribution faster than the original variable on the probabilites plotting paper     

On Smooth Statistical Tail Functionals

Many estimators of the extreme value index of a distribution function F that are based on a certain number k _n of largest order statistics can be represented as a statistical tail function al, that is a functional T applied to the empirical tail quantile function Q _n. We study the asymptotic behaviour of such estimators with a scale and location invariant functional T under weak second order conditions on F . For that purpose first a new approximation of the empirical tail quantile function is established. As a consequence we obtain weak consistency and asymptotic normality of T ( Q _n) if T is continuous and Hadamard differentiable, respectively, at the upper quantile function of a generalized Pareto distribution and k _pn tends to infinity sufficiently slowly. Then we investigate the asymptotic variance and bias. In particular, those functionals T re characterized that lead to an estimator with minimal asymptotic variance. Finally, we introduce a method to construct estimators of the extreme value index with a made-to-order asymptotic behaviour     

A general method of finding a minimax estimator of a distribution function when no equalizer rule is available

We propose a feasible general method for finding a minimax estimator of an unknown distribution function F in the nonparametric problem. As an application, some minimax estimators are proposed. Furthermore, some minimax binomial parametric problems are studied.     

Optimal and Robust Designs for Estimating the Concentration Curve and the AUC

The problem of interest is to estimate the concentration curve and the area under the curve (AUC) by estimating the parameters of a linear regression model with an autocorrelated error process. We introduce a simple linear unbiased estimator of the concentration curve and the AUC. We show that this estimator constructed from a sampling design generated by an appropriate density is asymptotically optimal in the sense that it has exactly the same asymptotic performance as the best linear unbiased estimator. Moreover, we prove that the optimal design is robust with respect to a minimax criterion. When repeated observations are available, this estimator is consistent and has an asymptotic normal distribution. Finally, a simulated annealing algorithm is applied to a pharmacokinetic model with correlated errors.     

Another look at Huber's estimator: A new minimax estimator in regression with stochastically bounded noise

Huber's estimator has had a long lasting impact, particularly on robust statistics. It is well known that under certain conditions, Huber's estimator is asymptotically minimax. A moderate generalization in rederiving Huber's estimator shows that Huber's estimator is not the only choice. We develop an alternative asymptotic minimax estimator and name it regression with stochastically bounded noise (RSBN). Simulations demonstrate that RSBN is slightly better in performance, although it is unclear how to justify such an improvement theoretically. We propose two numerical solutions: an iterative numerical solution, which is extremely easy to implement and is based on the proximal point method; and a solution by applying state-of-the-art nonlinear optimization software packages, e.g., SNOPT. Contribution: the generalization of the variational approach is interesting and should be useful in deriving other asymptotic minimax estimators in other problems.     

Simultaneous robust estimation of location and scale parameters: A minimum-distance approach

Simultaneous robust estimates of location and scale parameters are derived from minimizing a minimum-distance criterion function. The criterion function measures the squared distance between the pth power (p > 0) of the empirical distribution function and the pth power of the imperfectly determined model distribution function over the real line. We show that the estimator is uniquely defined, is asymptotically bivariate normal and for p > 0.3 has positive breakdown. If the scale parameter is known, when p = 0.9 the asymptotic variance (1.0436) of the location estimator for the normal model is smaller than the asymptotic variance of the Hodges-Lehmann (HL)estimator (1.0472). Efficiencies with respect to HL and maximum-likelihood estimators (MLE) are 1.0034 and 0.9582, respectively. Similarly, if the location parameter is known, when p = 0.97 the asymptotic variance (0.6158) of the scale estimator is minimum. The efficiency with respect to the MLE is 0.8119. We show that the estimator can tolerate more corrupted observations at oo than at – for p < 1, and vice versa for p > 1.     

A Study of Blockwise Wavelet Estimates Via Lower Bounds for a Spike Function

Abstract.  A blockwise shrinkage is a popular adaptive procedure for non-parametric series estimates. It possesses an impressive range of asymptotic properties, and there is a vast pool of blocks and shrinkage procedures used. Traditionally these estimates are studied via upper bounds on their risks. This article suggests the study of these adaptive estimates via non-asymptotic lower bounds established for a spike underlying function that plays a pivotal role in the wavelet and minimax statistics. While upper-bound inequalities help the statistician to find sufficient conditions for a desirable estimation, the non-asymptotic lower bounds yield necessary conditions and shed a new light on the popular method of adaptation. The suggested method complements and knits together two traditional techniques used in the analysis of adaptive estimates: a numerical study and an asymptotic minimax inference.     

An Adaptive Efficient Test for Gumbel Domain of Attraction

We consider n independent observations, generated identically by some distribution function, which belongs to the domain of attraction of an extreme value distribution with unknown shape and scale parameter. We treat the scale parameter as a nuisance parameter and establish an adaptive efficient test sequence, which is based on the k _n largest observations, for the Gumbel domain of attraction. Efficiency is achieved along certain contiguous extreme value alternatives within the concept of local asymptotic normality (LAN). Simulations exemplify the results     

Bias bound for the minimax estimator

The bias bound function of an estimator is an important quantity in order to perform globally robust inference. We show how to evaluate the exact bias bound for the minimax estimator of the location parameter for a wide class of unimodal symmetric location and scale family. We show, by an example, how to obtain an upper bound of the bias bound for a unimodal asymmetric location and scale family. We provide the exact bias bound of the minimum distance/disparity estimators under a contamination neighborhood generated from the same distance.     

Efficiency of projected score methods in rectangular array asymptotics

Summary. The paper considers a rectangular array asymptotic embedding for multistratum data sets, in which both the number of strata and the number of within-stratum replications increase, and at the same rate. It is shown that under this embedding the maximum likelihood estimator is consistent but not efficient owing to a non-zero mean in its asymptotic normal distribution. By using a projection operator on the score function, an adjusted maximum likelihood estimator can be obtained that is asymptotically unbiased and has a variance that attains the Cramér–Rao lower bound. The adjusted maximum likelihood estimator can be viewed as an approximation to the conditional maximum likelihood estimator.     

Minimax estimation of a probability of success under LINEX loss

Minimax estimation of a binomial probability under LINEX loss function is considered. It is shown that no equalizer estimator is available in the statistical decision problem under consideration. It is pointed out that the problem can be solved by determining the Bayes estimator with respect to a least favorable distribution having finite support. In this situation, the optimal estimator and the least favorable distribution can be determined only by using numerical methods. Some properties of the minimax estimators and the corresponding least favorable prior distributions are provided depending on the parameters of the loss function. The properties presented are exploited in computing the minimax estimators and the least favorable distributions. The results obtained can be applied to determine minimax estimators of a cumulative distribution function and minimax estimators of a survival function.     

Conditional Score Approach to Errors‐in‐Variable Current Status Data Under the Proportional Odds Model

Abstract. The conditional score approach is proposed to the analysis of errors‐in‐variable current status data under the proportional odds model. Distinct from the conditional scores in other applications, the proposed conditional score involves a high‐dimensional nuisance parameter, causing challenges in both asymptotic theory and computation. We propose a composite algorithm combining the Newton–Raphson and self‐consistency algorithms for computation and develop an efficient conditional score, analogous to the efficient score from a typical semiparametric likelihood, for building an asymptotic linear expression and hence the asymptotic distribution of the conditional‐score estimator for the regression parameter. Our proposal is shown to perform well in simulation studies and is applied to a zebrafish basal cell carcinoma data involving measurement errors in gene expression levels.     

Bayesian inverse problems with heterogeneous variance

We consider inverse problems in Hilbert spaces under correlated Gaussian noise, and use a Bayesian approach to find their regularized solution. We focus on mildly ill-posed inverse problems with fractional noise, using a novel wavelet-based vaguelette–vaguelette approach. It allows us to apply sequence space methods without assuming that all operators are simultaneously diagonalizable. The results are proved for more general bases and covariance operators. Our primary aim is to study posterior contraction rate in such inverse problems over Sobolev classes and compare it to the derived minimax rate. Secondly, we study effect of plugging in a consistent estimator of variances in sequence space on the posterior contraction rate. This result is applied to the problem with error in forward operator. Thirdly, we show that empirical Bayes posterior distribution with a plugged-in maximum marginal likelihood estimator of the prior scale contracts at the optimal rate, adaptively, in the minimax sense.