The Bayes rule of the variance parameter of the hierarchical normal and inverse gamma model under Stein's loss

For the variance parameter of the hierarchical normal and inverse gamma model, we analytically calculate the Bayes rule (estimator) with respect to a prior distribution IG (alpha, beta) under Stein's loss function. This estimator minimizes the posterior expected Stein's loss (PESL). We also analytically calculate the Bayes rule and the PESL under the squared error loss. Finally, the numerical simulations exemplify that the PESLs depend only on alpha and the number of observations. The Bayes rules and PESLs under Stein's loss are unanimously smaller than those under the squared error loss.     

Improved robust Bayes estimators of the error variance in linear models

We consider the problem of estimating the error variance in a general linear model when the error distribution is assumed to be spherically symmetric, but not necessary Gaussian. In particular we study the case of a scale mixture of Gaussians including the particularly important case of the multivariate-t distribution. Under Stein's loss, we construct a class of estimators that improve on the usual best unbiased (and best equivariant) estimator. Our class has the interesting double robustness property of being simultaneously generalized Bayes (for the same generalized prior) and minimax over the entire class of scale mixture of Gaussian distributions.     

Minimax estimation of normal precisions via expansion estimators

In this paper, the simultaneous estimation of the precision parameters of k normal distributions is considered under the squared loss function in a decision-theoretic framework. Several classes of minimax estimators are derived by using the chi-square identity, and the generalized Bayes minimax estimators are developed out of the classes. It is also shown that the improvement on the unbiased estimators is characterized by the superharmonic function. This corresponds to Stein's [1981. Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9, 1135–1151] result in simultaneous estimation of normal means.     

Generalized james-stein estimatoes

Let the p-component vector X be normally distributed with mean θ and covariance σ²I where I denotes the identity matrix. Stein's estimator of θ is kown to dominate the usual estimator X for p ≥ 3, We obtain a family of estimators which dominate Stein's estimator for p≥ 3     

The empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior under Stein's loss function

For the hierarchical Poisson and gamma model, we calculate the Bayes posterior estimator of the parameter of the Poisson distribution under Stein's loss function which penalizes gross overestimation and gross underestimation equally and the corresponding Posterior Expected Stein's Loss (PESL). We also obtain the Bayes posterior estimator of the parameter under the squared error loss and the corresponding PESL. Moreover, we obtain the empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior by two methods. In numerical simulations, we have illustrated: The two inequalities of the Bayes posterior estimators and the PESLs; the moment estimators and the Maximum Likelihood Estimators (MLEs) are consistent estimators of the hyperparameters; the goodness-of-fit of the model to the simulated data. The numerical results indicate that the MLEs are better than the moment estimators when estimating the hyperparameters. Finally, we exploit the attendance data on 314 high school juniors from two urban high schools to illustrate our theoretical studies.     

Minimax estimation of a lower-bounded scale parameter of a gamma distribution for scale-invariant squared-error loss

Let X have a gamma distribution with known shape parameter θr;aL and unknown scale parameter θ. Suppose it is known that θ ≥ a for some known a > 0. An admissible minimax estimator for scale-invariant squared-error loss is presented. This estimator is the pointwise limit of a sequence of Bayes estimators. Further, the class of truncated linear estimators C = {θ_ρ|θ_ρ(x) = max(a, ρ), ρ > 0} is studied. It is shown that each θ_ρ is inadmissible and that exactly one of them is minimax. Finally, it is shown that Katz's [Ann. Math. Statist., 32, 136–142 (1961)] estimator of θ is not minimax for our loss function. Some further properties of and comparisons among these estimators are also presented.     

Another look at Huber's estimator: A new minimax estimator in regression with stochastically bounded noise

Huber's estimator has had a long lasting impact, particularly on robust statistics. It is well known that under certain conditions, Huber's estimator is asymptotically minimax. A moderate generalization in rederiving Huber's estimator shows that Huber's estimator is not the only choice. We develop an alternative asymptotic minimax estimator and name it regression with stochastically bounded noise (RSBN). Simulations demonstrate that RSBN is slightly better in performance, although it is unclear how to justify such an improvement theoretically. We propose two numerical solutions: an iterative numerical solution, which is extremely easy to implement and is based on the proximal point method; and a solution by applying state-of-the-art nonlinear optimization software packages, e.g., SNOPT. Contribution: the generalization of the variational approach is interesting and should be useful in deriving other asymptotic minimax estimators in other problems.     

Asymptotically optimal shrinkage estimates for non-normal data

Motivated by several practical issues, we consider the problem of estimating the mean of a p-variate population (not necessarily normal) with unknown finite covariance. A quadratic loss function is used. We give a number of estimators (for the mean) with their loss functions admitting expansions to the order of p ^?1/2 as p→∞. These estimators contain Stein's [Inadmissibility of the usual estimator for the mean of a multivariate normal population, in Proceedings of the Third Berkeley Symposium in Mathematical Statistics and Probability, Vol. 1, J. Neyman, ed., University of California Press, Berkeley, 1956, pp. 197–206] estimate as a particular case and also contain ‘multiple shrinkage’ estimates improving on Stein's estimate. Finally, we perform a simulation study to compare the different estimates.     

Wavelet threshold based on Stein's unbiased risk estimators of restricted location parameter in multivariate normal

In this paper, the problem of estimating the mean vector under non-negative constraints on location vector of the multivariate normal distribution is investigated. The value of the wavelet threshold based on Stein''s unbiased risk estimators is calculated for the shrinkage estimator in restricted parameter space. We suppose that covariance matrix is unknown and we find the dominant class of shrinkage estimators under Balance loss function. The performance evaluation of the proposed class of estimators is checked through a simulation study by using risk and average mean square error values.     

Regularization through variable selection and conditional MLE with application to classification in high dimensions

It is often the case that high-dimensional data consist of only a few informative components. Standard statistical modeling and estimation in such a situation is prone to inaccuracies due to overfitting, unless regularization methods are practiced. In the context of classification, we propose a class of regularization methods through shrinkage estimators. The shrinkage is based on variable selection coupled with conditional maximum likelihood. Using Stein's unbiased estimator of the risk, we derive an estimator for the optimal shrinkage method within a certain class. A comparison of the optimal shrinkage methods in a classification context, with the optimal shrinkage method when estimating a mean vector under a squared loss, is given. The latter problem is extensively studied, but it seems that the results of those studies are not completely relevant for classification. We demonstrate and examine our method on simulated data and compare it to feature annealed independence rule and Fisher's rule.     

Estimation of a scale parameter in mixture models with unknown location

Estimation of the scale parameter in mixture models with unknown location is considered under Stein's loss. Under certain conditions, the inadmissibility of the “usual” estimator is established by exhibiting better estimators. In addition, robust improvements are found for a specified submodel of the original model. The results are applied to mixtures of normal distributions and mixtures of exponential distributions. Improved estimators of the variance of a normal distribution are shown to be robust under any scale mixture of normals having variance greater than the variance of that normal distribution. In particular, Stein's (Ann. Inst. Statist. Math. 16 (1964) 155) and Brewster's and Zidek's (Ann. Statist. 2 (1974) 21) estimators obtained under the normal model are robust under the t model, for arbitrary degrees of freedom, and under the double-exponential model. Improved estimators for the variance of a t distribution with unknown and arbitrary degrees of freedom are also given. In addition, improved estimators for the scale parameter of the multivariate Lomax distribution (which arises as a certain mixture of exponential distributions) are derived and the robustness of Zidek's (Ann. Statist. 1 (1973) 264) and Brewster's (Ann. Statist. 2 (1974) 553) estimators of the scale parameter of an exponential distribution is established under a class of modified Lomax distributions.     

On unbiased and improved loss estimation for the mean of a multivariate normal distribution with unknown variance

There is now a sizeable literature dealing with point estimation using Stein-type estimators. As discussed in Rukhin (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 409–418), instances arise in practice in which an estimation rule is to be accompanied by an estimate of its loss, which is unobservable. In the context of estimating the mean vector of a multi-normal distribution assuming a known population variance, Johnstone (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 361–379) derived an estimator that dominates the unbiased estimator of the quadratic loss incurred by the James–Stein estimator. By applying the Stein's lemma, this note generalizes Johnstone's (In: Gupta, S.S., Berger, J.O. (Eds.), Statistical Decision Theory and Related Topics, Vol. IV, Springer, New York, pp. 361–379) analysis to the setting of the unknown population variance. Computational evidence is provided about the risk magnitude of loss estimators associated with the James–Stein point estimator and its positive-part version.     

Estimation of a multivariate normal covariance matrix under a certain structure

We consider the estimation of Σ of the p-dimensional normal distribution N_p (0, Σ) when Σ?=?θ₀ I_p ?+?θ₁ aa′, where a is an unknown p-dimensional normalized vector and θ₀?>?0, θ₁?≥?0 are also unknown. First, we derive the restricted maximum likelihood (REML) estimator. Second, we propose a new estimator, which dominates the REML estimator with respect to Stein's loss function. Finally, we carry out Monte Carlo simulation to investigate the magnitude of the new estimator's superiority.     

Minimax estimation of a probability of success under LINEX loss

Minimax estimation of a binomial probability under LINEX loss function is considered. It is shown that no equalizer estimator is available in the statistical decision problem under consideration. It is pointed out that the problem can be solved by determining the Bayes estimator with respect to a least favorable distribution having finite support. In this situation, the optimal estimator and the least favorable distribution can be determined only by using numerical methods. Some properties of the minimax estimators and the corresponding least favorable prior distributions are provided depending on the parameters of the loss function. The properties presented are exploited in computing the minimax estimators and the least favorable distributions. The results obtained can be applied to determine minimax estimators of a cumulative distribution function and minimax estimators of a survival function.     

Equivariant estimation under the pitman closeness criterion

For the point estimation in models with group structures, an invariance approach to deriving superior estimators is discussed in the Pitman closeness (PC) criterion. When the maximal invariant statistic is parameter-free, that is, ancillary, the closest equivariant estimator to the true value in the PC criterion is presented. On the other hand, as an example where a distribution of the maximalinvariant statistic depends on unknown parameters, the paper treats the Stein problem in estimation of a variance and obtains an improved estimator in the PC criterion by Stein's invariance approach. Also the Stein problem in simultaneous estimation of a location vector of a spherical symmetric distribution is studied.     

Minimax Estimation of the Bounded Parameter of Some Discrete Distributions Under LINEX Loss Function

For a class of discrete distributions, including Poisson(θ), Generalized Poisson(θ), Borel(m, θ), etc., we consider minimax estimation of the parameter θ under the assumption it lies in a bounded interval of the form [0, m] and a LINEX loss function. Explicit conditions for the minimax estimator to be Bayes with respect to a boundary supported prior are given. Also for Bernoulli(θ)-distribution, which is not in the mentioned class of discrete distributions, we give conditions for which the Bayes estimator of θ ∈ [0, m], m < 1 with respect to a boundary supported prior is minimax under LINEX loss function. Numerical values are given for the largest values of m for which the corresponding Bayes estimators of θ are minimax.     

An asymptotic expansion of Wishart distribution when the population eigenvalues are infinitely dispersed

Takemura and Sheena [A. Takemura, Y. Sheena, Distribution of eigenvalues and eigenvectors of Wishart matrix when the population eigenvalues are infinitely dispersed and its application to minimax estimation of covariance matrix, J. Multivariate Anal. 94 (2005) 271–299] derived the asymptotic joint distribution of the eigenvalues and the eigenvectors of a Wishart matrix when the population eigenvalues become infinitely dispersed. They also showed necessary conditions for an estimator of the population covariance matrix to be tail minimax for typical loss functions by calculating the asymptotic risk of the estimator. In this paper, we further examine those distributions and risks by means of an asymptotic expansion. We obtain the asymptotic expansion of the distribution function of relevant elements of the sample eigenvalues and eigenvectors. We also derive the asymptotic expansion of the risk function of a scale and orthogonally equivariant estimator with respect to Stein’s loss. As an application, we prove non-minimaxity of Stein’s and Haff’s estimators, which has been an open problem for a long time.     

Quasi-minimax estimation in the partial linear model

This article discusses the minimax estimator in partial linear model y = Zβ + f + ε under ellipsoidal restrictions on the parameter space and quadratic loss function. The superiority of the minimax estimator over the two-step estimator is studied in the mean squared error matrix criterion.     

Minimax estimation in linear regression model

Estimationof the coefficient vector of a linear regression model subject to ellipsoidal constraints on the coefficients has been considered. Shrinkage methodology has been synthesized with minimax estimation technique and an estimator based on this has been proposed. The properties of this estimator have been derived under a quadratic loss set-up of the decision theory which are analysed to assess the behaviour of the proposed estimator. Superiority conditions for the dominance of this estimator over the existing minimax and least-squares estimators have also been derived.     

An MSE‐reduced estimator for the response proportion in a two‐stage clinical trial

Two‐stage design is very useful in clinical trials for evaluating the validity of a specific treatment regimen. When the second stage is allowed to continue, the method used to estimate the response rate based on the results of both stages is critical for the subsequent design. The often‐used sample proportion has an evident upward bias. However, the maximum likelihood estimator or the moment estimator tends to underestimate the response rate. A mean‐square error weighted estimator is considered here; its performance is thoroughly investigated via Simon's optimal and minimax designs and Shuster's design. Compared with the sample proportion, the proposed method has a smaller bias, and compared with the maximum likelihood estimator, the proposed method has a smaller mean‐square error. Copyright © 2010 John Wiley & Sons, Ltd.