Minimax property of stein's estimator

Stein's estimator and some other estimators of the mean of a K-variate normal distribution are known to dominate the maximum likelihood estimator under quadratic loss for K > 3, and are therefore minimax. In this paper it is shown that the minimax property of Stein's rule is preserved with respect to a generalized loss function.     

Minimax estimation of a lower-bounded scale parameter of a gamma distribution for scale-invariant squared-error loss

Let X have a gamma distribution with known shape parameter θr;aL and unknown scale parameter θ. Suppose it is known that θ ≥ a for some known a > 0. An admissible minimax estimator for scale-invariant squared-error loss is presented. This estimator is the pointwise limit of a sequence of Bayes estimators. Further, the class of truncated linear estimators C = {θ_ρ|θ_ρ(x) = max(a, ρ), ρ > 0} is studied. It is shown that each θ_ρ is inadmissible and that exactly one of them is minimax. Finally, it is shown that Katz's [Ann. Math. Statist., 32, 136–142 (1961)] estimator of θ is not minimax for our loss function. Some further properties of and comparisons among these estimators are also presented.     

The Bayes rule of the variance parameter of the hierarchical normal and inverse gamma model under Stein's loss

For the variance parameter of the hierarchical normal and inverse gamma model, we analytically calculate the Bayes rule (estimator) with respect to a prior distribution IG (alpha, beta) under Stein's loss function. This estimator minimizes the posterior expected Stein's loss (PESL). We also analytically calculate the Bayes rule and the PESL under the squared error loss. Finally, the numerical simulations exemplify that the PESLs depend only on alpha and the number of observations. The Bayes rules and PESLs under Stein's loss are unanimously smaller than those under the squared error loss.     

Asymptotically optimal shrinkage estimates for non-normal data

Motivated by several practical issues, we consider the problem of estimating the mean of a p-variate population (not necessarily normal) with unknown finite covariance. A quadratic loss function is used. We give a number of estimators (for the mean) with their loss functions admitting expansions to the order of p ^?1/2 as p→∞. These estimators contain Stein's [Inadmissibility of the usual estimator for the mean of a multivariate normal population, in Proceedings of the Third Berkeley Symposium in Mathematical Statistics and Probability, Vol. 1, J. Neyman, ed., University of California Press, Berkeley, 1956, pp. 197–206] estimate as a particular case and also contain ‘multiple shrinkage’ estimates improving on Stein's estimate. Finally, we perform a simulation study to compare the different estimates.     

Simultaneous estimation of the multivariate normal mean under balanced loss function

This paper considers simultaneous estimation of multivariate normal mean vector using Zellner's(1994) balanced loss function which is defined as follows:

where 0 < w < 1 and for i = 1,…,p and j = 1,…,n, X_ij is distributed as normal with mean θi and variance 1. It is shown that the sample mean, X, is admissible when p <3. For p ≥3, we obtain that James-Stein type estimator which has uniformly smaller risk than that of sample mean X.     

An inclusion-consistent solution to the problem of absurd confidence statements: 2. Consistent nonexact-confidence-interval estimation

Two consistent nonexact-confidence-interval estimation methods, both derived from the consistency-equivalence theorem in Plante (1991), are suggested for estimation of problematic parametric functions with no consistent exact solution and for which standard optimal confidence procedures are inadequate or even absurd, i.e., can provide confidence statements with a 95% empty or all-inclusive confidence set. A belt C(·) from a consistent nonexact-belt family, used with two confidence coefficients (γ = inf_θ P_θ [ θ ? C(X)] and γ⁺ = sup_θ P_θ[θ ? C(X)], is shown to provide a consistent nonexact-belt solution for estimating μ₂ -μ₁ in the Behrens-Fisher problem. A rule for consistent behaviour enables any confidence belt to be used consistently by providing each sample point with best upper and lower confidence levels [δ⁺(x) ≥ γ⁺, δ(x) ≤ γ], which give least-conservative consistent confidence statements ranging from practically exact through informative to noninformative. The rule also provides a consistency correction L(x) = δ⁺(x)-δ(X) enabling alternative confidence solutions to be compared on grounds of adequacy; this is demonstrated by comparing consistent conservative sample-point-wise solutions with inconsistent standard solutions for estimating μ₂/μ₁ (Creasy-Fieller-Neyman problem) and $\sqrt {\mu _1^2 + \mu _2^2 }$, a distance-estimation problem closely related to Stein's 1959 example     

Some limitation on stein's phenomenon

We show that Stein's phenomenon is impossible for the following decision setting: the parameter space is compact, the loss function is the sum of coordinatewise strictly convex continuous loss functions, and the probability density function having the same support for every parameter θ is continuous in θ. Therefore, any coordinatewise admissible decision rule is admissible.     

Estimation of a scale parameter in mixture models with unknown location

Estimation of the scale parameter in mixture models with unknown location is considered under Stein's loss. Under certain conditions, the inadmissibility of the “usual” estimator is established by exhibiting better estimators. In addition, robust improvements are found for a specified submodel of the original model. The results are applied to mixtures of normal distributions and mixtures of exponential distributions. Improved estimators of the variance of a normal distribution are shown to be robust under any scale mixture of normals having variance greater than the variance of that normal distribution. In particular, Stein's (Ann. Inst. Statist. Math. 16 (1964) 155) and Brewster's and Zidek's (Ann. Statist. 2 (1974) 21) estimators obtained under the normal model are robust under the t model, for arbitrary degrees of freedom, and under the double-exponential model. Improved estimators for the variance of a t distribution with unknown and arbitrary degrees of freedom are also given. In addition, improved estimators for the scale parameter of the multivariate Lomax distribution (which arises as a certain mixture of exponential distributions) are derived and the robustness of Zidek's (Ann. Statist. 1 (1973) 264) and Brewster's (Ann. Statist. 2 (1974) 553) estimators of the scale parameter of an exponential distribution is established under a class of modified Lomax distributions.     

Comparison of estimators of the location parameter using pitman's closeness criterion

Necessary and sufficient conditions for a linear estimator to dominate another linear estimator of a location parameter under the Pitman's criterion of comparison are discussed. Consequently it is demonstrated that a linear biased estimator can not dominate a linear unbiased estimator under Pitman's criterion and that the sample mean is the Closest Linear Unbiased Estimator (CLUE). It is also shown that the ridge regression estimator with a known biasing constant can not dominate the ordinary least squares estimator. If an estimator δdominates an estimator δin the average loss sense then sufficient conditions are obtained under which δis also preferred over δunder Pitman's criterion. Further we obtain sufficient conditions under which preference under the Pitman's criterion will lead to preference under the mean squared error sense.     

The empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior under Stein's loss function

For the hierarchical Poisson and gamma model, we calculate the Bayes posterior estimator of the parameter of the Poisson distribution under Stein's loss function which penalizes gross overestimation and gross underestimation equally and the corresponding Posterior Expected Stein's Loss (PESL). We also obtain the Bayes posterior estimator of the parameter under the squared error loss and the corresponding PESL. Moreover, we obtain the empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior by two methods. In numerical simulations, we have illustrated: The two inequalities of the Bayes posterior estimators and the PESLs; the moment estimators and the Maximum Likelihood Estimators (MLEs) are consistent estimators of the hyperparameters; the goodness-of-fit of the model to the simulated data. The numerical results indicate that the MLEs are better than the moment estimators when estimating the hyperparameters. Finally, we exploit the attendance data on 314 high school juniors from two urban high schools to illustrate our theoretical studies.     

Order-preserving Estimators and an Inequality on the Integration of Zonal Polynomial

ABSTRACT

Suppose X , p × p p.d. random matrix, has the distribution which depends on a p × p p.d. parameter matrix Σ and this distribution is orthogonally invariant. The orthogonally invariant estimator of Σ which has the eigenvalues of the same order as the eigenvalues of X is called order-preserving. We conjecture that a non-order-preserving estimator is dominated by modified order-preserving estimators with respect to the entropy (Stein's) loss function. We show that an inequality on the integration of zonal polynomial is sufficient for this conjecture. We also prove this inequality for the case p = 2.     

On the bias and variance of some proportion estimators

Providing certain parameters are known, almost any linear map from R^P to R¹ can be adjusted to yield a consistent and unbiased estimator in the context of estimating the mixing proportion θ on the basis of an unclassified sample of observations taken from a mixture of two p-dimensional distributions in proportions θ and 1-θ. Attention is focused on an estimator proposed recently, θ, which has minimum variance over all such linear maps. Unfortunately, the form of θ depends on the means of the component distributions and the covariance matrix of the mixture distribution. The effect of using appropriate sample estimates for these unknown parameters in forming θ is investigated by deriving the asymptotic mean and variance of the resulting estimator. The relative efficiency of this estimator under normality is derived. Also, a study is undertaken of the performance of a similar type of estimator appropriate in the context where an observed data vector is not an observation from either one or the other onent distributions, but is recorded as an integrated measurement over a surface area which is a mixture of two categories whose characteristics have different statistical distributions.The asymptotic bias in this case is compared with some available practical results.     

Non-parametric estimation of an acceleration parameter

Let X₁,… X_m be a random sample of m failure times under normal conditions with the underlying distribution F(x) and Y₁,…,Y_n a random sample of n failure times under accelerated condititons with underlying distribution G(x);G(x)=1?[1?F(x)]^θ with θ being the unknown parameter under study.Define:U_ij=1 otherwise.The joint distribution of _ijdoes not involve the distribution F and thus can be used to estimate the acceleration parameter θ.The second approach for estimating θ is to use the ranks of the Y-observations in the combined X- and Y-samples.In this paper we establish that the rank of the Y-observations in the pooled sample form a sufficient statistic for the information contained in the U_ii 's about the parameter θ and that there does not exist an unbiassed estimator for the parameter θ.We also construct several estimators and confidence interavals for the parameter θ.     

Monotonic minimax estimators of a 2×2 covariance matrix

Let S : 2 × 2 have a nonsingular Wishart distribution with unknown matrix σ and n degrees of freedom. For estimating σ two families of mimmax estimators, with respect to the entropy loss, are presented. These estimators are of the form σ(S) = Rø(L)R^t where R is orthogonal, L and Φ are diagonal, and RLR^T = S. Conditions under which the components of Φ and L follow the same order relation [i.e., writing Φ = diag(Φ₁,Φ₂) and L = diag(l₁,/₂) with l₁ ≥ l₂, we have Φ₁ ≥ Φ₂] are established. Comparisons with Stein's estimators and other orthogonally invariant estimators are discussed.     

Estimation of a multivariate normal covariance matrix under a certain structure

We consider the estimation of Σ of the p-dimensional normal distribution N_p (0, Σ) when Σ?=?θ₀ I_p ?+?θ₁ aa′, where a is an unknown p-dimensional normalized vector and θ₀?>?0, θ₁?≥?0 are also unknown. First, we derive the restricted maximum likelihood (REML) estimator. Second, we propose a new estimator, which dominates the REML estimator with respect to Stein's loss function. Finally, we carry out Monte Carlo simulation to investigate the magnitude of the new estimator's superiority.     

Wavelet threshold based on Stein's unbiased risk estimators of restricted location parameter in multivariate normal

In this paper, the problem of estimating the mean vector under non-negative constraints on location vector of the multivariate normal distribution is investigated. The value of the wavelet threshold based on Stein''s unbiased risk estimators is calculated for the shrinkage estimator in restricted parameter space. We suppose that covariance matrix is unknown and we find the dominant class of shrinkage estimators under Balance loss function. The performance evaluation of the proposed class of estimators is checked through a simulation study by using risk and average mean square error values.     

A reexamination of stein's antifiducial example

In Stein's 1959 example, for any sample with n sufficiently large, there is a confidence set embedded simultaneously within two regular confidence belts—one with coverage frequency smaller than an arbitrary positive ϵ, the other with coverage frequency larger than 1 — ϵ. Thus, Stein's example may be seen as an extreme case of mutually conflicting confidence statements, illustrating a possibility anticipated and denounced by Fisher.     

Robust Liu estimator for regression based on an M-estimator

Consider the regression model y = beta 0 1 + Xbeta + epsilon. Recently, the Liu estimator, which is an alternative biased estimator beta L (d) = (X'X + I) -1 (X'X + dI)beta OLS , where 0<d<1 is a parameter, has been proposed to overcome multicollinearity . The advantage of beta L (d) over the ridge estimator beta R (k) is that beta L (d) is a linear function of d. Therefore, it is easier to choose d than to choose k in the ridge estimator. However, beta L (d) is obtained by shrinking the ordinary least squares (OLS) estimator using the matrix (X'X + I) -1 (X'X + dI) so that the presence of outliers in the y direction may affect the beta L (d) estimator. To cope with this combined problem of multicollinearity and outliers, we propose an alternative class of Liu-type M-estimators (LM-estimators) obtained by shrinking an M-estimator beta M , instead of the OLS estimator using the matrix (X'X + I) -1 (X'X + dI).     

Expected Absolute Error of the Usual Estimator of the Binomial Parameter

For X with binomial (n, p) distribution the usual measure of the error of X/n as an estimator of p is its standard error S_n(p) = √{E(X/n – p)²} = √{p(1 – p)/n}. A somewhat more natural measure is the average absolute error D_n(p) = E‖X/n – p‖. This article considers use of D_n(p) instead of S_n(p) in a student's first introduction to statistical estimation. Exact and asymptotic values of D_n(p), and the appearance of its graph, are described in detail. The same is done for the Poisson distribution.     

Equivariant estimation under the pitman closeness criterion

For the point estimation in models with group structures, an invariance approach to deriving superior estimators is discussed in the Pitman closeness (PC) criterion. When the maximal invariant statistic is parameter-free, that is, ancillary, the closest equivariant estimator to the true value in the PC criterion is presented. On the other hand, as an example where a distribution of the maximalinvariant statistic depends on unknown parameters, the paper treats the Stein problem in estimation of a variance and obtains an improved estimator in the PC criterion by Stein's invariance approach. Also the Stein problem in simultaneous estimation of a location vector of a spherical symmetric distribution is studied.