On the bounded regret of empirical bayes estimators

Let (θ₁,x₁),…,(θ_n,x_n) be independent and identically distributed random vectors with E(xθ) = θ and Var(x|θ) = a + bθ + cθ². Let t_i be the linear Bayes estimator of θ_i and θ~_i be the linear empirical Bayes estimator of θ_i as proposed in Robbins (1983). When Ex and Var x are unknown to the statistician. The regret of using θ~_i instead of t_i because of ignorance of the mean and the variance is r_i = E(θ_i ? θ_i)² ?E(t_i ?θ_i)². Under appropriate conditions cumulative regret R_n = r₁+…r_n is shown to have a finite limit even when n tends to infinity. The limit can be explicitly computed in terms of a,b,c and the first four moments of x.     

Empirical bayes tests in a positive exponential family with partial information on priors

We investigate an empirical Bayes testing problem in a positive exponential family having pdf f{x/θ)=c(θ)u(x) exp(?x/θ), x>0, θ>0. It is assumed that θ is in some known compact interval [C₁, C₂]. The value C₁ is used in the construction of the proposed empirical Bayes test δ^* _n. The asymptotic optimality and rate of convergence of its associated Bayes risk is studied. It is shown that under the assumption that θ is in [C₁, C₂] δ^* _n is asymptotically optimal at a rate of convergence of order O(n^?1/n n). Also, δ^* _n is robust in the sense that δ^* _n still possesses the asymptotic optimality even the assumption that "C₁≦θ≦C₂ may not hold.     

Minimax estimation of a lower-bounded scale parameter of a gamma distribution for scale-invariant squared-error loss

Let X have a gamma distribution with known shape parameter θr;aL and unknown scale parameter θ. Suppose it is known that θ ≥ a for some known a > 0. An admissible minimax estimator for scale-invariant squared-error loss is presented. This estimator is the pointwise limit of a sequence of Bayes estimators. Further, the class of truncated linear estimators C = {θ_ρ|θ_ρ(x) = max(a, ρ), ρ > 0} is studied. It is shown that each θ_ρ is inadmissible and that exactly one of them is minimax. Finally, it is shown that Katz's [Ann. Math. Statist., 32, 136–142 (1961)] estimator of θ is not minimax for our loss function. Some further properties of and comparisons among these estimators are also presented.     

Non-parametric estimation of an acceleration parameter

Let X₁,… X_m be a random sample of m failure times under normal conditions with the underlying distribution F(x) and Y₁,…,Y_n a random sample of n failure times under accelerated condititons with underlying distribution G(x);G(x)=1?[1?F(x)]^θ with θ being the unknown parameter under study.Define:U_ij=1 otherwise.The joint distribution of _ijdoes not involve the distribution F and thus can be used to estimate the acceleration parameter θ.The second approach for estimating θ is to use the ranks of the Y-observations in the combined X- and Y-samples.In this paper we establish that the rank of the Y-observations in the pooled sample form a sufficient statistic for the information contained in the U_ii 's about the parameter θ and that there does not exist an unbiassed estimator for the parameter θ.We also construct several estimators and confidence interavals for the parameter θ.     

CERTAIN BOUNDS CONCERNING CHARACTERISTIC FUNCTIONS1

The largest value of the constant c for which holds over the class of random variables X with non-zero mean and finite second moment, is c=π. Let the random variable (r.v.) X with distribution function F(·) have non-zero mean and finite second moment. In studying a certain random walk problem (Daley, 1976) we sought a bound on the characteristic function of the form for some positive constant c. Of course the inequality is non-trivial only provided that . This note establishes that the best possible constant c =π. The wider relevance of the result is we believe that it underlines the use of trigonometric inequalities in bounding the (modulus of a) c.f. (see e.g. the truncation inequalities in §12.4 of Loève (1963)). In the present case the bound thus obtained is the best possible bound, and is better than the bound (2) |1-?(θ)| ≥ |θEX|-θ²EX²\2 obtained by applying the triangular inequality to the relation which follows from a two-fold integration by parts in the defining equation (*). The treatment of the counter-example furnished below may also be of interest. To prove (1) with c=π, recall that sin u > u(1-u/π) (all real u), so Since |E sinθX|-|E sin(-θX)|, the modulus sign required in (1) can be inserted into (4). Observe that since sin u > u for u < 0, it is possible to strengthen (4) to (denoting max(0,x) by x₊) To show that c=π is the best possible constant in (1), assume without loss of generality that EX > 0, and take θ > 0. Then (1) is equivalent to (6) c < θEX²/{EX-|1-?(θ)|/θ} for all θ > 0 and all r.v.s. X with EX > 0 and EX². Consider the r.v. where 0 < x < 1 and 0 < γ < ∞. Then EX=1, EX²=1+γx², From (4) it follows that |1-?(θ)| > 0 for 0 < |θ| <π|EX|/EX² but in fact this positivity holds for 0 < |θ| < 2π|EX|/EX² because by trigonometry and the Cauchy-Schwartz inequality, |1-?(θ)| > |Re(1-?(θ))| = |E(1-cosθX)| = 2|E sin²θX/2| (10) >2(E sinθX/2)² (11) >(|θEX|-θ²EX²/2π)²/2 > 0, the inequality at (11) holding provided that |θEX|-θ²EX²/2π > 0, i.e., that 0 < |θ| < 2π|EX|/EX². The random variable X at (7) with x= 1 shows that the range of positivity of |1-?(θ)| cannot in general be extended. If X is a non-negative r.v. with finite positive mean, then the identity shows that (1-?(θ))/iθEX is the c.f. of a non-negative random variable, and hence (13) |1-?(θ)| < |θEX| (all θ). This argument fans if pr{X < 0}pr{X> 0} > 0, but as a sharper alternative to (14) |1-?(θ)| < |θE|X||, we note (cf. (2) and (3)) first that (15) |1-?(θ)| < |θEX| +θ²EX²/2. For a bound that is more precise for |θ| close to 0, |1-?(θ)|²= (Re(1-?(θ)))²+ (Im?(θ))² <(θ²EX²/2)²+(|θEX| +θ²EX²_-/π)², so (16) |1-?(θ)| <(|θEX| +θ²EX²_-/π) + |θ|³(EX²)²/8|EX|.     

Minimax Estimation of the Bounded Parameter of Some Discrete Distributions Under LINEX Loss Function

For a class of discrete distributions, including Poisson(θ), Generalized Poisson(θ), Borel(m, θ), etc., we consider minimax estimation of the parameter θ under the assumption it lies in a bounded interval of the form [0, m] and a LINEX loss function. Explicit conditions for the minimax estimator to be Bayes with respect to a boundary supported prior are given. Also for Bernoulli(θ)-distribution, which is not in the mentioned class of discrete distributions, we give conditions for which the Bayes estimator of θ ∈ [0, m], m < 1 with respect to a boundary supported prior is minimax under LINEX loss function. Numerical values are given for the largest values of m for which the corresponding Bayes estimators of θ are minimax.     

A note on a modified double stage shrinkage estimator

A modified double stage shrinkage estimator has been proposed for the single parameter θ of a distribution function _Fθ. It is shown to be locally better in comparison to the usual double stage shrinkage estimator in the sense of smaller mean squared error in a certain neighbourhood of prior estimate θ_o of θ.     

THE LIMITING FORM OF THE NON-CENTRAL WISHART DISTRIBUTION1,2

Let S (p×p) have a Wishart distribution -with v degrees of freedom and non-centrality matrix θ= [θ_jK] (p×p). Define θ₀= min {| θ_jk |}, let θ₀→∞, and suppose that | θ_jK | = 0(θ_o). Then the limiting form of the standardized non-central distribution, as θ while n? remains fixed, is a multivariate Gaussian distribution. This result in turn is used to obtain known asymptotic properties of multivariate chi-square and Rayleigh distributions under somewhat weaker conditions.     

Deficiencies of minimum discrepancy estimators

Suppose the multinomial parameters p_r (θ) are functions of a real valued parameter 0, r = 1,2, …, k. A minimum discrepancy (m.d.) estimator θ of θ is defined as one which minimises the discrepancy function D = Σ n_rf(p_r/n_r), for a suitable function f where n_r is the relative frequency in r-th cell, r = 1,2, …, k. All the usual estimators like maximum likelihood (m. l), minimum chi-square (m. c. s.)., etc. are m.d. estimators. All m.d. estimators have the same asymptotic (first order) efficiency. They are compared on the basis of their deficiencies, a concept recently introduced by Hodges and Lehmann [2]. The expression for least deficiency at any θ is derived. It is shown that in general uniformly least deficient estimators do not exist. Necessary and sufficient conditions on p_r (0) for m. t. and m. c. s. estimators to be uniformly least deficient are obtained.     

Estimation of a multivariate normal covariance matrix under a certain structure

We consider the estimation of Σ of the p-dimensional normal distribution N_p (0, Σ) when Σ?=?θ₀ I_p ?+?θ₁ aa′, where a is an unknown p-dimensional normalized vector and θ₀?>?0, θ₁?≥?0 are also unknown. First, we derive the restricted maximum likelihood (REML) estimator. Second, we propose a new estimator, which dominates the REML estimator with respect to Stein's loss function. Finally, we carry out Monte Carlo simulation to investigate the magnitude of the new estimator's superiority.     

ON ASYMPTOTIC OPTIMALITY OF SOME TESTS FOR EXPONENTIAL DISTRIBUTIONS

The probability density function (pdf) of a two parameter exponential distribution is given by f(x; p, s?) =s?^-1 exp {-(x - ρ)/s?} for x≥ρ and 0 elsewhere, where 0 < ρ < ∞ and 0 < s?∞. Suppose we have k independent random samples where the ith sample is drawn from the ith population having the pdf f(x; ρ_i, s?_i), 0 < ρ_i < ∞, 0 < s?_i < s?_i < and f(x; ρ, s?) is as given above. Let X_i1 < X_i2 <… < X_iri denote the first r_i order statistics in a random sample of size n_i, drawn from the ith population with pdf f(x; ρ_i, s?_i), i = 1, 2,…, k. In this paper we show that the well known tests of hypotheses about the parameters ρ_i, s?_i, i = 1, 2,…, k based on the above observations are asymptotically optimal in the sense of Bahadur efficiency. Our results are similar to those for normal distributions.     

On the bias and variance of some proportion estimators

Providing certain parameters are known, almost any linear map from R^P to R¹ can be adjusted to yield a consistent and unbiased estimator in the context of estimating the mixing proportion θ on the basis of an unclassified sample of observations taken from a mixture of two p-dimensional distributions in proportions θ and 1-θ. Attention is focused on an estimator proposed recently, θ, which has minimum variance over all such linear maps. Unfortunately, the form of θ depends on the means of the component distributions and the covariance matrix of the mixture distribution. The effect of using appropriate sample estimates for these unknown parameters in forming θ is investigated by deriving the asymptotic mean and variance of the resulting estimator. The relative efficiency of this estimator under normality is derived. Also, a study is undertaken of the performance of a similar type of estimator appropriate in the context where an observed data vector is not an observation from either one or the other onent distributions, but is recorded as an integrated measurement over a surface area which is a mixture of two categories whose characteristics have different statistical distributions.The asymptotic bias in this case is compared with some available practical results.     

Asymptotics of Likelihood Ratio Tests for General One-sided Hypotheses in the Two-sample Normal Model

We consider the general one-sided hypotheses testing problem expressed as H₀: θ₁ ? h(θ₂) versus H₁: θ₁ < h(θ₂), where h( · ) is not necessary differentiable. The values of the right and the left differential coefficients, h′_?( · ) and h′₊( · ), at nondifferentiable points play an essential role in constructing the appropriate testing procedures with asymptotic size α on the basis of the likelihood ratio principle. The likelihood ratio testing procedure is related to an intersection–union testing procedure when h′_?(θ₂) ? h′₊(θ₂) for all θ₂, and to a union–intersection testing procedure when there exists a θ₂ such that h′_?(θ₂) < h′₊(θ₂).     

An inclusion-consistent solution to the problem of absurd confidence statements: 2. Consistent nonexact-confidence-interval estimation

Two consistent nonexact-confidence-interval estimation methods, both derived from the consistency-equivalence theorem in Plante (1991), are suggested for estimation of problematic parametric functions with no consistent exact solution and for which standard optimal confidence procedures are inadequate or even absurd, i.e., can provide confidence statements with a 95% empty or all-inclusive confidence set. A belt C(·) from a consistent nonexact-belt family, used with two confidence coefficients (γ = inf_θ P_θ [ θ ? C(X)] and γ⁺ = sup_θ P_θ[θ ? C(X)], is shown to provide a consistent nonexact-belt solution for estimating μ₂ -μ₁ in the Behrens-Fisher problem. A rule for consistent behaviour enables any confidence belt to be used consistently by providing each sample point with best upper and lower confidence levels [δ⁺(x) ≥ γ⁺, δ(x) ≤ γ], which give least-conservative consistent confidence statements ranging from practically exact through informative to noninformative. The rule also provides a consistency correction L(x) = δ⁺(x)-δ(X) enabling alternative confidence solutions to be compared on grounds of adequacy; this is demonstrated by comparing consistent conservative sample-point-wise solutions with inconsistent standard solutions for estimating μ₂/μ₁ (Creasy-Fieller-Neyman problem) and $\sqrt {\mu _1^2 + \mu _2^2 }$, a distance-estimation problem closely related to Stein's 1959 example     

A test for the equality of k medians against the simple tree alternative under right censorship

We consider a test for the equality of k population medians, θ_i i=1,2,….,k, when it is believed a priori, that θ _i: The observations are subject to right censorhip. The distributions of the censoring variables for each population are assumed to be equal. This test is compared with the general k-sample test proposed by Breslow     

Estimating population size with truncated sampling

Let X₁, X₂,…,X_n be independent, indentically distributed random variables with density f(x,θ) with respect to a σ-finite measure μ. Let R be a measurable set in the sample space X. The value of X is observable if X ? (X?R) and not otherwise. The number J of observable X’s is binomial, N, Q, Q = 1?P(X ? R). On the basis of J observations, it is desired to estimate N and θ. Estimators considered are conditional and unconditional maximum likelihood and modified maximum likelihood using a prior weight function to modify the likelihood before maximizing. Asymptotic expansions are developed for the [Ncirc]’s of the form [Ncirc] = N + α√N + β + o_p(1), where α and β are random variables. All estimators have the same α, which has mean 0, variance σ² (a function of θ) and is asymptotically normal. Hence all are asymptotically equivalent by the usual limit distributional theory. The β’s differ and Eβ can be considered an “asymptotic bias”. Formulas are developed to compare the asymptotic biases of the various estimators. For a scale parameter family of absolutely continuous distributions with X = (0,∞) and R = (T,∞), special formuli are developed and a best estimator is found.     

ESTIMATING A PARAMETER WHEN IT IS KNOWN THAT THE PARAMETER EXCEEDS A GIVEN VALUE

In some statistical problems a degree of explicit, prior information is available about the value taken by the parameter of interest, θ say, although the information is much less than would be needed to place a prior density on the parameter's distribution. Often the prior information takes the form of a simple bound, ‘θ > θ₁ ’ or ‘θ < θ₁ ’, where θ₁ is determined by physical considerations or mathematical theory, such as positivity of a variance. A conventional approach to accommodating the requirement that θ > θ₁ is to replace an estimator, , of θ by the maximum of and θ₁. However, this technique is generally inadequate. For one thing, it does not respect the strictness of the inequality θ > θ₁ , which can be critical in interpreting results. For another, it produces an estimator that does not respond in a natural way to perturbations of the data. In this paper we suggest an alternative approach, in which bootstrap aggregation, or bagging, is used to overcome these difficulties. Bagging gives estimators that, when subjected to the constraint θ > θ₁ , strictly exceed θ₁ except in extreme settings in which the empirical evidence strongly contradicts the constraint. Bagging also reduces estimator variability in the important case for which is close to θ₁, and more generally produces estimators that respect the constraint in a smooth, realistic fashion.     

Optimality of the posterior predictive p-value based on the posterior. Odds

Thompson (1997) considered a wide definition of p-value and found the Baves p-value for testing a ooint null hypothesis H₀: θ= θ₀ versus H₁: θ ≠ θ₀. In this paper, the general case of testing H₀: θ ∈ ?₀ versus H₁: θ ∈ ?^c ₀ is studied. A generalization of the concept of p-value is given, and it is proved that the posterior predictive p-value based on the posterior odds is (asymptotically) a Bayes p-value. Finally, it is suggested that this posterior predictive p-value could be used as a reference p-value     

Local rates of convergence for consistent estimators of location of translation families

We consider the estimation of a location parameter θ in a one-sample problem. A measure of the asymptotic performance of an estimator sequence {T_n} = T is given by the exponential rate of convergence to zero of the tail probability, which for consistent estimator sequences is bounded by a constant, B (θ, ?), called the Bahadur bound. We consider two consistent estimators: the maximum-likelihood estimator (mle) and a consistent estimator based on a likelihood-ratio statistic, which we call the probability-ratio estimator (pre). In order to compare the local behaviour of these estimators, we obtain Taylor series expansions in ? for B (θ, ?) and the exponential rates of the mle and pre. Finally, some numerical work is presented in which we consider a variety of underlying distributions.