On the bounded regret of empirical bayes estimators

Let (θ₁,x₁),…,(θ_n,x_n) be independent and identically distributed random vectors with E(xθ) = θ and Var(x|θ) = a + bθ + cθ². Let t_i be the linear Bayes estimator of θ_i and θ~_i be the linear empirical Bayes estimator of θ_i as proposed in Robbins (1983). When Ex and Var x are unknown to the statistician. The regret of using θ~_i instead of t_i because of ignorance of the mean and the variance is r_i = E(θ_i ? θ_i)² ?E(t_i ?θ_i)². Under appropriate conditions cumulative regret R_n = r₁+…r_n is shown to have a finite limit even when n tends to infinity. The limit can be explicitly computed in terms of a,b,c and the first four moments of x.     

Consistency of least-squares estimator and its jackknife variance estimator in nonlinear models

The purpose of this paper is twofold: (1) We establish the consistency of the least-squares estimator in a nonlinear modely_i = f(x_i,θ) +σ_ie_i where the range of the parameter θ is noncompact, the regression function is unbounded, and the σ_i,'s are not necessarily equal. This extends the results in Jennrich (1969) and Wu (1981). (2) Under the same model, the jackknife estimator of the asymptotic covariance matrix of the least-squares estimator is shown to be consistent, which provides a theoretical justification of the empirical results in Duncan (1978) and the use of the jackknife method in large-sample inferences.     

A note on a modified double stage shrinkage estimator

A modified double stage shrinkage estimator has been proposed for the single parameter θ of a distribution function _Fθ. It is shown to be locally better in comparison to the usual double stage shrinkage estimator in the sense of smaller mean squared error in a certain neighbourhood of prior estimate θ_o of θ.     

Estimation of a multivariate mean with constraints on the norm

Estimation of the mean θ of a spherical distribution with prior knowledge concerning the norm ||θ|| is considered. The best equivariant estimator is obtained for the local problem ||θ|| = λ₀, and its risk is evaluated. This yields a sharp lower bound for the risk functions of a large class of estimators. The risk functions of the best equivariant estimator and the best linear estimator are compared under departures from the assumption ||θ|| = λ₀.     

Bootstrap test of goodness of fit to a linear model when errors are correlated

Given the regression model Y_i = m(x_i) +ε_i (x_i ε C, i = l,…,n, C a compact set in R) where m is unknown and the random errors {ε_i} present an ARMA structure, we design a bootstrap method for testing the hypothesis that the regression function follows a general linear model: H_o : m ε {mθ(.) = A^t(.)θ : θ ε ? ? R^q} with A a functional from R to R^q. The criterion of the test derives from a Cramer-von-Mises type functional distance D = d²([mcirc]_n, A^t(.)θ_n), between [mcirc]_n, a Gasser-Miiller non-parametric estimator of m, and the member of the class defined in H_o that is closest to m_n in terms of this distance. The consistency of the bootstrap distribution of D and θ_n is obtained under general conditions. Finally, simulations show the good behavior of the bootstrap approximation with respect to the asymptotic distribution of D = d².     

BAYESIAN SMOOTHING WITH OCCASIONAL JUMPS

Consider a sequence of independent random variables X ₁, X ₂,…,X _n observed at n equally spaced time points where X _i has a probability distribution which is known apart from the values of a parameter θ _i ∈ R which may change from observation to observation. We consider the problem of estimating θ = (θ₁, θ₂,…,θ _n ) given the observed values of X ₁, X ₂,…,X _n . The paper proposes a prior distribution for the parameters θ for which sets of parameter values exhibiting no change, or no change apart from a few sudden large changes, or lots of small changes, all have positive prior probability. Markov chain sampling may be used to calculate Bayes estimates of the parameters. We report the results of a Monte Carlo study based on Poisson distributed data which compares the Bayes estimator with estimators obtained using cubic splines and with estimators derived from the Schwarz criterion. We conclude that the Bayes method is preferable in a minimax sense since it never produces the disastrously large errors of the other methods and pays only a modest price for this degree of safety. All three methods are used to smooth mortality rates for oesophageal cancer in Irish males aged 65–69 over the period 1955 through 1994.     

Non-parametric estimation of an acceleration parameter

Let X₁,… X_m be a random sample of m failure times under normal conditions with the underlying distribution F(x) and Y₁,…,Y_n a random sample of n failure times under accelerated condititons with underlying distribution G(x);G(x)=1?[1?F(x)]^θ with θ being the unknown parameter under study.Define:U_ij=1 otherwise.The joint distribution of _ijdoes not involve the distribution F and thus can be used to estimate the acceleration parameter θ.The second approach for estimating θ is to use the ranks of the Y-observations in the combined X- and Y-samples.In this paper we establish that the rank of the Y-observations in the pooled sample form a sufficient statistic for the information contained in the U_ii 's about the parameter θ and that there does not exist an unbiassed estimator for the parameter θ.We also construct several estimators and confidence interavals for the parameter θ.     

On the admissibility of c[Xbar] + d with respect to the linex loss function

Let X₁, …,X_n be a random sample from a normal distribution with mean θ and variance σ² The problem is to estimate θ with loss function L(θ,e) = v(e-θ) where v(x) = b(exp(ax)-ax-l) and where a, b are constants with b>0, a¦0. Zellner (1986), showed that [Xbar] ? σ²a/2n dominates [Xbar] and hence [Xbar] is inadmissible. The question of what values of c and d render c[Xbar]+ d admissible is studied here.     

Adaptive parametric test in a semiparametric regression model

Consider the semiparametric regression model Y_i = x′_iβ +g(t_i)+e_i for i=1,2, …,n. Here the design points (x_i,t_i) are known and nonrandom and the e_i are iid random errors with Ee₁ = 0 and Ee² ₁ = α²<∞. Based on g(.) approximated by a B-spline function, we consider using atest statistic for testing H₀ : β = 0. Meanwhile, an adaptive parametric test statistic is constructed and a large sample study for this adaptive parametric test statistic is presented.     

Simultaneous selection of extreme populations:a bayesian approach

The problem of simultaneously selecting two non-empty subsets, S_Land S_U, of k populations which contain the lower extreme population (LEP) and the upper extreme population (UEP), respectively, is considered. Unknown parameters θ₁,…,θ_kcharacterize the populations π₁,…,π_kand the populations associated with θ_[1]=min θ_i. and θ_[k]= max θ_i. are called the LEP and the UEP, respectively. It is assumed that the underlying distributions possess the monotone likelihood ratio property and that the prior distribution of θ= (θ₁,…,θ_k) is exchangeable. The Bayes rule with respect to a general loss function is obtained. Bayes rule with respect to a semi-additive and non-negative loss function is also determined and it is shown that it is minimax and admissible. When the selected subsets are required to be disjoint, it shown that the Bayes rule with respect to a specific loss function can be obtained by comparing certain computable integrals, Application to normal distributions with unknown means θ₁,…,θ_kand a common known variance is also considered.     

Weak and strong uniform consistency rates of kernel density estimates for randomly censored data

Let X₁,., X_n, be i.i.d. random variables with distribution function F, and let Y₁,.,.,Y_n be i.i.d. with distribution function G. For i = 1, 2,.,., n set δ_i, = 1 if X_i ≤ Y_i, and 0 otherwise, and X_i, = min{X_i, K_i}. A kernel-type density estimate of f, the density function of F w.r.t. Lebesgue measure on the Borel o-field, based on the censored data (δ_i, X_i), i = 1,.,.,n, is considered. Weak and strong uniform consistency properties over the whole real line are studied. Rates of convergence results are established under higher-order differentiability assumption on f. A procedure for relaxing such assumptions is also proposed.     

A predictive approach for the selection of a fixed number of good treatments

This paper offers a predictive approach for the selection of a fixed number (= t) of treatments from k treatments with the goal of controlling for predictive losses. For the ith treatment, independent observations X _ij (j = 1,2,…,n) can be observed where X _ij ’s are normally distributed N(θ _i ; σ ²). The ranked values of θ _i ’s and X _i ’s are θ ₍₁₎ ≤ … ≤ θ _(k) and X _[1] ≤ … ≤ X _[k] and the selected subset S = {[k], [k? 1], … , [k ? t+1]} will be considered. This paper distinguishes between two types of loss functions. A type I loss function associated with a selected subset S is the loss in utility from the selector’s view point and is a function of θ _i with i ? S. A type II loss function associated with S measures the unfairness in the selection from candidates’ viewpoint and is a function of θ _i with i ? S. This paper shows that under mild assumptions on the loss functions S is optimal and provides the necessary formulae for choosing n so that the two types of loss can be controlled individually or simultaneously with a high probability. Predictive bounds for the losses are provided, Numerical examples support the usefulness of the predictive approach over the design of experiment approach.     

Local asymptotic normality of intermediate and central order statistics

Consider n independent random variables Z_i,…, Z_n on R with common distribution function F, whose upper tail belongs to a parametric family F(t) = F_θ(t),t ≥ x₀, where θ ∈ ? ? R ^d. A necessary and sufficient condition for the family F_θ, θ ∈ ?, is established such that the k-th largest order statistic Z_n?k+1:n alone constitutes the central sequence yielding local asymptotic normality ( LAN ) of the loglikelihood ratio of the vector (Z_n?i+1:n)¹ _i=kof the k largest order statistics. This is achieved for k = k(n)→_n→∞∞ with k/n→_n→∞ 0.

In the case of vectors of central order statistics ( Z_r:n, Z_r+1:n,…, Z_s:n ), with r/n and s/n both converging to q ∈ ( 0,1 ), it turns out that under fairly general conditions any order statistic Z_m:n with r ≤ m ≤s builds the central sequence in a pertaining LAN expansion.These results lead to asymptotically optimal tests and estimators of the underlying parameter, which depend on single order statistics only     

Maximum Likelihood Estimation for Stochastic Differential Equations with Random Effects

Abstract. We consider N independent stochastic processes (X _i (t), t ∈ [0,T _i ]), i=1,…, N, defined by a stochastic differential equation with drift term depending on a random variable φ _i . The distribution of the random effect φ _i depends on unknown parameters which are to be estimated from the continuous observation of the processes X_i. We give the expression of the exact likelihood. When the drift term depends linearly on the random effect φ _i and φ _i has Gaussian distribution, an explicit formula for the likelihood is obtained. We prove that the maximum likelihood estimator is consistent and asymptotically Gaussian, when T _i =T for all i and N tends to infinity. We discuss the case of discrete observations. Estimators are computed on simulated data for several models and show good performances even when the length time interval of observations is not very large.     

Empirical bayes interval estimates involving uniform distributions

Bayesian and empirical Bayesian decision rules are exhibited for the interval estimation of the parameter 0 of a Uniform (0,θ) distribution. The estimate ?,δ>resulting in the interval [?,?+δ]suffers loss given by L(?,δ>,θ)=1-[?≦e≦?+δ]+c₁((?-θ)²+(?+δ?θ)²))+c₂δ. The solution is presented for prior distributions G which have bounded support, no point masses,∫θ^?mdG(θ)<∞ and for some integer m. An example is presented involving a particular parametric form for G and rates of risk convergence in the empirical Bayes problem for this example are calculated.     

Minimax Estimation of the Bounded Parameter of Some Discrete Distributions Under LINEX Loss Function

For a class of discrete distributions, including Poisson(θ), Generalized Poisson(θ), Borel(m, θ), etc., we consider minimax estimation of the parameter θ under the assumption it lies in a bounded interval of the form [0, m] and a LINEX loss function. Explicit conditions for the minimax estimator to be Bayes with respect to a boundary supported prior are given. Also for Bernoulli(θ)-distribution, which is not in the mentioned class of discrete distributions, we give conditions for which the Bayes estimator of θ ∈ [0, m], m < 1 with respect to a boundary supported prior is minimax under LINEX loss function. Numerical values are given for the largest values of m for which the corresponding Bayes estimators of θ are minimax.     

Consistent directions for least-squares estimates

The least-squares estimate θ_n = X_n⁺y_n of the parameter θ in the linear model Y_n = X_nθ + ?_n may not be consistent, but there may be directions u such that u'θ_n tends to u'θ in some sense. This set of directions u has been characterized in two different papers: Drygas (1976) and Wu (1980). The conditions for consistency appear to be different in the two papers. The purpose of this note is to show that the two conditions are equivalent and that they both show that the consistent directions depend upon the geometry of the row vectors v_i, i = 1,…, n, of X_n with respect to the direction u.     

Some subset selection problems

Consider that we have a collection of k populations π₁, π₂…,π_k. The quality of the ith population is characterized by a real parameter θ_i and the population is to be designated as superior or inferior depending on how much the θ_i differs from θ_max = max{θ₁, θ₂,…,θ_k}. From the set {π₁, π₂,…,π_k}, we wish to select the subset of superior populations. In this paper we devise rules of selection which have the property that their selected set excludes all the inferior populations with probability at least 1?α, where a is a specified number.     

Fisher information in weighted distributions

Suppose that a density f_θ (x) belongs to an exponential family, but that inference about θ must be based on data that are obtained from a density that is proportional to W(x)f_θ(x). The authors study the Fisher information about θ in observations obtained from such weighted distributions and give conditions under which this information is greater than under the original density. These conditions involve the hazard- and reversed-hazard-rate functions.     

Some locally optimal subset selection rules for comparison with a control

Let π₀,π₁,…,π_k be k+1 independent populations. For i=0,1,…,k,π_i has the densit f(x,θ_i), where the (unknown) parameter θ_i belongs to an interval of the real line. Our goal is to select from π₁,… π_k (experimental treatments) those populations, if any, that are better (suitably defined) than π₀ which is the control population. A locally optimal rule is derived in the class of rules for which Pr(π_i is selected)γ_i, i=1,…,k, when θ₀=θ₁=?=θ_k. The criterion used for local optimality amounts to maximizing the efficiency in a certain sense of the rule in picking out the superior populations for specific configurations of θ=(θ₀,…,θ_k) in a neighborhood of an equiparameter configuration. The general result is then applied to the following special cases: (a) normal means comparison — common known variance, (b) normal means comparison — common unknown variance, (c) gamma scale parameters comparison — known (unequal) shape parameters, and (d) comparison of regression slopes. In all these cases, the rule is obtained based on samples of unequal sizes.