Adaptive nonparametric empirical Bayes estimation via wavelet series: The minimax study

In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the system depends on the size of wavelet support and smoothness of the Bayes estimator. An adaptive choice of the resolution level is carried out using Lepski et al. (1997) method. The method is computationally efficient and provides asymptotically optimal adaptive EB estimators. The theory is supplemented by numerous examples.     

Minimax linear empirical bayes estimation of the binomial parameter

The minimax linear Empirical Bayes estimators for a binomial parameter are obtained, assuming some information about the moments of the prior. The form of these estimates is used to propose a criterion which may be helpful in determining whether Empirical Bayes estimation is Indicated for a given problem.     

A novel weighted likelihood estimation with empirical Bayes flavor

We propose a novel approach to estimation, where a set of estimators of a parameter is combined into a weighted average to produce the final estimator. The weights are chosen to be proportional to the likelihood evaluated at the estimators. We investigate the method for a set of estimators obtained by using the maximum likelihood principle applied to each individual observation. The method can be viewed as a Bayesian approach with a data-driven prior distribution. We provide several examples illustrating the new method and argue for its consistency, asymptotic normality, and efficiency. We also conduct simulation studies to assess the performance of the estimators. This straightforward methodology produces consistent estimators comparable with those obtained by the maximum likelihood method. The method also approximates the distribution of the estimator through the “posterior” distribution.     

Bayes and empirical bayes estimation of the probability that z > x + y

Let X, Y and Z be independent random variables with common unknown distribution F. Using the Dirichlet process prior for F and squared erro loss function, the Bayes and empirical Bayes estimators of the parameters λ(F). the probability that Z > X + Y, are derived. The limiting Bayes estimator of λ(F) under some conditions on the parameter of the process is shown to be asymptotically normal. The aysmptotic optimality of the empirical Bayes estimator of λ(F) is established. When X, Y and Z have support on the positive real line, these results are derived for randomly right censored data. This problem relates to testing whether than used discussed by Hollander and Proshcan (1972) and Chen, Hollander and Langberg (1983).     

Multivariate limited translation empirical Bayes estimators

The paper develops multivariate limited translation empirical Bayes estimators of the normal mean vector which serve as a compromise between the empirical Bayes and the maximum likelihood estimators. These compromise estimators perform better than the regular empirical Bayes estimators, in a frequentist sense, when there is wide departure of an individual observation from the grand average.     

Nonparametrio bayes and empirical bayes estimation of the residual survival function at age t

Nonparametric Bayes and empirical Bayes estimations of the

survival function of a unit of age t (> 0) using Dirichlet

process prior are presented. The proposed empirical Bayes

estimators are found to be “asymptotically optimal” in the sense of Robbins (1955). The performances of the proposed

empirical Bayes estimators are compared with those of certain

rival estimators in terms of relative savings loss, The exact

expressions for Bayes risks are also provided in certain cases.     

Asymptotic optimality of a two-stage procedure in Bayes sequential estimation

The problem of sequential estimation of the mean with quadratic loss and fixed cost per observation is considered within the Bayesian framework. Instead of fully sequential sampling, a two-stage sampling technique is introduced to solve the problem. The proposed two-stage procedure is robust in the sense that it does not depend on the distribution of outcome variables and the prior. It is shown to be asymptotically not worse than the optimal fixed-sample-size procedures for the arbitrary distributions, and to be asymptotically Bayes for the distributions of one-parameter exponential family.     

Bayes minimax estimation of a bernoulli p in a restricted parameter space

In many estimation problems the parameter of interest is known,a priori, to belong to a proper subspace of the natural parameter space. Although useful in practice this type of additional information can lead to surprising theoretical difficulties. In this paper the problem of minimax estimation of a Bernoulli pwhen pis restricted to a symmetric subinterval of the natural parameter space is considered. For the sample sizes n = 1,2,3, and 4 least favorable priors with finite support are provided and the corresponding Bayes estimators are shown to be minimax. For n = 5 and 6 the usual constant risk minimax estimator is shown to be the Bayes minimax estimator corresponding to a least favorable prior with finite support, provided the restriction on the parameter space is not too tight.     

Empirical Bayes sequential estimation of a finite population mean

In this article, we consider the Bayes and empirical Bayes problem of the current population mean of a finite population when the sample data is available from other similar (m-1) finite populations. We investigate a general class of linear estimators and obtain the optimal linear Bayes estimator of the finite population mean under a squared error loss function that considered the cost of sampling. The optimal linear Bayes estimator and the sample size are obtained as a function of the parameters of the prior distribution. The corresponding empirical Bayes estimates are obtained by replacing the unknown hyperparameters with their respective consistent estimates. A Monte Carlo study is conducted to evaluate the performance of the proposed empirical Bayes procedure.     

Adaptive power priors with empirical Bayes for clinical trials

Incorporating historical information into the design and analysis of a new clinical trial has been the subject of much discussion as a way to increase the feasibility of trials in situations where patients are difficult to recruit. The best method to include this data is not yet clear, especially in the case when few historical studies are available. This paper looks at the power prior technique afresh in a binomial setting and examines some previously unexamined properties, such as Box P values, bias, and coverage. Additionally, it proposes an empirical Bayes‐type approach to estimating the prior weight parameter by marginal likelihood. This estimate has advantages over previously criticised methods in that it varies commensurably with differences in the historical and current data and can choose weights near 1 when the data are similar enough. Fully Bayesian approaches are also considered. An analysis of the operating characteristics shows that the adaptive methods work well and that the various approaches have different strengths and weaknesses.     

The empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior under Stein's loss function

For the hierarchical Poisson and gamma model, we calculate the Bayes posterior estimator of the parameter of the Poisson distribution under Stein's loss function which penalizes gross overestimation and gross underestimation equally and the corresponding Posterior Expected Stein's Loss (PESL). We also obtain the Bayes posterior estimator of the parameter under the squared error loss and the corresponding PESL. Moreover, we obtain the empirical Bayes estimators of the parameter of the Poisson distribution with a conjugate gamma prior by two methods. In numerical simulations, we have illustrated: The two inequalities of the Bayes posterior estimators and the PESLs; the moment estimators and the Maximum Likelihood Estimators (MLEs) are consistent estimators of the hyperparameters; the goodness-of-fit of the model to the simulated data. The numerical results indicate that the MLEs are better than the moment estimators when estimating the hyperparameters. Finally, we exploit the attendance data on 314 high school juniors from two urban high schools to illustrate our theoretical studies.     

Protection against outliers in empirical bayes estimation

The proven optimality properties of empirical Bayes estimators and their documented successful performance in practice have made them popular. Although many statisticians have used these estimators since the landmark paper of James and Stein (1961), relatively few have proposed techniques for protecting them from the effects of outlying observations or outlying parameters. One notable series of studies in protection against outlying parameters was conducted by Efron and Morris (1971, 1972, 1975). In the fully Bayesian case, a general discussion on robust procedures can be found in Berger (1984, 1985). Here we implement and evaluate a different approach for outlier protection in a random-effects model which is based on appropriate specification of the prior distribution. When unusual parameters are present, we estimate the prior as a step function, as suggested by Laird and Louis (1987). This procedure is evaluated empirically, using a number of simulated data sets to compare the effects of the step-function prior with those of the normal and Laplace priors on the prediction of small-area proportions.     

Empirical bayes estimation of a distribution function with a gamma process prior

A sequence of empirical Bayes estimators is given for estimating a distribution function. It is shown that ‘i’ this sequence is asymptotically optimum relative to a Gamma process prior, ‘ii’ the overall expected loss approaches the minimum Bayes risk at a rate of n , and ‘iii’ the estimators form a sequence of proper distribution functions. Finally, the numerical example presented by Susarla and Van Ryzin ‘Ann. Statist., 6, 1978’ reworked by Phadia ‘Ann. Statist., 1, 1980, to appear’ has been analyzed and the results are compared to the numerical results by Phadia     

Bayes and empirical Bayes shrinkage estimation of regression coefficients

For a moderate or large number of regression coefficients, shrinkage estimates towards an overall mean are obtained by Bayes and empirical Bayes methods. For a special case, the Bayes and empirical Bayes shrinking weights are shown to be asymptotically equivalent as the amount of shrinkage goes to zero. Based on comparisons between Bayes and empirical Bayes solutions, a modification of the empirical Bayes shrinking weights designed to guard against unreasonable overshrinking is suggested. A numerical example is given.     

Adaptive wavelet series estimation in separable nonparametric regression models

It is well-known that multivariate curve estimation suffers from the curse of dimensionality. However, reasonable estimators are possible, even in several dimensions, under appropriate restrictions on the complexity of the curve. In the present paper we explore how much appropriate wavelet estimators can exploit a typical restriction on the curve such as additivity. We first propose an adaptive and simultaneous estimation procedure for all additive components in additive regression models and discuss rate of convergence results and data-dependent truncation rules for wavelet series estimators. To speed up computation we then introduce a wavelet version of functional ANOVA algorithm for additive regression models and propose a regularization algorithm which guarantees an adaptive solution to the multivariate estimation problem. Some simulations indicate that wavelets methods complement nicely the existing methodology for nonparametric multivariate curve estimation.     

On empirical bayes sequential estimation

We study the empirical Bayes approach to the sequential estimation problem. An empirical Bayes sequential decision procedure, which consists of a stopping rule and a terminal decision rule, is constructed for use in the component. Asymptotic behaviors of the empirical Bayes risk and the empirical Bayes stopping times are investigated as the number of components increase.     

Comparison of testing procedures utilizing P-values and Bayes factors in some common situations

Bayesian alternatives to classical tests for several testing problems are considered. One-sided and two-sided sets of hypotheses are tested concerning an exponential parameter, a Binomial proportion, and a normal mean. Hierarchical Bayes and noninformative Bayes procedures are compared with the appropriate classical procedure, either the uniformly most powerful test or the likelihood ratio test, in the different situations. The hierarchical prior employed is the conjugate prior at the first stage with the mean being the test parameter and a noninformative prior at the second stage for the hyper parameter(s) of the first stage prior. Fair comparisons are attempted in which fair means the likelihood of making a type I error is approximately the same for the different testing procedures; once this condition is satisfied, the power of the different tests are compared, the larger the power, the better the test. This comparison is difficult in the two-sided case due to the unsurprising discrepancy between Bayesian and classical measures of evidence that have been discussed for years. The hierarchical Bayes tests appear to compete well with the typical classical test in the one-sided cases.