Consistency of Bernstein polynomial posteriors

A Bernstein prior is a probability measure on the space of all the distribution functions on [0, 1]. Under very general assumptions, it selects absolutely continuous distribution functions, whose densities are mixtures of known beta densities. The Bernstein prior is of interest in Bayesian nonparametric inference with continuous data. We study the consistency of the posterior from a Bernstein prior. We first show that, under mild assumptions, the posterior is weakly consistent for any distribution function P ₀ on [0, 1] with continuous and bounded Lebesgue density. With slightly stronger assumptions on the prior, the posterior is also Hellinger consistent. This implies that the predictive density from a Bernstein prior, which is a Bayesian density estimate, converges in the Hellinger sense to the true density (assuming that it is continuous and bounded). We also study a sieve maximum likelihood version of the density estimator and show that it is also Hellinger consistent under weak assumptions. When the order of the Bernstein polynomial, i.e. the number of components in the beta distribution mixture, is truncated, we show that under mild restrictions the posterior concentrates on the set of pseudotrue densities. Finally, we study the behaviour of the predictive density numerically and we also study a hybrid Bayes–maximum likelihood density estimator.     

On Properties of Predictive Priors in Linear Models

Utilizing the notion of matching predictives as in Berger and Pericchi, we show that for the conjugate family of prior distributions in the normal linear model, the symmetric Kullback-Leibler divergence between two particular predictive densities is minimized when the prior hyperparameters are taken to be those corresponding to the predictive priors proposed in Ibrahim and Laud and Laud and Ibrahim. The main application for this result is for Bayesian variable selection.     

On asymptotic optimality of bayes empirical bayes estimators

In an empirical Bayes decision problem, a prior distribution ? is placed on a one-dimensfonal family G of priors G_w, wεΩ, to produce a Bayes empirical Bayes estimator, The asymptotic optimaiity of the Bayes estimator is established when the support of ? is Ω and the marginal distributions H_w have monotone likelihood ratio and continuous Kullback-Leibler information number.     

Asymptotical improvement of maximum likelihood estimators on Kullback–Leibler loss

We discuss the general form of a first-order correction to the maximum likelihood estimator which is expressed in terms of the gradient of a function, which could for example be the logarithm of a prior density function. In terms of Kullback–Leibler divergence, the correction gives an asymptotic improvement over maximum likelihood under rather general conditions. The theory is illustrated for Bayes estimators with conjugate priors. The optimal choice of hyper-parameter to improve the maximum likelihood estimator is discussed. The results based on Kullback–Leibler risk are extended to a wide class of risk functions.     

Stress–strength reliability of a non-identical-component-strengths system based on upper record values from the family of Kumaraswamy generalized distributions

In an attempt to produce more realistic stress–strength models, this article considers the estimation of stress–strength reliability in a multi-component system with non-identical component strengths based on upper record values from the family of Kumaraswamy generalized distributions. The maximum likelihood estimator of the reliability, its asymptotic distribution and asymptotic confidence intervals are constructed. Bayes estimates under symmetric squared error loss function using conjugate prior distributions are computed and corresponding highest probability density credible intervals are also constructed. In Bayesian estimation, Lindley approximation and the Markov Chain Monte Carlo method are employed due to lack of explicit forms. For the first time using records, the uniformly minimum variance unbiased estimator and the closed form of Bayes estimator using conjugate and non-informative priors are derived for a common and known shape parameter of the stress and strength variates distributions. Comparisons of the performance of the estimators are carried out using Monte Carlo simulations, the mean squared error, bias and coverage probabilities. Finally, a demonstration is presented on how the proposed model may be utilized in materials science and engineering with the analysis of high-strength steel fatigue life data.     

Bayesian predictive distribution for a Poisson model with a parametric restriction

Abstract

Predictive probability estimation for a Poisson distribution is addressed when the parameter space is restricted. The Bayesian predictive probability against the prior on the restricted space is compared with the non-restricted Bayes predictive probability. It is shown that the former predictive probability dominates the latter under some conditions when the predictive probabilities are evaluated by the risk function relative to the Kullback-Leibler divergence. This result is proved by first showing the corresponding dominance result for estimating the restricted parameter and then translating it into the framework of predictive probability estimation.     

Bayesian predictive densities based on latent information priors

Construction methods for prior densities are investigated from a predictive viewpoint. Predictive densities for future observables are constructed by using observed data. The simultaneous distribution of future observables and observed data is assumed to belong to a parametric submodel of a multinomial model. Future observables and data are possibly dependent. The discrepancy of a predictive density to the true conditional density of future observables given observed data is evaluated by the Kullback-Leibler divergence. It is proved that limits of Bayesian predictive densities form an essentially complete class. Latent information priors are defined as priors maximizing the conditional mutual information between the parameter and the future observables given the observed data. Minimax predictive densities are constructed as limits of Bayesian predictive densities based on prior sequences converging to the latent information priors.     

Bayesian predictive modeling for Inverse Gamma-Pareto composite distribution

Inverse Gamma-Pareto composite distribution is considered as a model for heavy tailed data. The maximum likelihood (ML), smoothed empirical percentile (SM), and Bayes estimators (informative and non-informative) for the parameter θ, which is the boundary point for the supports of the two distributions are derived. A Bayesian predictive density is derived via a gamma prior for θ and the density is used to estimate risk measures. Accuracy of estimators of θ and the risk measures are assessed via simulation studies. It is shown that the informative Bayes estimator is consistently more accurate than ML, Smoothed, and the non-informative Bayes estimators.     

Empirical bayes shrinkage estimation of reliability in the exponential distribution

In this paper we propose two empirical Bayes shrinkage estimators for the reliability of the exponential distribution and study their properties. Under the uniform prior distribution and the inverted gamma prior distribution these estimators are developed and compared with a preliminary test estimator and with a shrinkage testimator in terms of mean squared error. The proposed empirical Bayes shrinkage estimator under the inverted gamma prior distribution is shown to be preferable to the preliminary test estimator and the shrinkage testimator when the prior value of mean life is clsoe to the true mean life.     

Empirical Bayes approach to wavelet regression using ϵ-contaminated priors

We consider an empirical Bayes approach to standard nonparametric regression estimation using a nonlinear wavelet methodology. Instead of specifying a single prior distribution on the parameter space of wavelet coefficients, which is usually the case in the existing literature, we elicit the ?-contamination class of prior distributions that is particularly attractive to work with when one seeks robust priors in Bayesian analysis. The type II maximum likelihood approach to prior selection is used by maximizing the predictive distribution for the data in the wavelet domain over a suitable subclass of the ?-contamination class of prior distributions. For the prior selected, the posterior mean yields a thresholding procedure which depends on one free prior parameter and it is level- and amplitude-dependent, thus allowing better adaptation in function estimation. We consider an automatic choice of the free prior parameter, guided by considerations on an exact risk analysis and on the shape of the thresholding rule, enabling the resulting estimator to be fully automated in practice. We also compute pointwise Bayesian credible intervals for the resulting function estimate using a simulation-based approach. We use several simulated examples to illustrate the performance of the proposed empirical Bayes term-by-term wavelet scheme, and we make comparisons with other classical and empirical Bayes term-by-term wavelet schemes. As a practical illustration, we present an application to a real-life data set that was collected in an atomic force microscopy study.     

Estimation of the scale parameter of the Rayleigh distribution under general progressive censoring

Based on a general progressively type II censored sample, the maximum likelihood estimator (MLE), Bayes estimator under squared error loss and credible intervals for the scale parameter and the reliability function of the Rayleigh distribution are derived. Also, the Bayes predictive estimator and highest posterior density (HPD) prediction interval for future observation are considered. Comparisons among estimators are investigated through Monte Carlo simulations. An illustrative example with real data concerning 23 ball bearings in a life test is presented.     

Classical and Bayesian estimation of stress-strength reliability in type II censored Pareto distributions

ABSTRACT

In this paper, the stress-strength reliability, R, is estimated in type II censored samples from Pareto distributions. The classical inference includes obtaining the maximum likelihood estimator, an exact confidence interval, and the confidence intervals based on Wald and signed log-likelihood ratio statistics. Bayesian inference includes obtaining Bayes estimator, equi-tailed credible interval, and highest posterior density (HPD) interval given both informative and non-informative prior distributions. Bayes estimator of R is obtained using four methods: Lindley's approximation, Tierney-Kadane method, Monte Carlo integration, and MCMC. Also, we compare the proposed methods by simulation study and provide a real example to illustrate them.     

Nonparametric Bayes estimation of gap-time distribution with recurrent event data

Nonparametric Bayes (NPB) estimation of the gap-time survivor function governing the time to occurrence of a recurrent event in the presence of censoring is considered. In our Bayesian approach, the gap-time distribution, denoted by F, has a Dirichlet process prior with parameter α. We derive NPB and nonparametric empirical Bayes (NPEB) estimators of the survivor function F?=1?F and construct point-wise credible intervals. The resulting Bayes estimator of F? extends that based on single-event right-censored data, and the PL-type estimator is a limiting case of this Bayes estimator. Through simulation studies, we demonstrate that the PL-type estimator has smaller biases but higher root-mean-squared errors (RMSEs) than those of the NPB and the NPEB estimators. Even in the case of a mis-specified prior measure parameter α, the NPB and the NPEB estimators have smaller RMSEs than the PL-type estimator, indicating robustness of the NPB and NPEB estimators. In addition, the NPB and NPEB estimators are smoother (in some sense) than the PL-type estimator.     

Multivariate generalized Birnbaum—Saunders kernel density estimators

In this article, we first propose the classical multivariate generalized Birnbaum–Saunders kernel estimator for probability density function estimation in the context of multivariate non negative data. Then, we apply two multiplicative bias correction (MBC) techniques for multivariate kernel density estimator. Some properties (bias, variance, and mean integrated squared error) of the corresponding estimators are also investigated. Finally, the performances of the classical and MBC estimators based on family of generalized Birnbaum–Saunders kernels are illustrated by a simulation study.     

Empirical Bayes estimation for the mean of the selected normal population when there are additional data

ABSTRACT

This paper is concerned with the problem of estimation for the mean of the selected population from two normal populations with unknown means and common known variance in a Bayesian framework. The empirical Bayes estimator, when there are available additional observations, is derived and its bias and risk function are computed. The expected bias and risk of the empirical Bayes estimator and the intuitive estimator are compared. It is shown that the empirical Bayes estimator is asymptotically optimal and especially dominates the intuitive estimator in terms of Bayes risk, with respect to any normal prior. Also, the Bayesian correlation between the mean of the selected population (random parameter) and some interested estimators are obtained and compared.     

A note on bias reduction in variable-kernel density estimates

It is well known that the inverse-square-root rule of Abramson (1982) for the bandwidth h of a variable-kernel density estimator achieves a reduction in bias from the fixed-bandwidth estimator, even when a nonnegative kernel is used. Without some form of “clipping” device similar to that of Abramson, the asymptotic bias can be much greater than O(h⁴) for target densities like the normal (Terrell and Scott 1992) or even compactly supported densities. However, Abramson used a nonsmooth clipping procedure intended for pointwise estimation. Instead, we propose a smoothly clipped estimator and establish a globally valid, uniformly convergent bias expansion for densities with uniformly continuous fourth derivatives. The main result extends Hall's (1990) formula (see also Terrell and Scott 1992) to several dimensions, and actually to a very general class of estimators. By allowing a clipping parameter to vary with the bandwidth, the usual O(h⁴) bias expression holds uniformly on any set where the target density is bounded away from zero.     

Consistency of minimizing a penalized density power divergence estimator for mixing distribution

In this paper, we study the MDPDE (minimizing a density power divergence estimator), proposed by Basu et al. (Biometrika 85:549–559, 1998), for mixing distributions whose component densities are members of some known parametric family. As with the ordinary MDPDE, we also consider a penalized version of the estimator, and show that they are consistent in the sense of weak convergence. A simulation result is provided to illustrate the robustness. Finally, we apply the penalized method to analyzing the red blood cell SLC data presented in Roeder (J Am Stat Assoc 89:487–495, 1994). This research was supported (in part) by KOSEF through Statistical Research Center for Complex Systems at Seoul National University.     

The Bayes rule of the variance parameter of the hierarchical normal and inverse gamma model under Stein's loss

For the variance parameter of the hierarchical normal and inverse gamma model, we analytically calculate the Bayes rule (estimator) with respect to a prior distribution IG (alpha, beta) under Stein's loss function. This estimator minimizes the posterior expected Stein's loss (PESL). We also analytically calculate the Bayes rule and the PESL under the squared error loss. Finally, the numerical simulations exemplify that the PESLs depend only on alpha and the number of observations. The Bayes rules and PESLs under Stein's loss are unanimously smaller than those under the squared error loss.     

Empirical bayes estimation of binomial parameter with symmetric priors

This paper deals with the problem of estimating the binomial parameter via the nonparametric empirical Bayes approach. This estimation problem has the feature that estimators which are asymptotically optimal in the usual empirical Bayes sense do not exist (Robbins (1958, 1964)), However, as pointed out by Liang (1934) and Gupta and Liang (1988), it is possible to construct asymptotically optimal empirical Bayes estimators if the unknown prior is symmetric about the point 1/2, In this paper, assuming symmetric priors a monotone empirical Bayes estimator is constructed by using the isotonic regression method. This estimator is asymptotically optimal in the usual empirical Bayes sense. The corresponding rate of convergence is investigated and shown to be of order n^-1, where n is the number of past observations at hand.     

Bayesian shrinkage estimation based on Rayleigh type-II censored data

In this paper, we construct a Bayes shrinkage estimator for the Rayleigh scale parameter based on censored data under the squared log error loss function. Risk-unbiased estimator is derived and its risk is computed. A Bayes shrinkage estimator is obtained when a prior point guess value is available for the scale parameter. Risk-bias of the Bayes shrinkage estimator is considered. A comparison between the proposed Bayes shrinkage estimator and the risk-unbiased estimator is provided using calculation of the relative efficiency. A numerical example is presented for illustrative and comparative purposes.