Variational Inference for Heteroscedastic Semiparametric Regression

We develop fast mean field variational methodology for Bayesian heteroscedastic semiparametric regression, in which both the mean and variance are smooth, but otherwise arbitrary, functions of the predictors. Our resulting algorithms are purely algebraic, devoid of numerical integration and Monte Carlo sampling. The locality property of mean field variational Bayes implies that the methodology also applies to larger models possessing variance function components. Simulation studies indicate good to excellent accuracy, and considerable time savings compared with Markov chain Monte Carlo. We also provide some illustrations from applications.     

Variational Bayes for estimating the parameters of a hidden Potts model

Hidden Markov random field models provide an appealing representation of images and other spatial problems. The drawback is that inference is not straightforward for these models as the normalisation constant for the likelihood is generally intractable except for very small observation sets. Variational methods are an emerging tool for Bayesian inference and they have already been successfully applied in other contexts. Focusing on the particular case of a hidden Potts model with Gaussian noise, we show how variational Bayesian methods can be applied to hidden Markov random field inference. To tackle the obstacle of the intractable normalising constant for the likelihood, we explore alternative estimation approaches for incorporation into the variational Bayes algorithm. We consider a pseudo-likelihood approach as well as the more recent reduced dependence approximation of the normalisation constant. To illustrate the effectiveness of these approaches we present empirical results from the analysis of simulated datasets. We also analyse a real dataset and compare results with those of previous analyses as well as those obtained from the recently developed auxiliary variable MCMC method and the recursive MCMC method. Our results show that the variational Bayesian analyses can be carried out much faster than the MCMC analyses and produce good estimates of model parameters. We also found that the reduced dependence approximation of the normalisation constant outperformed the pseudo-likelihood approximation in our analysis of real and synthetic datasets.     

GENERALIZED EXTREME VALUE ADDITIVE MODEL ANALYSIS VIA MEAN FIELD VARIATIONAL BAYES

We develop Mean Field Variational Bayes methodology for fast approximate inference in Bayesian Generalized Extreme Value additive model analysis. Such models are useful for flexibly assessing the impact of continuous predictor variables on sample extremes. The new methodology allows large Bayesian models to be fitted and assessed without the significant computing costs of Markov Chain Monte Carlo methods. We illustrate our new methodology with maximum rainfall data from the Sydney, Australia, hinterland. Comparisons are made between the Mean Field Variational Bayes and Markov Chain Monte Carlo approaches.     

Model influence functions based on mixtures

Influence functions are considered as diagnostics for model departures in parametric Bayesian inference. A baseline model density is expressed as a mixture; then the mixing distribution is perturbed. This is designed to engender perturbations which are plausible a priori. The influence of perturbations is measured for both Bayes estimates and their associated posterior expected losses. To assess the plausibility of perturbations a posteriori, an additional influence function is constructed for the Bayes factor comparing the perturbed and baseline models. The effect of perturbation on various estimands is incorporated in the analysis.     

Confidence distributions applied to propagating uncertainty to inference based on estimating the local false discovery rate: A fiducial continuum from confidence sets to empirical Bayes set estimates as the number of comparisons increases

Empirical Bayes estimates of the local false discovery rate can reflect uncertainty about the estimated prior by supplementing their Bayesian posterior probabilities with confidence levels as posterior probabilities. This use of coherent fiducial inference with hierarchical models generates set estimators that propagate uncertainty to varying degrees. Some of the set estimates approach estimates from plug-in empirical Bayes methods for high numbers of comparisons and can come close to the usual confidence sets given a sufficiently low number of comparisons.     

Bayes estimates in multivariate semiparametric linear models

Bayes estimates are derived in multivariate linear models with unknown distribution. The prior distribution is defined using a Dirichlet prior for the unknown error distribution and a normal-Wishart distribution for the parameters. The posterior distribution is determined and explicit expressions are given in the special cases of location-scale and two-sample models. The calculation of self-informative limits of Bayes estimates yields standard estimates.     

Estimation under inverse Weibull distribution based on Balakrishnan’s unified hybrid censored scheme

In this article, point and interval estimations of the parameters α and β of the inverse Weibull distribution (IWD) have been studied based on Balakrishnan’s unified hybrid censoring scheme (UHCS), see Balakrishnan et al. In point estimation, the maximum likelihood (ML) and Bayes (B) methods have been used. The Bayes estimates have been computed based on squared error loss (SEL) function and Linex loss function and using Markov Chain Monte Carlo (MCMC) algorithm. In interval estimation, a (1 ? τ) × 100% approximate, bootstrap-p, credible and highest posterior density (HPD) confidence intervals (CI′s) for the parameters α and β have been introduced. Based on Monte Carlo simulation, Bayes estimates have been compared with their corresponding maximum likelihood estimates by computing the mean squared errors (MSE′s) of all estimators. Finally, point and interval estimations of all parameters have been studied based on a real data set as an illustrative example.     

Particle Learning for Fat-Tailed Distributions

It is well known that parameter estimates and forecasts are sensitive to assumptions about the tail behavior of the error distribution. In this article, we develop an approach to sequential inference that also simultaneously estimates the tail of the accompanying error distribution. Our simulation-based approach models errors with a t_ν-distribution and, as new data arrives, we sequentially compute the marginal posterior distribution of the tail thickness. Our method naturally incorporates fat-tailed error distributions and can be extended to other data features such as stochastic volatility. We show that the sequential Bayes factor provides an optimal test of fat-tails versus normality. We provide an empirical and theoretical analysis of the rate of learning of tail thickness under a default Jeffreys prior. We illustrate our sequential methodology on the British pound/U.S. dollar daily exchange rate data and on data from the 2008–2009 credit crisis using daily S&P500 returns. Our method naturally extends to multivariate and dynamic panel data.     

Alternative Bayes factors for model selection

Several alternative Bayes factors have been recently proposed in order to solve the problem of the extreme sensitivity of the Bayes factor to the priors of models under comparison. Specifically, the impossibility of using the Bayes factor with standard noninformative priors for model comparison has led to the introduction of new automatic criteria, such as the posterior Bayes factor (Aitkin 1991), the intrinsic Bayes factors (Berger and Pericchi 1996b) and the fractional Bayes factor (O'Hagan 1995). We derive some interesting properties of the fractional Bayes factor that provide justifications for its use additional to the ones given by O'Hagan. We further argue that the use of the fractional Bayes factor, originally introduced to cope with improper priors, is also useful in a robust analysis. Finally, using usual classes of priors, we compare several alternative Bayes factors for the problem of testing the point null hypothesis in the univariate normal model.     

Empirical Bayes sequential estimation of a finite population mean

In this article, we consider the Bayes and empirical Bayes problem of the current population mean of a finite population when the sample data is available from other similar (m-1) finite populations. We investigate a general class of linear estimators and obtain the optimal linear Bayes estimator of the finite population mean under a squared error loss function that considered the cost of sampling. The optimal linear Bayes estimator and the sample size are obtained as a function of the parameters of the prior distribution. The corresponding empirical Bayes estimates are obtained by replacing the unknown hyperparameters with their respective consistent estimates. A Monte Carlo study is conducted to evaluate the performance of the proposed empirical Bayes procedure.     

Spatially-adaptive Penalties for Spline Fitting

The paper studies spline fitting with a roughness penalty that adapts to spatial heterogeneity in the regression function. The estimates are p th degree piecewise polynomials with p − 1 continuous derivatives. A large and fixed number of knots is used and smoothing is achieved by putting a quadratic penalty on the jumps of the p th derivative at the knots. To be spatially adaptive, the logarithm of the penalty is itself a linear spline but with relatively few knots and with values at the knots chosen to minimize the generalized cross validation (GCV) criterion. This locally-adaptive spline estimator is compared with other spline estimators in the literature such as cubic smoothing splines and knot-selection techniques for least squares regression. Our estimator can be interpreted as an empirical Bayes estimate for a prior allowing spatial heterogeneity. In cases of spatially heterogeneous regression functions, empirical Bayes confidence intervals using this prior achieve better pointwise coverage probabilities than confidence intervals based on a global-penalty parameter. The method is developed first for univariate models and then extended to additive models.     

Parameter and reliability estimation for an exponentiated half-logistic distribution under progressive type II censoring

In this article, we deal with a two-parameter exponentiated half-logistic distribution. We consider the estimation of unknown parameters, the associated reliability function and the hazard rate function under progressive Type II censoring. Maximum likelihood estimates (M LEs) are proposed for unknown quantities. Bayes estimates are derived with respect to squared error, linex and entropy loss functions. Approximate explicit expressions for all Bayes estimates are obtained using the Lindley method. We also use importance sampling scheme to compute the Bayes estimates. Markov Chain Monte Carlo samples are further used to produce credible intervals for the unknown parameters. Asymptotic confidence intervals are constructed using the normality property of the MLEs. For comparison purposes, bootstrap-p and bootstrap-t confidence intervals are also constructed. A comprehensive numerical study is performed to compare the proposed estimates. Finally, a real-life data set is analysed to illustrate the proposed methods of estimation.     

Bayesian estimation of P (Y<X) from Burr-type X model containing spurious observations

We consider the problem of estimating R=P(Y<X) when X and Y are independent Burr-type X random variables. We assume that the sample from each population contains one spurious observation. Bayes estimates are derived for exchangeable and identifiable cases. Monte Carlo simulation is carried out to compare the bias and the expected loss of R.     

Bayesian inference on P(X > Y) in bivariate Rayleigh model

In the literature, assuming independence of random variables X and Y, statistical estimation of the stress–strength parameter R = P(X > Y) is intensively investigated. However, in some real applications, the strength variable X could be highly dependent on the stress variable Y. In this paper, unlike the common practice in the literature, we discuss on estimation of the parameter R where more realistically X and Y are dependent random variables distributed as bivariate Rayleigh model. We derive the Bayes estimates and highest posterior density credible intervals of the parameters using suitable priors on the parameters. Because there are not closed forms for the Bayes estimates, we will use an approximation based on Laplace method and a Markov Chain Monte Carlo technique to obtain the Bayes estimate of R and unknown parameters. Finally, simulation studies are conducted in order to evaluate the performances of the proposed estimators and analysis of two data sets are provided.     

Inferential statistics on the dynamic system model with time-dependent failure-rate

This study focuses on the classical and Bayesian analysis of a k-components load-sharing parallel system in which components have time-dependent failure rates. In the classical set up, the maximum likelihood estimates of the load-share parameters with their standard errors (SEs) are obtained. (1?γ) 100% simultaneous and two bootstrap confidence intervals for the parameters and system reliability and hazard functions have been constructed. Further, on recognizing the fact that life-testing experiments are very time consuming, the parameters involved in the failure time distribution of the system are expected to follow some random variations. Therefore, Bayes estimates along with their posterior SEs of the parameters and system reliability and hazard functions are obtained by assuming gamma and Jeffrey's priors of the unknown parameters. Markov chain Monte Carlo technique such as Gibbs sampler has been used to obtain Bayes estimates and highest posterior density credible intervals.     

A ROBUST BAYES FACTOR FOR LINEAR MODELS

This paper proposes a new robust Bayes factor for comparing two linear models. The factor is based on a pseudo‐model for outliers and is more robust to outliers than the Bayes factor based on the variance‐inflation model for outliers. If an observation is considered an outlier for both models this new robust Bayes factor equals the Bayes factor calculated after removing the outlier. If an observation is considered an outlier for one model but not the other then this new robust Bayes factor equals the Bayes factor calculated without the observation, but a penalty is applied to the model considering the observation as an outlier. For moderate outliers where the variance‐inflation model is suitable, the two Bayes factors are similar. The new Bayes factor uses a single robustness parameter to describe a priori belief in the likelihood of outliers. Real and synthetic data illustrate the properties of the new robust Bayes factor and highlight the inferior properties of Bayes factors based on the variance‐inflation model for outliers.     

Analyzing the dynamic system model with discrete failure time distribution

The present study deals with the method of estimation of the parameters of k-components load-sharing parallel system model in which each component’s failure time distribution is assumed to be geometric. The maximum likelihood estimates of the load-share parameters with their standard errors are obtained. (1 − γ) 100% joint, Bonferroni simultaneous and two bootstrap confidence intervals for the parameters have been constructed. Further, recognizing the fact that life testing experiments are time consuming, it seems realistic to consider the load-share parameters to be random variable. Therefore, Bayes estimates along with their standard errors of the parameters are obtained by assuming Jeffrey’s invariant and gamma priors for the unknown parameters. Since, Bayes estimators can not be found in closed form expressions, Tierney and Kadane’s approximation method have been used to compute Bayes estimates and standard errors of the parameters. Markov Chain Monte Carlo technique such as Gibbs sampler is also used to obtain Bayes estimates and highest posterior density credible intervals of the load-share parameters. Metropolis–Hastings algorithm is used to generate samples from the posterior distributions of the unknown parameters.     

On Measuring Uncertainty of Benchmarked Predictors with Application to Disease Risk Estimate

Empirical Bayes (EB) estimates in general linear mixed models are useful for the small area estimation in the sense of increasing precision of estimation of small area means. However, one potential difficulty of EB is that the overall estimate for a larger geographical area based on a (weighted) sum of EB estimates is not necessarily identical to the corresponding direct estimate such as the overall sample mean. Another difficulty is that EB estimates yield over‐shrinking, which results in the sampling variance smaller than the posterior variance. One way to fix these problems is the benchmarking approach based on the constrained empirical Bayes (CEB) estimators, which satisfy the constraints that the aggregated mean and variance are identical to the requested values of mean and variance. In this paper, we treat the general mixed models, derive asymptotic approximations of the mean squared error (MSE) of CEB and provide second‐order unbiased estimators of MSE based on the parametric bootstrap method. These results are applied to natural exponential families with quadratic variance functions. As a specific example, the Poisson‐gamma model is dealt with, and it is illustrated that the CEB estimates and their MSE estimates work well through real mortality data.     

Bayes estimation of Gompertz distribution parameters and acceleration factor under partially accelerated life tests with type-I censoring

In this paper, the Bayesian approach is applied to the estimation problem in the case of step stress partially accelerated life tests with two stress levels and type-I censoring. Gompertz distribution is considered as a lifetime model. The posterior means and posterior variances are derived using the squared-error loss function. The Bayes estimates cannot be obtained in explicit forms. Approximate Bayes estimates are computed using the method of Lindley [D.V. Lindley, Approximate Bayesian methods, Trabajos Estadistica 31 (1980), pp. 223–237]. The advantage of this proposed method is shown. The approximate Bayes estimates obtained under the assumption of non-informative priors are compared with their maximum likelihood counterparts using Monte Carlo simulation.     

Exact or approximate inference in graphical models: why the choice is dictated by the treewidth,and how variable elimination can be exploited

Probabilistic graphical models offer a powerful framework to account for the dependence structure between variables, which is represented as a graph. However, the dependence between variables may render inference tasks intractable. In this paper, we review techniques exploiting the graph structure for exact inference, borrowed from optimisation and computer science. They are built on the principle of variable elimination whose complexity is dictated in an intricate way by the order in which variables are eliminated. The so‐called treewidth of the graph characterises this algorithmic complexity: low‐treewidth graphs can be processed efficiently. The first point that we illustrate is therefore the idea that for inference in graphical models, the number of variables is not the limiting factor, and it is worth checking the width of several tree decompositions of the graph before resorting to the approximate method. We show how algorithms providing an upper bound of the treewidth can be exploited to derive a ‘good' elimination order enabling to realise exact inference. The second point is that when the treewidth is too large, algorithms for approximate inference linked to the principle of variable elimination, such as loopy belief propagation and variational approaches, can lead to accurate results while being much less time consuming than Monte‐Carlo approaches. We illustrate the techniques reviewed in this article on benchmarks of inference problems in genetic linkage analysis and computer vision, as well as on hidden variables restoration in coupled Hidden Markov Models.