Robust Bayesian analysis of generalized inverted family of distributions

ABSTRACT

In the current study we develop the robust Bayesian inference for the generalized inverted family of distributions (GIFD) under an ε-contamination class of prior distributions for the shape parameter α, with different possibilities of known and unknown scale parameter. We used Type II censoring and Bartholomew sampling scheme (1963) for the following derivations under the squared-error loss function (SELF) and linear exponential (LINEX) loss function : ML-II Bayes estimators of the i) parameters; ii) Reliability function and; iii) Hazard function. We also present simulation study and analysis of a real data set.     

On Tiku's robust procedure — a Bayesian insight

Tiku's robust procedure for testing mean and variance from nonnormal universe is examined from the Bayesian viewpoint. The posterior distribution of the scale parameter is derived and then approximated by a Laguerre polynomial expansion while the posterior distribution of the location parameter is approximated by a linear combination of t-distributions. For the example with Darwin's data, the approximations appear to be extremely good.     

Bayesian estimates in a one-way ANOVA random effects model

Bayesian estimators of variance components are developed, based on posterior mean and posterior mode, respectively, in a one-way ANOVA random effects model with independent prior distributions. The formulas for the proposed estimators are simple. The estimators give sensible results for 'badly-behaved' datasets, where the standard unbiased estimates are negative. They are markedly robust as compared to the existing estimators such as the maximum likelihood estimators and the maximum posterior density estimators.     

Qualitative robustness in abstract inference

Hampel's concept of qualitative robustness (or stability) is applied to estimates of ‘generalized parameters’ (that is, estimates which take values in an abstract metric space). After a general introduction, the interest is focussed on three particular topics:

• 1.(a) Multivariate density estimates: We prove the incompatibility between robustness and consistency. As a consequence, a result of instability for kernel estimates is shown. A robust version of these estimates is also obtained.
• 2.(b) Estimation of the mean function of stochastic process with continuous trajectories on [0, 1]: The ‘sampling mean’ is proved to be unstable (thus, the situation is analogous to that of the parametric case). A general method for constructing robust estimates is given.
• 3.(c) Bayesian inference: The posterior distribution is considered as a generalized estimate taking values in the metric space of probability measures. We apply Hampel's definition to this estimate and we show its instability under two specific Bayesian models.

A discussion of the results is included in the last section.     

Median loss decision theory

In this paper, we argue that replacing the expectation of the loss in statistical decision theory with the median of the loss leads to a viable and useful alternative to conventional risk minimization particularly because it can be used with heavy tailed distributions. We investigate three possible definitions for such medloss estimators and derive examples of them in several standard settings. We argue that the medloss definition based on the posterior distribution is better than the other two definitions that do not permit optimization over large classes of estimators. We argue that median loss minimizing estimates often yield improved performance, have resistance to outliers as high as the usual robust estimates, and are resistant to the specific loss used to form them. In simulations with the posterior medloss formulation, we show how the estimates can be obtained numerically and that they can have better robustness properties than estimates derived from risk minimization.     

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

When Gaussian errors are inappropriate in a multivariate linear regression setting, it is often assumed that the errors are iid from a distribution that is a scale mixture of multivariate normals. Combining this robust regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX‐DA algorithm that can be used to explore this posterior. This paper provides conditions (on the mixing density) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo algorithms. Letting d denote the dimension of the response, the main result shows that the DA and Haar PX‐DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log‐normal, inverted Gamma (with shape parameter larger than d /2) or Fréchet (with shape parameter larger than d /2). The results also apply to certain subsets of the Gamma, F and Weibull families.     

A Bayesian Approach for Multiple Response Surface Optimization in the Presence of Noise Variables

An approach for the multiple response robust parameter design problem based on a methodology by Peterson (2000) is presented. The approach is Bayesian, and consists of maximizing the posterior predictive probability that the process satisfies a set of constraints on the responses. In order to find a solution robust to variation in the noise variables, the predictive density is integrated not only with respect to the response variables but also with respect to the assumed distribution of the noise variables. The maximization problem involves repeated Monte Carlo integrations, and two different methods to solve it are evaluated. A Matlab code was written that rapidly finds an optimal (robust) solution in case it exists. Two examples taken from the literature are used to illustrate the proposed method.     

Posterior ranges of functions of parameters under priors with specified quantiles

Suppose some quantiles of the prior distribution of a nonnegative parameter θ are specified. Instead of eliciting just one prior density function, consider the class Γ of all the density functions compatible with the quantile specification. Given a likelihood function, find the posterior upper and lower bounds for the expected value of any real-valued function h(θ), as the density varies in Γ. Such a scheme agrees with a robust Bayesian viewpoint. Under mild regularity conditions about h(θ) and the likelihood, a procedure for finding bounds is derived and applied to an example, after transforming the given functional optimisation problems into finite-dimensional ones.     

Nonparametric Bayes methods for directional data

A model for directional data in q dimensions is studied. The data are assumed to arise from a distribution with a density on a sphere of q — 1 dimensions. The density is unimodal and rotationally symmetric, but otherwise of unknown form. The posterior distribution of the unknown mode (mean direction) is derived, and small-sample posterior inference is discussed. The posterior mean of the density is also given. A numerical method for evaluating posterior quantities based on sampling a Markov chain is introduced. This method is generally applicable to problems involving unknown monotone functions.     

A procedure to select a ML-II prior in a multivariate normal case

Berger (1985) derived a procedure to select a maximum likelihood II prior distribution. In this paper a method is suggested to construct such a prior distribution from a multivariate ε-contamination class of distributions. The method is illustrated by the conetruction of a ML-II prior in the multivariate normal case.     

Robust likelihood functions in Bayesian inference

In order to deal with mild deviations from the assumed parametric model, we propose a procedure for accounting for model uncertainty in the Bayesian framework. In particular, in the derivation of posterior distributions, we discuss the use of robust pseudo-likelihoods, which offer the advantage of preventing the effects caused by model misspecifications, i.e. when the underlying distribution lies in a neighborhood of the assumed model. The influence functions of posterior summaries, such as the posterior mean, are investigated as well as the asymptotic properties of robust posterior distributions. Although the use of a pseudo-likelihood cannot be considered orthodox in the Bayesian perspective, it is shown that, also through some illustrative examples, how a robust pseudo-likelihood, with the same asymptotic properties of a genuine likelihood, can be useful in the inferential process in order to prevent the effects caused by model misspecifications.     

On D-optimal robust second order slope-rotatable designs

Das and Park (2006) introduced slope-rotatable designs overall directions for correlated observations which is known as A-optimal robust slope-rotatable designs. This article focuses D-optimal slope-rotatable designs for second-order response surface model with correlated observations. It has been established that robust second-order rotatable designs are also D-optimal robust slope-rotatable designs. A class of D-optimal robust second-order slope-rotatable designs has been derived for special correlation structures of errors.     

Adaptive density estimation: A curse of support?

This paper deals with the classical problem of density estimation on the real line. Most of the existing papers devoted to minimax properties assume that the support of the underlying density is bounded and known. But this assumption may be very difficult to handle in practice. In this work, we show that, exactly as a curse of dimensionality exists when the data lie in R^d, there exists a curse of support as well when the support of the density is infinite. As for the dimensionality problem where the rates of convergence deteriorate when the dimension grows, the minimax rates of convergence may deteriorate as well when the support becomes infinite. This problem is not purely theoretical since the simulations show that the support-dependent methods are really affected in practice by the size of the density support, or by the weight of the density tail. We propose a method based on a biorthogonal wavelet thresholding rule that is adaptive with respect to the nature of the support and the regularity of the signal, but that is also robust in practice to this curse of support. The threshold, that is proposed here, is very accurately calibrated so that the gap between optimal theoretical and practical tuning parameters is almost filled.     

Nonparametric hierarchical Bayes analysis of binomial data via Bernstein polynomial priors

For binomial data analysis, many methods based on empirical Bayes interpretations have been developed, in which a variance‐stabilizing transformation and a normality assumption are usually required. To achieve the greatest model flexibility, we conduct nonparametric Bayesian inference for binomial data and employ a special nonparametric Bayesian prior—the Bernstein–Dirichlet process (BDP)—in the hierarchical Bayes model for the data. The BDP is a special Dirichlet process (DP) mixture based on beta distributions, and the posterior distribution resulting from it has a smooth density defined on [0, 1]. We examine two Markov chain Monte Carlo procedures for simulating from the resulting posterior distribution, and compare their convergence rates and computational efficiency. In contrast to existing results for posterior consistency based on direct observations, the posterior consistency of the BDP, given indirect binomial data, is established. We study shrinkage effects and the robustness of the BDP‐based posterior estimators in comparison with several other empirical and hierarchical Bayes estimators, and we illustrate through examples that the BDP‐based nonparametric Bayesian estimate is more robust to the sample variation and tends to have a smaller estimation error than those based on the DP prior. In certain settings, the new estimator can also beat Stein's estimator, Efron and Morris's limited‐translation estimator, and many other existing empirical Bayes estimators. The Canadian Journal of Statistics 40: 328–344; 2012 © 2012 Statistical Society of Canada     

Objective Bayesian analysis for the multivariate skew-t model

We propose a novel Bayesian analysis of the p-variate skew-t model, providing a new parameterization, a set of non-informative priors and a sampler specifically designed to explore the posterior density of the model parameters. Extensions, such as the multivariate regression model with skewed errors and the stochastic frontiers model, are easily accommodated. A novelty introduced in the paper is given by the extension of the bivariate skew-normal model given in Liseo and Parisi (2013) to a more realistic p-variate skew-t model. We also introduce the R package mvst, which produces a posterior sample for the parameters of a multivariate skew-t model.     

On D-optimal robust designs for lifetime improvement experiments

Locating the optimal operating conditions of the process parameters is critical in a lifetime improvement experiment. For log-normal lifetime distribution with compound error structure (i.e., symmetry, inter-class and intra-class correlation error structures), we have developed methods for construction of D-optimal robust first order designs. It is shown that D-optimal robust first order designs are always robust first order rotatable but the converse is not always true.     

Robust Bayesian estimation of a two-parameter exponential distribution under generalized Type-I progressive hybrid censoring

A generalized Type-I progressive hybrid censoring scheme was proposed recently to overcome the limitations of the progressive hybrid censoring scheme. In this article, we provide a robust Bayesian method to estimate the unknown parameters of the two-parameter exponential distribution of a generalized Type-I progressive hybrid censored sample. For each parameter, we derive the marginal posterior density functions and the corresponding Bayesian estimators under the squared error loss function. To assess the proposed method, Monte Carlo simulations are performed using a real dataset.     

An automatic robust Bayesian approach to principal component regression

Principal component regression uses principal components (PCs) as regressors. It is particularly useful in prediction settings with high-dimensional covariates. The existing literature treating of Bayesian approaches is relatively sparse. We introduce a Bayesian approach that is robust to outliers in both the dependent variable and the covariates. Outliers can be thought of as observations that are not in line with the general trend. The proposed approach automatically penalises these observations so that their impact on the posterior gradually vanishes as they move further and further away from the general trend, corresponding to a concept in Bayesian statistics called whole robustness. The predictions produced are thus consistent with the bulk of the data. The approach also exploits the geometry of PCs to efficiently identify those that are significant. Individual predictions obtained from the resulting models are consolidated according to model-averaging mechanisms to account for model uncertainty. The approach is evaluated on real data and compared to its nonrobust Bayesian counterpart, the traditional frequentist approach and a commonly employed robust frequentist method. Detailed guidelines to automate the entire statistical procedure are provided. All required code is made available, see ArXiv:1711.06341.     

Semiparametric Bayesian Techniques for Problems in Circular Data

In this paper, we consider the problems of prediction and tests of hypotheses for directional data in a semiparametric Bayesian set-up. Observations are assumed to be independently drawn from the von Mises distribution and uncertainty in the location parameter is modelled by a Dirichlet process. For the prediction problem, we present a method to obtain the predictive density of a future observation, and, for the testing problem, we present a method of computing the Bayes factor by obtaining the posterior probabilities of the hypotheses under consideration. The semiparametric model is seen to be flexible and robust against prior misspecifications. While analytical expressions are intractable, the methods are easily implemented using the Gibbs sampler. We illustrate the methods with data from two real-life examples.     

Concentration function and sensitivity to the prior

Summary In robust bayesian analysis, ranges of quantities of interest (e. g. posterior means) are usually considered when the prior probability measure varies in a class Γ. Such quantities describe the variation of just one aspect of the posterior measure. The concentration function describes changes in the posterior probability measure more globally, detecting differences in probability concentration and providing, simultaneously, bounds on the posterior probability of all measurable subsets. In this paper, we present a novel use of the concentration function, and two concentration indices, to study such posterior changes for a general class Γ, restricting then our attention to some ∈-contamination classes of priors.