A Robust Conflict Measure of Inconsistencies in Bayesian Hierarchical Models

Abstract.  O'Hagan ( Highly Structured Stochastic Systems , Oxford University Press, Oxford, 2003) introduces some tools for criticism of Bayesian hierarchical models that can be applied at each node of the model, with a view to diagnosing problems of model fit at any point in the model structure. His method relies on computing the posterior median of a conflict index, typically through Markov chain Monte Carlo simulations. We investigate a Gaussian model of one-way analysis of variance, and show that O'Hagan's approach gives unreliable false warning probabilities. We extend and refine the method, especially avoiding double use of data by a data-splitting approach, accompanied by theoretical justifications from a non-trivial special case. Through extensive numerical experiments we show that our method detects model mis-specification about as well as the method of O'Hagan, while retaining the desired false warning probability for data generated from the assumed model. This also holds for Student's- t and uniform distribution versions of the model.     

Extensions of a Conflict Measure of Inconsistencies in Bayesian Hierarchical Models

Abstract.  In a recent paper we extended and refined some tools introduced by O'Hagan for criticism of Bayesian hierarchical models. Especially, avoiding double use of data by a data-splitting approach was a main concern. Such tools can be applied at each node of the model, with a view to diagnosing problems of model fit at any point in the model structure. As O'Hagan, we investigated a Gaussian model of one-way analysis of variance. Through extensive Markov chain Monte Carlo simulations it was shown that our method detects model misspecification about as well as the one of O'Hagan, when this is properly calibrated, while retaining the desired false warning probability for data generated from the assumed model. In the present paper, we suggest some new measures of conflict based on tail probabilities of the so-called integrated posterior distributions introduced in our recent paper. These new measures are equivalent to the measure applied in the latter paper in simple Gaussian models, but seem more appropriately adjusted to deviations from normality and to conflicts not concerning location parameters. A general linear normal model with known covariance matrices is considered in detail.