A Robust Conflict Measure of Inconsistencies in Bayesian Hierarchical Models

Abstract.  O'Hagan ( Highly Structured Stochastic Systems , Oxford University Press, Oxford, 2003) introduces some tools for criticism of Bayesian hierarchical models that can be applied at each node of the model, with a view to diagnosing problems of model fit at any point in the model structure. His method relies on computing the posterior median of a conflict index, typically through Markov chain Monte Carlo simulations. We investigate a Gaussian model of one-way analysis of variance, and show that O'Hagan's approach gives unreliable false warning probabilities. We extend and refine the method, especially avoiding double use of data by a data-splitting approach, accompanied by theoretical justifications from a non-trivial special case. Through extensive numerical experiments we show that our method detects model mis-specification about as well as the method of O'Hagan, while retaining the desired false warning probability for data generated from the assumed model. This also holds for Student's- t and uniform distribution versions of the model.