Elicitation of multivariate prior distributions: A nonparametric Bayesian approach

In the context of Bayesian statistical analysis, elicitation is the process of formulating a prior density f(·)$f(·)$ about one or more uncertain quantities to represent a person's knowledge and beliefs. Several different methods of eliciting prior distributions for one unknown parameter have been proposed. However, there are relatively few methods for specifying a multivariate prior distribution and most are just applicable to specific classes of problems and/or based on restrictive conditions, such as independence of variables. Besides, many of these procedures require the elicitation of variances and correlations, and sometimes elicitation of hyperparameters which are difficult for experts to specify in practice. Garthwaite et al. (2005) discuss the different methods proposed in the literature and the difficulties of eliciting multivariate prior distributions. We describe a flexible method of eliciting multivariate prior distributions applicable to a wide class of practical problems. Our approach does not assume a parametric form for the unknown prior density f(·)$f(·)$, instead we use nonparametric Bayesian inference, modelling f(·)$f(·)$ by a Gaussian process prior distribution. The expert is then asked to specify certain summaries of his/her distribution, such as the mean, mode, marginal quantiles and a small number of joint probabilities. The analyst receives that information, treating it as a data set D   with which to update his/her prior beliefs to obtain the posterior distribution for f(·)$f(·)$. Theoretical properties of joint and marginal priors are derived and numerical illustrations to demonstrate our approach are given.     

The 'heuristics and biases' bias in expert elicitation

Summary.  In the early 1970s Tversky and Kahneman published a series of papers on 'heuristics and biases' describing human inadequacies in assessing probabilities, culminating in a highly popular article in Science . This seminal research has been heavily cited in many fields, including statistics, as the definitive research on probability assessment. Curiously, although this work was debated at the time and more recent work has largely refuted many of the claims, this apparent heuristics and biases bias in elicitation research has gone unremarked. Over a decade of research into the frequency effect, the importance of framing, and cognitive models more generally, has been almost completely ignored by the statistical literature on expert elicitation. To remedy this situation, this review offers a guide to the psychological research on assessing probabilities, both old and new, and gives concrete guidelines for eliciting expert knowledge.     

Non-conjugate prior distribution assessment for multivariate normal sampling

Elicitation methods are proposed for quantifying expert opinion about a multivariate normal sampling model. The natural conjugate prior family imposes a relationship between the mean vector and the covariance matrix that can portray an expert's opinion poorly. Instead we assume that opinions about the mean and the covariance are independent and suggest innovative forms of question which enable the expert to quantify separately his or her opinion about each of these parameters. Prior opinion about the mean vector is modelled by a multivariate normal distribution and about the covariance matrix by both an inverse Wishart distribution and a generalized inverse-Wishart (GIW) distribution. To construct the latter, results are developed that give insight into the GIW parameters and their interrelationships. Certain of the elicitation methods exploit unconditional assessments as fully as possible, since these can reflect an expert's beliefs more accurately than conditional assessments. Methods are illustrated through an example.     

Robust modelling for asset management

We argue that the principal way to achieve robustness is through good elicitation. This means asking the right questions, to elicit only those judgements which can be given reliably. Bayes linear methods minimise the number of prior judgements employed in the analysis. Only first- and second-order moments are specified. We present an example in which a complex problem is modelled in terms of a relatively small number of meaningful prior judgements. Sensitivity to variations in those judgements is explored here and in Goldstein and Wooff (1992).     

《社会学原理》课程与启发式教学法

本文通过对《社会学原理》课程基本结构的分析．展示了启发式教学法与此门课程教学重点和难点的关系,并阐述了该方法在此门课程中的四种运用形式．     

An application of robust Bayesian analysis to a medical experiment

This paper reviews a design for a clinical trail that uses expert opinion to guide the selection of treatments for patients in a way intended to be more favorable than random selection. The problems of analyzing data from the design are discussed. Using real data from a trial of two agents for treating hypertension after open heart surgery, issues of how to display the data are considered, and the extent to which the design and analysis may be robust to elicitation error is discussed.     

A Bayesian approach to Markov modelling in cost-effectiveness analyses: application to taxane use in advanced breast cancer

Summary. The paper demonstrates how cost-effectiveness decision analysis may be implemented from a Bayesian perspective, using Markov chain Monte Carlo simulation methods for both the synthesis of relevant evidence input into the model and the evaluation of the model itself. The desirable aspects of a Bayesian approach for this type of analysis include the incorporation of full parameter uncertainty, the ability to perform all the analysis, including each meta-analysis, in a single coherent model and the incorporation of expert opinion either directly or regarding the relative credibility of different data sources. The method is described, and its ease of implementation demonstrated, through a practical example to evaluate the cost-effectiveness of using taxanes for the second-line treatment of advanced breast cancer compared with conventional treatment. For completeness, the results from the Markov chain Monte Carlo simulation model are compared and contrasted with those from a classical Monte Carlo simulation model.     

Comment: A Bayesian Approach to the Nonresponse Problem,Using Covariates à la Freedman

We propose a flexible method to approximate the subjective cumulative distribution function of an economic agent about the future realization of a continuous random variable. The method can closely approximate a wide variety of distributions while maintaining weak assumptions on the shape of distribution functions. We show how moments and quantiles of general functions of the random variable can be computed analytically and/or numerically. We illustrate the method by revisiting the determinants of income expectations in the United States. A Monte Carlo analysis suggests that a quantile-based flexible approach can be used to successfully deal with censoring and possible rounding levels present in the data. Finally, our analysis suggests that the performance of our flexible approach matches that of a correctly specified parametric approach and is clearly better than that of a misspecified parametric approach.     

Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems*

There have been a number of multiattribute decision aids developed to aid selection problems. Multiattribute value theory and the analytic hierarchy process are two commonly used techniques. Different systems can result in radically different conclusions if they inaccurately and inconsistently reflect the preference structure of decision makers, or if they are based on inappropriate theoretical models. This study examines the impact of the underlying theoretical model, the method in which preference information is elicited, and the structure of alternatives as influences on the results from using various decision aids. It was found that two systems based on the multiattribute value theory model were just as diverse in their conclusions as were results between AHP and the multiattribute value theory models. Therefore, accuracy of information reflecting decision maker preference is an important consideration. Feedback capable of assuring the decision maker that information provided is consistent is a necessary feature required of decision aids applied to selection problems. The study also found that the way in which information is elicited influenced the result more than did the underlying model. Exact numerical data for complex concepts such as attribute importance and alternative performance on attributes is not necessary, and elicitation procedures that are more natural for the user are likely to be more accurate.     

Bias modelling in evidence synthesis

Summary.  Policy decisions often require synthesis of evidence from multiple sources, and the source studies typically vary in rigour and in relevance to the target question. We present simple methods of allowing for differences in rigour (or lack of internal bias) and relevance (or lack of external bias) in evidence synthesis. The methods are developed in the context of reanalysing a UK National Institute for Clinical Excellence technology appraisal in antenatal care, which includes eight comparative studies. Many were historically controlled, only one was a randomized trial and doses, populations and outcomes varied between studies and differed from the target UK setting. Using elicited opinion, we construct prior distributions to represent the biases in each study and perform a bias-adjusted meta-analysis. Adjustment had the effect of shifting the combined estimate away from the null by approximately 10%, and the variance of the combined estimate was almost tripled. Our generic bias modelling approach allows decisions to be based on all available evidence, with less rigorous or less relevant studies downweighted by using computationally simple methods.