The Aggregation of Expert Judgment: Do Good Things Come to Those Who Weight?

Good policy making should be based on available scientific knowledge. Sometimes this knowledge is well established through research, but often scientists must simply express their judgment, and this is particularly so in risk scenarios that are characterized by high levels of uncertainty. Usually in such cases, the opinions of several experts will be sought in order to pool knowledge and reduce error, raising the question of whether individual expert judgments should be given different weights. We argue—against the commonly advocated “classical method”—that no significant benefits are likely to accrue from unequal weighting in mathematical aggregation. Our argument hinges on the difficulty of constructing reliable and valid measures of substantive expertise upon which to base weights. Practical problems associated with attempts to evaluate experts are also addressed. While our discussion focuses on one specific weighting scheme that is currently gaining in popularity for expert knowledge elicitation, our general thesis applies to externally imposed unequal weighting schemes more generally.     

Bayesian Inference Under Partial Prior Information

Partial prior information on the marginal distribution of an observable random variable is considered. When this information is incorporated into the statistical analysis of an assumed parametric model, the posterior inference is typically non‐robust so that no inferential conclusion is obtained. To overcome this difficulty a method based on the standard default prior associated to the model and an intrinsic procedure is proposed. Posterior robustness of the resulting inferences is analysed and some illustrative examples are provided.     

Elicitation of Expert Judgments of Uncertainty in the Risk Assessment of Herbicide-Tolerant Oilseed Crops

One of the lay public's concerns about genetically modified (GM) organisms (GMO) and related emerging technologies is that not all the important risks are evaluated or even identified yet--and that ignorance of the unanticipated risks could lead to severe environmental or public health consequences. To some degree, even the scientists who participated in the analysis of the risks from GMOs (arguably the people most qualified to critique these analyses) share some of this concern. To formally explore the uncertainty in the risk assessment of a GM crop, we conducted detailed interviews of seven leading experts on GM oilseed crops to obtain qualitative and quantitative information on their understanding of the uncertainties associated with the risks to agriculture from GM oilseed crops (canola or rapeseed). The results of these elicitations revealed three issues of potential concern that are currently left outside the scope of risk assessments. These are (1) the potential loss of the agronomic and environmental benefits of glyphosate (a herbicide widely used in no-till agriculture) due to the combined problems of glyphosate-tolerant canola and wheat volunteer plants, (2) the growing problem of seed lot contamination, and (3) the potential market impacts. The elicitations also identified two areas where knowledge is insufficient. These are: the occurrence of hybridization between canola and wild relatives and the ability of the hybrids to perpetuate themselves in nature, and the fate of the herbicide-tolerance genes in soil and their interaction with soil microfauna and -flora. The methodological contribution of this work is a formal approach to analyzing the uncertainty surrounding complex problems.     

Identity boxes: using materials and metaphors to elicit experiences

This paper considers the use of identity boxes as a data collection method to elicit experiences. Participants were asked to respond to questions using objects to represent their answers. The rationale for using identity boxes was to allow for a more embodied, less textual approach. The identity boxes were then also used to create an artistic installation as one form of data analysis. The reflection section in this paper shows that the approach posed potential risks around the emotional vulnerability of participants, with participants experiencing the project as cathartic and therapeutic. Some participants struggled with the process of thinking through objects as difficult. Although using a less conventional approach to research led to deeper, richer reflections and thus relevant, interesting data, concerns were raised around relevance, generalisability, and more generally the evaluation of the research. The paper concludes with a reconsideration of the identity boxes as a method.     

Researching family practices in everyday life: methodological reflections from two studies

The study of the everyday is recognised as central to the understanding of identities, agency and social life. Yet, attempts to research everyday life often fail to capture the complexity of the mundane. This paper draws on findings from two studies: fatherhood across three generations and adult narratives of childhood language brokering to illuminate that complexity. The purpose of bringing these studies together is methodological; in particular, it is to examine how the storied material generated by narrative approaches can contribute to understanding of the everyday practices of family life, practices that are often unacknowledged, hidden or assumed. One study adopted a narrative form of interviewing while the second combined narrative photo-elicitation techniques with narrative accounts. Methodologically, the two studies illustrate that no one method produces ‘objective’, comprehensive knowledge of family practices. Together, however, they produce new insights into family practices around fatherhood, migration and language brokering and how participants ‘do’, ‘display’ and commemorate family. The paper argues that narrative approaches, sometimes alongside visual methods, can assist holistic analyses of family practices from sociocultural, intergenerational and life course perspectives.     

Correlated defaults,temporal correlation,expert information and predictability of default rates

ABSTRACT

Dependence among defaults both across assets and over time is an important characteristic of financial risk. A Bayesian approach to default rate estimation is proposed and illustrated using prior distributions assessed from an experienced industry expert. Two extensions of the binomial model are proposed. The first allows correlated defaults yet remains consistent with Basel II’s asymptotic single-factor model. The second adds temporal correlation in default rates through autocorrelation in the systemic factor. Implications for the predictability of default rates are considered. The single-factor model generates more forecast uncertainty than does the parameter uncertainty. A robustness exercise illustrates that the correlation indicated by the data is much smaller than that specified in the Basel II regulations.     

Experimental Evidence Against the Paradigm of Mortality Risk Aversion

This article deals with the question of how societal impacts of fatal accidents can be integrated into the management of natural or man‐made hazards. Today, many governmental agencies give additional weight to the number of potential fatalities in their risk assessments to reflect society's aversion to large accidents. Although mortality risk aversion has been proposed in numerous risk management guidelines, there has been no evidence that lay people want public decisionmakers to overweight infrequent accidents of large societal consequences against more frequent ones of smaller societal consequences. Furthermore, it is not known whether public decisionmakers actually do such overweighting when they decide upon the mitigation of natural or technical hazards. In this article, we report on two experimental tasks that required participants to evaluate negative prospects involving 1–100 potential fatalities. Our results show that neither lay people nor hazard experts exhibit risk‐averse behavior in decisions on mortality risks.     

Eliciting Information from Experts on the Likelihood of Rapid Climate Change

The threat of so‐called rapid or abrupt climate change has generated considerable public interest because of its potentially significant impacts. The collapse of the North Atlantic Thermohaline Circulation or the West Antarctic Ice Sheet, for example, would have potentially catastrophic effects on temperatures and sea level, respectively. But how likely are such extreme climatic changes? Is it possible actually to estimate likelihoods? This article reviews the societal demand for the likelihoods of rapid or abrupt climate change, and different methods for estimating likelihoods: past experience, model simulation, or through the elicitation of expert judgments. The article describes a survey to estimate the likelihoods of two characterizations of rapid climate change, and explores the issues associated with such surveys and the value of information produced. The surveys were based on key scientists chosen for their expertise in the climate science of abrupt climate change. Most survey respondents ascribed low likelihoods to rapid climate change, due either to the collapse of the Thermohaline Circulation or increased positive feedbacks. In each case one assessment was an order of magnitude higher than the others. We explore a high rate of refusal to participate in this expert survey: many scientists prefer to rely on output from future climate model simulations.     

The Self-Fulfilling Property of Trust: An Experimental Study

A person is said to be ‘trust responsive’ if she fulfils trust because she believes the truster trusts her. The experiment we report was designed to test for trust responsiveness and its robustness across payoff structures, and to discriminate it from other possible factors making for trustworthiness, including perceived kindness, perceived need and inequality aversion. We elicit the truster’s confidence that the trustee will fulfil, and the trustee’s belief about the truster’s confidence after the trustee receives evidence relevant to this. We find evidence of strong trust responsiveness. We also find that perceptions of kindness and of need increase trust responsiveness, and that they do so only in conjunction with trust responsiveness.      

Updating Subjective Risks in the Presence of Conflicting Information: An Application to Climate Change

Willingness to support public programs for risk management often depends on individual subjective risk perceptions in the face of uncertain science. As part of a larger study concerning climate change, we explore individual updated subjective risks as a function of individual priors, the nature of external information, and individual attributes. We examine several rival hypotheses about how subjective risks change in the face of new information (Bayesian updating, alarmist learning, and ambiguity aversion). The source and nature of external information, as well as its collective ambiguity, can have varying effects across the population, in terms of both expectations and uncertainty.JEL Classification  D8, Q51, Q54