Differences in Expert and Lay Judgments of Risk: Myth or Reality?

This article evaluates the nine empirical studies that have been conducted on expert versus lay judgments of risk. Contrary to received wisdom, this study finds that there is little empirical evidence for the propositions (1) that experts judge risk differently from members of the public or (2) that experts are more veridical in their risk assessments. Methodological weaknesses in the early research are documented, and it is shown that the results of more recent studies are confounded by social and demographic factors that have been found to correlate with judgments of risk. Using a task-analysis taxonomy, a template is provided for the documentation of future studies of expert-lay differences/similarities that will facilitate analytic comparison.     

Risk Perception in the U.K. Oil and Gas Production Industry: Are Expert Loss‐Prevention Managers' Perceptions Different From Those of Members of the Public?

This article investigates potential differences in risk perception between experts (loss-prevention managers in the U.K. oil and gas production industry) and nonexperts (managers and students). Extant research on expert versus nonexpert perceptions of risk is reviewed, followed by the present study concerning risk perceptions of seven pen-picture scenarios involving the occurrence of hazardous events in the U.K. oil and gas production industry. In contrast to many of the earlier studies of expert versus nonexpert perceptions of risk, the present analysis concludes that experts did not judge the overall riskiness of the portrayed hazardous events as less risky than the nonexperts. Nevertheless, the experts believe more strongly than our nonexperts that the risks portrayed in the scenarios pose little threat to future generations, are more precisely known, and are relatively controllable. Use of multiple regression analysis to help uncover the basis of overall riskiness assessments for expert and lay respondents was inconclusive, however. Finally, little evidence was found that nonexperts were any more heterogeneous in their risk perceptions than experts. It may be that the nature of the risks assessed in the present study may account for the general lack of clear expert versus nonexpert differences in overall perceptions of the riskiness of hazardous events in the North Sea. Earlier findings of strong expert versus nonexpert differences in risk perception assessed hazards of major public concern. It is inferred that using such extreme hazards may have resulted in an exaggerated view of differences in expert versus public (nonexpert) perception of risk.     

Stability and Inoculation of Risk Comparisons'' Effects Under Conflict: Replicating and Extending the "Asbestos Jury" Study by Slovic et al.

Promotion or criticism of risk comparisons in risk communication has far exceeded empirical tests of their effects. Slovic et al. (1990) experimented with a hypothetical jury trial in which an asbestos-installing firm was accused of subjecting school occupants to unreasonable risk. A risk comparison sharply reduced subjects' estimates of risk and judgments that the firm was guilty, but a critique of the risk comparison had risk estimates and guilt judgments rebounding to the original (without risk comparison) level. Slovic et al. concluded that risk comparisons' effects were highly unstable, at least in conflict-ridden situations such as a jury trial. The present study replicates and extends this important study, using the same stimuli and questions. The respective effects of the risk comparison and the critique recurred, although much less sharply than in Slovic et al. Moreover, judgments of guilt, risk, and other aspects of the case seemed shaped more by demographics and beliefs about risk generically (e.g., about the likelihood of cancer after exposure to a carcinogen) than by either risk comparison or critique. A variant design, in which the defense's expert witness dismissed potential criticisms of the risk comparison, appeared to "inoculate" people against shifting their views after seeing the critique. Overall, these results show that risk comparisons might change some beliefs about risks in conflict and that "inoculation" can reduce vulnerability to criticism. However, the results also show strong limits on effects of both comparisons and their critiques: they shifted only a minority of judgments and had small effects relative to people's social locations and prior risk beliefs.     

Perception of Ecological Risk to Water Environments

This paper examines lay and expert perceptions of the ecological risks associated with a range of human activities that could adversely affect water resource environments. It employs the psychometric paradigm pioneered in characterizing perceptions of human health risks, which involves surveys to obtain judgments from subjects about risk items in terms of several important characteristics of the risks. The paper builds on a previous study that introduced ecological risk perception. This second study employs a larger, more diverse sample, a more focused topic area, and comparisons between lay and expert judgments. The results confirm that a small set of underlying factors explain a great deal of variability in lay judgments about ecological risks. These have been termed Ecological Impact, Human Benefits, Controllability , and Knowledge. The results are useful in explaining subjects' judgments of the general riskiness of, and need for regulation of, various risk items. The results also indicate several differences and areas of agreement among the lay and expert samples that point to potential key issues in future ecological risk management efforts for water resources.     

Intuitive Toxicology: Expert and Lay Judgments of Chemical Risks

Human beings have always been intuitive toxicologists, relying on their senses of sight, taste, and smell to detect harmful or unsafe food, water, and air. As we have come to recognize that our senses are not adequate to assess the dangers inherent in exposure to a chemical substance, we have created the sciences of toxicology and risk assessment to perform this function. Yet despite this great effort to overcome the limitations of intuitive toxicology, it has become evident that even our best scientific methods still depend heavily on extrapolations and judgments in order to infer human health risks from animal data. Many observers have acknowledged the inherent subjectivity in the assessment of chemical risks and have indicated a need to examine the subjective or intuitive elements of expert and lay risk judgments. We have begun such an examination by surveying members of the Society of Toxicology and the lay public about basic toxicological concepts, assumptions, and interpretations. Our results demonstrate large differences between toxicologists and laypeople, as well as differences between toxicologists working in industry, academia, and government. In addition, we find that toxicologists are sharply divided in their opinions about the ability to predict a chemical's effect on human health on the basis of animal studies. We argue that these results place the problems of risk communication in a new light. Although the survey identifies misconceptions that experts should clarify for the public, it also suggests that controversies over chemical risks may be fueled as much by limitations of the science of risk assessment and disagreements among experts as by public misconceptions.     

Modeling Risk‐Related Knowledge in Tunneling Projects

Knowledge on failure events and their associated factors, gained from past construction projects, is regarded as potentially extremely useful in risk management. However, a number of circumstances are constraining its wider use. Such knowledge is usually scarce, seldom documented, and even unavailable when it is required. Further, there exists a lack of proven methods to integrate and analyze it in a cost‐effective way. This article addresses possible options to overcome these difficulties. Focusing on limited but critical potential failure events, the article demonstrates how knowledge on a number of important potential failure events in tunnel works can be integrated. The problem of unavailable or incomplete information was addressed by gathering judgments from a group of experts. The elicited expert knowledge consisted of failure scenarios and associated probabilistic information. This information was integrated using Bayesian belief‐networks‐based models that were first customized in order to deal with the expected divergence in judgments caused by epistemic uncertainty of risks. The work described in the article shows that the developed models that integrate risk‐related knowledge provide guidance as to the use of specific remedial measures.     

Technical and Democratic Values in Risk Analysis1

So, we can argue that the lay public are not fools in their attitudes about risk. That nonexperts should show more concern over hazardous waste facilities in their neighborhoods than radon levels in their homes is not a sign of irrationality (because aggregate and individual risks are greater from the radon), but simply a sign that nonexperts are working from a different set of criteria. These criteria are incorporated in what I call the democratic model. The democratic model evaluates risk based on its social and political consequences, such as possible disruption in the social fabric or a loss of communality. Lay criteria for assessing the impact of risk decisions are not explicit, like the those of the risk analyst, but are embedded in cultural values. Similarly, lay evaluations of risk incorporate substantive and procedural democratic values, such as the acceptability of processes for making decisions, the ethics of the distribution of risk, and the capacity to control a source of risk in the community's interests. Finally, the democratic model relates judgments about risks to the competence (Can we trust them?) and the legitimacy (Should we trust them?) of the social institutions that impose and control those risks. The public's judgments about risk are not inferior, but different, and arguably richer than those of the experts.     

Determinants of Priority for Risk Reduction: The Role of Worry

One hundred twenty-two members (experts) of the Society for Risk Analysis completed a mailed questionnaire and 150 nonexperts completed a similar questionnaire on the World Wide Web. Questions asked included those about priorities on personal and government action for risk reduction, badness of the risk, number of people affected, worry, and probabilities for self and others. Individual differences in mean desire for action were largely explained in terms of worry. Worry, in turn, was largely affected by probability judgments, which were lower for experts than for nonexperts. Differences across risks in the desire for action, within each subject, were also determined largely by worry and probability. Belief in expert knowledge about the risk increased worry and the priority for risk reduction. A second study involving 91 nonexperts (42 interviewed and 49 on the Web) replicated the main findings for nonexperts from the first study. Interviews also probed the determinants of worry, attitudes toward government versus personal control, and protective behaviors.     

Implicit Attitudes Toward Nuclear Power and Mobile Phone Base Stations: Support for the Affect Heuristic

The implicit association test (IAT) measures automatic associations. In the present research, the IAT was adapted to measure implicit attitudes toward technological hazards. In Study 1, implicit and explicit attitudes toward nuclear power were examined. Implicit measures (i.e., the IAT) revealed negative attitudes toward nuclear power that were not detected by explicit measures (i.e., a questionnaire). In Study 2, implicit attitudes toward EMF (electro-magnetic field) hazards were examined. Results showed that cell phone base stations and power lines are judged to be similarly risky and, further, that base stations are more closely related to risk concepts than home appliances are. No differences between experts and lay people were observed. Results of the present studies are in line with the affect heuristic proposed by Slovic and colleagues. Affect seems to be an important factor in risk perception.     

Exploring the "Psychometric Paradigm": Comparisons Between Aggregate and Individual Analyses

The "psychometric paradigm" developed by Slovic, Fischhoff, and Lichtenstein was a landmark in research about public attitudes toward risks. One problem with this work, however, was that (at least initially) it did not attempt to distinguish between individuals or groups of people, except "experts" vs. "lay people." This paradigm produced a "cognitive map" of hazards, and the assumption seemed to be that the characteristics identified were inherent attributes of risk. This paper examines the validity of this assumption. A questionnaire survey similar to those designed by Slovic et al. was conducted, but the data were analyzed at both the aggregate level, using mean scores, and at the level of individuals ( N = 131 Norwich residents). The results reported here demonstrate that (1) individuals vary in their perception of the same risk issue; (2) individuals vary in their rating of the same risk characteristics on the same risk issue; and (3) some of the strong intercorrelations observed between risk characteristics at the aggregate level are not supported when the same data are analysed at the level of individuals. Despite these findings, the relationship between risk characteristics and risk perceptions inferred by the psychometric paradigm did hold true at the level of individuals, for most—but not all—of the characteristics. In particular, the relationship between "lack of knowledge to those exposed" and risk perceptions appears to be a complex one, a finding which has important implications for risk communication strategies.     

Combining Experts’ Judgments: Comparison of Algorithmic Methods Using Synthetic Data

Expert judgment (or expert elicitation) is a formal process for eliciting judgments from subject‐matter experts about the value of a decision‐relevant quantity. Judgments in the form of subjective probability distributions are obtained from several experts, raising the question how best to combine information from multiple experts. A number of algorithmic approaches have been proposed, of which the most commonly employed is the equal‐weight combination (the average of the experts’ distributions). We evaluate the properties of five combination methods (equal‐weight, best‐expert, performance, frequentist, and copula) using simulated expert‐judgment data for which we know the process generating the experts’ distributions. We examine cases in which two well‐calibrated experts are of equal or unequal quality and their judgments are independent, positively or negatively dependent. In this setting, the copula, frequentist, and best‐expert approaches perform better and the equal‐weight combination method performs worse than the alternative approaches.     

Risk assessment of outsourcing e-procurement services: integrating SWOT analysis with a modified ANP-based fuzzy inference system

With electronic procurement solutions becoming increasingly sophisticated, many firms opt to source these services from third-party providers, effectively transferring (outsourcing) significant responsibility to these services companies. This action, however, entails certain risks, which are oftentimes difficult to assess. To guide managerial practice and to advance academic inquiry in this domain, we identify e-procurement risk factors through a strengths, weaknesses, opportunities, threats analysis, grounded in transaction cost economics (TCE), and propose a risk assessment framework based on the opinions of a group of experts. The approach taken is that of a modified analytic network process methodology, combined with a fuzzy inference system, which is versatile enough to accept the expert opinions in different input formats (such as linguistic variables and ranges). The proposed framework has the capability to aggregate expert judgements’ to estimate risk likelihoods, risk severities and risk factor indices, and derive overall risk magnitudes. The multi-method approach was motivated and is illustrated by a real-life case study of an Indian manufacturing company currently in the process of contract renewal with its existing e-procurement service provider.     

Determining Differences in Expert Judgment: Implications for Knowledge Acquisition and Validation*

In knowledge acquisition, it is often desirable to aggregate the judgments of multiple experts into a single system. In some cases this takes the form of averaging the judgments of those experts. In these situations it is desirable to determine if the experts have different views of the world before their individual judgments are aggregated. In validation, multiple experts often are employed to compare the performance of expert systems and other human actors. Often those judgments are then averaged to establish performance quality of the expert system. An important part of the comparison process should be determining if the experts have a similar view of the world. If the experts do not have similar views, their evaluations of performance may differ, resulting in a meaningless average performance measure. Alternatively, if all the validating experts do have similar views of the world then the validation process may result in paradigm myopia.     

The Effect of Risk Beliefs on Property Values: A Case Study of a Hazardous Waste Site1

Health risk beliefs of homeowners near a landfill site were assessed in a survey and compared to expert judgments of the health risks of living near the site. A bimodal distribution of health risk beliefs suggested sharp disagreement between the experts and at least some of the residents. Correlates of high risk beliefs included perception of odor from the site, exposure to media coverage of the problem, having children living at home, age (younger respondents more concerned), and gender (females more concerned). An aggregated neighborhood health risk belief predicted reductions in home prices even after controlling for home physical characteristics, such as size and other disamenities such as proximity to a freeway. In the 4100 homes near the site, the estimated depression in property values was estimated to total about $40.2 million before the site was closed and to be about $19.7 million after closure. Implications of these results for community conflict and for benefit-cost analysis of hazard site remediation are discussed.     

Eliciting Information from Experts on the Likelihood of Rapid Climate Change

The threat of so‐called rapid or abrupt climate change has generated considerable public interest because of its potentially significant impacts. The collapse of the North Atlantic Thermohaline Circulation or the West Antarctic Ice Sheet, for example, would have potentially catastrophic effects on temperatures and sea level, respectively. But how likely are such extreme climatic changes? Is it possible actually to estimate likelihoods? This article reviews the societal demand for the likelihoods of rapid or abrupt climate change, and different methods for estimating likelihoods: past experience, model simulation, or through the elicitation of expert judgments. The article describes a survey to estimate the likelihoods of two characterizations of rapid climate change, and explores the issues associated with such surveys and the value of information produced. The surveys were based on key scientists chosen for their expertise in the climate science of abrupt climate change. Most survey respondents ascribed low likelihoods to rapid climate change, due either to the collapse of the Thermohaline Circulation or increased positive feedbacks. In each case one assessment was an order of magnitude higher than the others. We explore a high rate of refusal to participate in this expert survey: many scientists prefer to rely on output from future climate model simulations.     

Rating Threat Mitigators: Faith in Experts,Governments, and Individuals Themselves to Create a Safer World

This research explores public judgments about the threat-reducing potential of experts, individual behavior, and government spending. The data are responses of a national sample of 1225 to mail surveys that include measures of several dimensions of public judgments about violent crime, automobile accidents, hazardous chemical waste, air pollution, water pollution, global warming, AIDS, heart disease, and cancer. Beliefs about who can best mitigate threats are specific to classes of threats. In general, there is little faith that experts can do much about violent crime and automobile accidents, moderate faith in their ability to address problems of global warming, and greater expectations for expert solutions to the remaining threats. People judge individual behavior as effective in reducing the threats of violent crime, AIDS, heart disease, and automobile accidents but less so for the remaining threats. Faith in more government spending is highest for AIDS and the other two health items, lowest for the trio of violent crime, automobile accidents, and global warming, and moderate for the remaining threats. For most threats, people are not distributed at the extremes in judging mitigators. Strong attitudinal and demographic cleavages are also lacking, although some interesting relationships occur. This relative lack of sharp cleavages and the generally moderate opinion indicate ample opportunity for public education and risk communication.     

Subjective machines: Probabilistic risk assessment based on deep learning of soft information

For several years machine learning methods have been proposed for risk classification. While machine learning methods have also been used for failure diagnosis and condition monitoring, to the best of our knowledge, these methods have not been used for probabilistic risk assessment. Probabilistic risk assessment is a subjective process. The problem of how well machine learning methods can emulate expert judgments is challenging. Expert judgments are based on mental shortcuts, heuristics, which are susceptible to biases. This paper presents a process for developing natural language-based probabilistic risk assessment models, applying deep learning algorithms to emulate experts’ quantified risk estimates. This allows the risk analyst to obtain an a priori risk assessment when there is limited information in the form of text and numeric data. Universal sentence embedding (USE) with gradient boosting regression (GBR) trees trained over limited structured data presented the most promising results. When we apply these models’ outputs to generate survival distributions for autonomous systems’ likelihood of loss with distance, we observe that for open water and ice shelf operating environments, the differences between the survival distributions generated by the machine learning algorithm and those generated by the experts are not statistically significant.     

Which Paradigm for Managing the Risk of Ionizing Radiation?

This article tackles the problem of controversies expressed by experts in the field of estimating and managing ionizing radiation risks. We analyze the paradigms that were conceived on this subject, in particular the studies carried out by the International Commission on Radiological Protection (ICRP), as well as the papers stating either that the effect of low doses is relatively weak or, on the contrary, relatively serious. Uncertainties, which taint the risk estimations, assume a particular importance because they are at the origin of the request for expert and value judgments and represent the critical point of the discussions on the ionizing radiation risks. Our study allows us to look further into the problem of the paradigm's formation, uncertainties, and expert and value judgments, and provides areas for consideration that may contribute to a better understanding of certain gridlocks in the decision-making process, as regards to environmental, health, and energy policies.     

Predicting the Validity of Expert Judgments in Assessing the Impact of Risk Mitigation Through Failure Prevention and Correction

Operational risk management of autonomous vehicles in extreme environments is heavily dependent on expert judgments and, in particular, judgments of the likelihood that a failure mitigation action, via correction and prevention, will annul the consequences of a specific fault. However, extant research has not examined the reliability of experts in estimating the probability of failure mitigation. For systems operations in extreme environments, the probability of failure mitigation is taken as a proxy of the probability of a fault not reoccurring. Using a priori expert judgments for an autonomous underwater vehicle mission in the Arctic and a posteriori mission field data, we subsequently developed a generalized linear model that enabled us to investigate this relationship. We found that the probability of failure mitigation alone cannot be used as a proxy for the probability of fault not reoccurring. We conclude that it is also essential to include the effort to implement the failure mitigation when estimating the probability of fault not reoccurring. The effort is the time taken by a person (measured in person-months) to execute the task required to implement the fault correction action. We show that once a modicum of operational data is obtained, it is possible to define a generalized linear logistic model to estimate the probability a fault not reoccurring. We discuss how our findings are important to all autonomous vehicle operations and how similar operations can benefit from revising expert judgments of risk mitigation to take account of the effort required to reduce key risks.     

What Do We Know About Making Risk Comparisons?

The risks of unfamiliar technologies are often evaluated by comparing them with the risks of more familiar ones. Such risk comparisons have been criticized for neglecting critical dimensions of risky decisions. In a guide written for the Chemical Manufacturers Association, Covello et al. (¹) have summarized these critiques and developed a taxonomy that characterizes possible risk comparisons in terms of their acceptability (or objectionableness). We asked four diverse groups of subjects to judge the acceptability of 14 statements produced by Covello et al. as examples of their categories. We found no correlation between the judgments of acceptability produced by our subjects and those predicted by Covello et al. .