Weight of Evidence: A Review of Concept and Methods

"Weight of evidence" (WOE) is a common term in the published scientific and policy-making literature, most often seen in the context of risk assessment (RA). Its definition, however, is unclear. A systematic review of the scientific literature was undertaken to characterize the concept. For the years 1994 through 2004, PubMed was searched for publications in which "weight of evidence" appeared in the abstract and/or title. Of the 276 papers that met these criteria, 92 were selected for review: 71 papers published in 2003 and 2004 (WOE appeared in abstract/title) and 21 from 1994 through 2002 (WOE appeared in title). WOE has three characteristic uses in this literature: (1) metaphorical, where WOE refers to a collection of studies or to an unspecified methodological approach; (2) methodological, where WOE points to established interpretative methodologies (e.g., systematic narrative review, meta-analysis, causal criteria, and/or quality criteria for toxicological studies) or where WOE means that "all" rather than some subset of the evidence is examined, or rarely, where WOE points to methods using quantitative weights for evidence; and (3) theoretical, where WOE serves as a label for a conceptual framework. Several problems are identified: the frequent lack of definition of the term "weight of evidence," multiple uses of the term and a lack of consensus about its meaning, and the many different kinds of weights, both qualitative and quantitative, which can be used in RA. A practical recommendation emerges: the WOE concept and its associated methods should be fully described when used. A research agenda should examine the advantages of quantitative versus qualitative weighting schemes, how best to improve existing methods, and how best to combine those methods (e.g., epidemiology's causal criteria with toxicology's quality criteria).     

Conventional Wisdom on Risk Communication and Evidence from a Field Experiment

A recent comprehensive review of the literature identified a number of facts and principles governing risk communication. This paper evaluates several of these propositions using recent evidence from a field experiment in communicating the risks from radon in homes. At this point in the research, data relates primarily to the response of risk perceptions to different information treatments and different personal characteristics. The effect of various causal factors is sensitive to the particular test of risk perception applied. No information treatment was clearly superior for all tasks. An important implication of these findings is that risk communicators must determine what specific task or tasks the information program should enable people to do.     

Quantitative Risk Assessments as Evidence in Civil Litigation

Those who prepare quantitative risk assessments do not always appreciate that those assessments might be used as evidence in civil litigation. This paper suggests that litigation attorneys, judges, and juries be regarded as audiences to whom the information in the risk assessment must be communicated. The way that a risk assessment is prepared can affect significantly whether litigation is brought at all, the resolution of evidentiary motions involving the risk assessment, as well as the ultimate outcome of the litigation. This paper discusses certain procedural and evidentiary aspects of the civil litigation process in the hope that a better understanding of that process might lead to the preparation of risk assessments that are more adequately understood by juries, judges, and litigants.     

Improving Weight of Evidence Approaches to Chemical Evaluations

Federal and other regulatory agencies often use or claim to use a weight of evidence (WoE) approach in chemical evaluation. Their approaches to the use of WoE, however, differ significantly, rely heavily on subjective professional judgment, and merit improvement. We review uses of WoE approaches in key articles in the peer‐reviewed scientific literature, and find significant variations. We find that a hypothesis‐based WoE approach, developed by Lorenz Rhomberg et al., can provide a stronger scientific basis for chemical assessment while improving transparency and preserving the appropriate scope of professional judgment. Their approach, while still evolving, relies on the explicit specification of the hypothesized basis for using the information at hand to infer the ability of an agent to cause human health impacts or, more broadly, affect other endpoints of concern. We describe and endorse such a hypothesis‐based WoE approach to chemical evaluation.     

Use of Multicriteria Decision Analysis to Support Weight of Evidence Evaluation

Weight of evidence (WOE) methods are key components of ecological and human health risk assessments. Most WOE applications rely on the qualitative integration of diverse lines of evidence (LOE) representing impact on ecological receptors and humans. Recent calls for transparency in assessments and justifiability of management decisions are pushing the community to consider quantitative methods for integrated risk assessment and management. This article compares and contrasts the type of information required for application of individual WOE techniques and the outcomes that they provide in ecological risk assessment and proposes a multicriteria decision analysis (MCDA) framework for integrating individual LOE in support of management decisions. The use of quantitative WOE techniques is illustrated for a hypothetical but realistic case study of selecting remedial alternatives at a contaminated aquatic site. Use of formal MCDA does not necessarily eliminate biases and judgment calls necessary for selecting remedial alternatives, but allows for transparent evaluation and fusion of individual LOE. It also provides justifiable methods for selecting remedial alternatives consistent with stakeholder and decision‐maker values.     

Summarizing Risk Using Risk Measures and Risk Indices

Our society is fascinated with risk in many different areas and disciplines. One of the main ways to describe and communicate the level of risk is through risk indices, which summarize risk using numbers or categories such as words, letters, or colors. These indices are used to communicate risks to the public, understand how risk is changing over time, compare among different risks, and support decision making. Given the different methods to construct risk indices, including flawed methods such as risk matrices, this article develops specific steps that analysts can follow to create a risk index. This article emphasizes the importance of describing risk with a probability distribution, developing a numerical risk measure that summarizes the probability distribution, and finally translating the risk measure to an index. Measuring the risk is the most difficult part and requires the analyst to summarize a probability distribution into one or possibly a few numbers. The risk measure can then be transformed to a numerical or categorical index. I apply the method outlined in this article to construct a risk index that compares the risk of fatalities in aviation and highway transportation.     

Understanding Household Preferences for Hurricane Risk Mitigation Information: Evidence from Survey Responses

Risk information is critical to adopting mitigation measures, and seeking risk information is influenced by a variety of factors. An essential component of the recently adopted My Safe Florida Home (MSFH) program by the State of Florida is to provide homeowners with pertinent risk information to facilitate hurricane risk mitigation activities. We develop an analytical framework to understand household preferences for hurricane risk mitigation information through allowing an intensive home inspection. An empirical analysis is used to identify major drivers of household preferences to receive personalized information regarding recommended hurricane risk mitigation measures. A variety of empirical specifications show that households with home insurance, prior experience with damages, and with a higher sense of vulnerability to be affected by hurricanes are more likely to allow inspection to seek information. However, households with more members living in the home and households who live in manufactured/mobile homes are less likely to allow inspection. While findings imply MSFH program's ability to link incentives offered by private and public agencies in promoting mitigation, households that face a disproportionately higher level of risk can get priority to make the program more effective.     

Presenting Uncertainty in Health Risk Assessment: Initial Studies of Its Effects on Risk Perception and Trust

Some analysts suggest that discussing uncertainties in health risk assessments might reduce citizens'perceptions of risk and increase their respect for the risk-assessing agency. We tested this assumption with simulated news stories varying simple displays of uncertainty (e.g., a range of risk estimates, with and without graphics). Subjects from Eugene, Oregon, read one story each, and then answered a questionnaire. Three studies tested between 180 and 272 subjects each. Two focus groups obtained more detailed responses to these stories. The results suggested that (1) people are unfamiliar with uncertainty in risk assessments and in science; (2) people may recognize uncertainty when it is presented simply; (3) graphics may help people recognize uncertainty; (4) reactions to the environmental problems in the stories seemed affected less by presentation of uncertainty than by general risk attitudes and perceptions; (5) agency discussion of uncertainty in risk estimates may signal agency honesty and agency incompetence for some people; and (6) people seem to see lower risk estimates (10^-6, as opposed to 10^-3) as less credible. These findings, if confirmed, would have important implications for risk communication.     

Informing and Educating the Public About Risk

The objective of informing and educating the public about risk issues seems easy to attain in principle, but, in practice, may be difficult to accomplish. This paper attempts to illustrate why this is so. To be effective, risk communicators must recognize and overcome a number of obstacles that have their roots in the limitations of scientific risk assessment and the idiosyncrasies of the human mind. Doing an adequate job of communicating means finding comprehensible ways of presenting complex technical material that is clouded by uncertainty and inherently difficult to understand. The problems may not be insurmountable, however, if designers of risk information programs are sensitive to the difficulties.     

Evaluating Weight of Evidence in the Mystery of Balkan Endemic Nephropathy

Balkan endemic nephropathy (BEN) is a chronic, progressive wasting disease of the kidneys, endemic in certain rural regions of the Balkan nations Croatia, Serbia, Bulgaria, and Romania. It is irreversible and ultimately fatal. Though this disease was first described in the 1950s, its causes have been a mystery and a source of much academic and clinical contention. Possible etiologic agents that have been explored include exposure to metals and metalloids, viruses and bacteria, and the dietary toxins aristolochic acid (AA) and ochratoxin A (OTA). AA is a toxin produced by weeds of the genus Aristolochia, common in Balkan wheat fields. Aristolochia seeds may intermingle with harvested grains and thus inadvertently enter human diets. OTA is a mycotoxin (fungal toxin) common in many foods, including cereal grains. In this study, we analyzed the weight of evidence for each of the suspected causes of BEN using the Bradford Hill criteria (BHC): nine conditions that determine weight of evidence for a causal relationship between an agent and a disease. Each agent postulated to cause BEN was evaluated using the nine criteria, and for each criterion was given a rating based on the strength of the association between exposure to the substance and BEN. From the overall available scientific evidence for each of these suspected risk factors, AA is the agent with the greatest weight of evidence in causing BEN. We describe other methods for testing causality from epidemiological studies, which support this conclusion of AA causing BEN.     

Evidence maps: communicating risk assessments in societal controversies: the case of engineered nanoparticles

The transparent and fair characterization of scientific evidence for reporting the results of a hazard assessment is a demanding task. In this article, we present an approach for characterizing evidence--the evidence map approach. The theoretical starting point is to view evidence characterization as a form of argumentation. Thus, evidence maps are designed to depict the evidence base, the pro and con arguments, and the remaining uncertainties, which together lead experts to their conclusions when summarizing and evaluating the scientific evidence about a potential hazard. To illustrate its use, the evidence maps approach is applied to characterizing the health-relevant effects of engineered nanoparticles. Empirical data from an online survey suggests that the use of evidence maps improves the reporting of hazard assessments. Nonexperts prefer to receive the information included in an evidence map in order to come to an informed judgment. Furthermore, the benefits and limitations of evidence maps are discussed in the light of recent literature on risk communication. Finally, the article underlines the need for further research in order to increase quality of evidence reporting.     

Risk Perception and Communication Unplugged: Twenty Years of Process1

Over the past twenty years, risk communication researchers and practitioners have learned some lessons, often at considerable personal price. For the most part, the mistakes that they have made have been natural, even intelligent ones. As a result, the same pitfalls may tempt newcomers to the field. This essay offers a personal (even confessional) history of the field over this period. It identifies a series of developmental stages. Progress through the stages involves consolidating the skills needed to execute it and learning its limitations. Knowing about their existence might speed the learning process and alert one to how much there still is to learn.     

Stability and Inoculation of Risk Comparisons'' Effects Under Conflict: Replicating and Extending the "Asbestos Jury" Study by Slovic et al.

Promotion or criticism of risk comparisons in risk communication has far exceeded empirical tests of their effects. Slovic et al. (1990) experimented with a hypothetical jury trial in which an asbestos-installing firm was accused of subjecting school occupants to unreasonable risk. A risk comparison sharply reduced subjects' estimates of risk and judgments that the firm was guilty, but a critique of the risk comparison had risk estimates and guilt judgments rebounding to the original (without risk comparison) level. Slovic et al. concluded that risk comparisons' effects were highly unstable, at least in conflict-ridden situations such as a jury trial. The present study replicates and extends this important study, using the same stimuli and questions. The respective effects of the risk comparison and the critique recurred, although much less sharply than in Slovic et al. Moreover, judgments of guilt, risk, and other aspects of the case seemed shaped more by demographics and beliefs about risk generically (e.g., about the likelihood of cancer after exposure to a carcinogen) than by either risk comparison or critique. A variant design, in which the defense's expert witness dismissed potential criticisms of the risk comparison, appeared to "inoculate" people against shifting their views after seeing the critique. Overall, these results show that risk comparisons might change some beliefs about risks in conflict and that "inoculation" can reduce vulnerability to criticism. However, the results also show strong limits on effects of both comparisons and their critiques: they shifted only a minority of judgments and had small effects relative to people's social locations and prior risk beliefs.     

Agency Communication, Community Outrage, and Perception of Risk: Three Simulation Experiments

Three experimental studies were conducted employing hypothetical news stories to compare the effects on reader risk perceptions of two situations: when agency communication behavior was reported to be responsive to citizens' risk concerns, vs. when the agency was reported to be unresponsive. In the first two experiments, news stories of public meetings filled with distrust and controversy led to ratings indicating greater perceived risk than news stories reporting no distrust or controversy, even though the risk information was held constant. This effect appeared clearly when the differences in meeting tone were extreme and subjects made their ratings from their recall of the stories, but it was much weaker when the differences were moderate and subjects were allowed to go back over the news stories to help separate risk information from conflict information. In the third experiment, news stories about a spill cleanup systematically varied the seriousness of the spill, the amount of technical information provided in the story, and the agency behavior and resulting community outrage. The outrage manipulation significantly affected affective and cognitive components of perceived risk, but not hypothetical behavioral intentions. Seriousness and technical detail had very little effect on perceived risk.     

Setting Risk Priorities: A Formal Model

This article presents a model designed to capture the major aspects of setting priorities among risks, a common task in government and industry. The model has both design features, under the control of the rankers (e.g., how success is evaluated), and context features, properties of the situations that they are trying to understand (e.g., how quickly uncertainty can be reduced). The model is demonstrated in terms of two extreme ranking strategies. The first, sequential risk ranking , devotes all its resources, in a given period, to learning more about a single risk, and its place in the overall ranking. This strategy characterizes the process for a society (or organization or individual) that throws itself completely into dealing with one risk after another. The other extreme strategy, simultaneous risk ranking , spreads available resources equally across all risks. It characterizes the most methodical of ranking exercises. Given ample ranking resources, simultaneous risk ranking will eventually provide an accurate set of priorities, whereas sequential ranking might never get to some risks. Resource constraints, however, may prevent simultaneous rankers from examining any risk very thoroughly. The model is intended to clarify the nature of ranking tasks, predict the efficacy of alternative strategies, and improve their design.     

Risk Perception and Risk Talk: The Case of the Fukushima Daiichi Nuclear Radiation Risk

Individuals’ perceptions and their interpersonal communication about a risk event, or risk talk, can play a significant role in the formation of societal responses to the risk event. As they formulate their risk opinions and speak to others, risk information can circulate through their social networks and contribute to the construction of their risk information environment. In the present study, Japanese citizens’ risk perception and risk talk were examined in the context of the Fukushima Daiichi nuclear radiation risk. We hypothesized and found that the risk information environment and risk literacy (i.e., competencies to understand and use risk information) interact to influence their risk perception and risk talk. In particular, risk literacy tends to stabilize people's risk perceptions and their risk communications. Nevertheless, there were some subtle differences between risk perception and communication, suggesting the importance of further examination of interpersonal risk communication and its role in the societal responses to risk events.     

挖掘客户潜在需求确定指标重要程度的证据组合方法

在现实中,属性(例如产品的各种指标)的权重常常与特定客户的需求有关,因此这些客户的偏好信息对权重的确定非常重要.这种信息不一定就直接体现在对属性的重视程度上,而可能表现在对一组具体产品的偏好关系上.在对产品的指标评价时还会存在一些无知性.针对这些情况,本文给出一个基于证据理论和目标规划、挖掘客户潜在需求的指标权重估计方法,并结合一个计算示例,给出具体的计算过程.     

High Risk or Low: How Location on a "Risk Ladder" Affects Perceived Risk

Efforts to explain risk magnitude often rely on a "risk ladder" in which exposure levels and associated risk estimates are arrayed with low levels at the bottom and high ones at the top. Two experiments were conducted to test the hypothesis that perceived threat and intended mitigation vary with the location of the subject's assigned level on the risk ladder. Subjects were New Jersey homeowners, asked to assume a particular level of radon or asbestos contamination in their homes, to read a brochure explaining the risk, and then to complete a questionnaire. Both studies found that the difference between an assigned level one-quarter of the way up the ladder and the same level three-quarters of the way up the ladder significantly affected threat perception; the effect on mitigation intentions was significant in only one of the studies. Variations in assigned risk also affected threat perception and mitigation intentions. Variations in test magnitude (e.g., 15 fibers per liter vs. 450 fibers per cubic foot, roughly equivalent risks) had no effect, nor did the distinction between radon and asbestos affect the dependent variables. These findings suggest that communicators can design risk ladders to emphasize particular risk characteristics.     

Heuristic-Systematic Information Processing and Risk Judgment

The heuristic-systematic information processing model (HSM) holds that individuals will use one or both of these modes of information processing when attempting to evaluate information in order to arrive at a judgment. Systematic processing is defined by effortful scrutiny and comparison of information, whereas heuristic processing is defined by the use of cues to arrive more easily at a judgment. Antecedents to the two processing modes include information sufficiency, motivation, and self-efficacy. Structural equation modeling is used to examine competing configuration of this model and to evaluate the model as appropriate for predicting risk judgment. The model also is evaluated across three groups that vary with respect to their level of concern. These analyses are executed within a case study involving an epidemiological investigation of a suspected cancer cluster. The analysis confirms the HSM's theoretically proposed structure and shows it to be a useful vehicle for evaluating risk judgment. In the overall analysis, antecedent variables generally function as specified by theory. Systematic processing is predicted by greater motivation. Heuristic processing is predicted by information sufficiency. Self-efficacy is a significant predictor of both processing modes. And heuristic processing is shown to be associated with judgment of less risk. However, when the analysis is contrasted across three groups (those concerned about cancer, not concerned and uncertain) it is shown that the model is significantly more robust for the uncertain group. This finding may have implications for the use of the HSM in risk research specifically, and in field research generally.