Risk Characterization and the Weight of Evidence: Adapting Gatekeeping Concepts from the Courts

Risk characterization objectives include evaluating the weight of evidence underlying risk determinations, communicating that evaluation to nonexperts, guiding risk assessors to achieve consistency, and preserving deference for those reasonable expert judgments inherent in any risk determination. Similar objectives are shared by American courts that face the gatekeeping task of screening scientific evidence before it is presented to nonexpert factfinders, such as juries. This article surveys the judicial gatekeeping concepts of relevance, evidentiary reliability, legal sufficiency, presumptions, and standards of proof (particularly, preponderance of the evidence). It examines recent court decisions that have applied these concepts to the kinds of scientific information common in risk assessments, and suggests how to adapt these gatekeeping concepts for use in weight-of-evidence characterization. If we can develop and adopt a neutral framework for characterizing the weight of evidence underlying risk assessments, it might help clarify not only the current debate over risk characterization and risk management, but also the drafting of treaty provisions, such as those invoking the Precautionary Principle of international environmental law.     

Applications of probabilistic risk assessments: the selection of appropriate tools

Probabilistic risk assessment (PRA) is an important methodology for assessing the risks of complex technologies. This paper discusses the strengths and weaknesses of PRA. Its application is explored in three different settings: adversarial policy processes, regulatory/licensing procedures, and plant safety audits. It is concluded that PRA is a valuable tool for auditing safety precautions of existing or planned technologies, especially when it is carried out as an interactive process involving designers and plant personnel who are familiar with actual, everyday operations. PRA has not proven to be as well-suited in providing absolute risk estimates in public-policy debates concerning the acceptability of a technology, or for the licensing and regulatory procedures. The reasons for this are discussed.     

Quantitative Risk Assessments as Evidence in Civil Litigation

Those who prepare quantitative risk assessments do not always appreciate that those assessments might be used as evidence in civil litigation. This paper suggests that litigation attorneys, judges, and juries be regarded as audiences to whom the information in the risk assessment must be communicated. The way that a risk assessment is prepared can affect significantly whether litigation is brought at all, the resolution of evidentiary motions involving the risk assessment, as well as the ultimate outcome of the litigation. This paper discusses certain procedural and evidentiary aspects of the civil litigation process in the hope that a better understanding of that process might lead to the preparation of risk assessments that are more adequately understood by juries, judges, and litigants.     

The influence of mapped hazards on risk beliefs: a proximity-based modeling approach

Interview findings suggest perceived proximity to mapped hazards influences risk beliefs when people view environmental hazard maps. For dot maps, four attributes of mapped hazards influenced beliefs: hazard value, proximity, prevalence, and dot patterns. In order to quantify the collective influence of these attributes for viewers’ perceived or actual map locations, we present a model to estimate proximity‐based hazard or risk (PBH) and share study results that indicate how modeled PBH and map attributes influenced risk beliefs. The randomized survey study among 447 university students assessed risk beliefs for 24 dot maps that systematically varied by the four attributes. Maps depicted water test results for a fictitious hazardous substance in private residential wells and included a designated “you live here” location. Of the nine variables that assessed risk beliefs, the numerical susceptibility variable was most consistently and strongly related to map attributes and PBH. Hazard value, location in or out of a clustered dot pattern, and distance had the largest effects on susceptibility. Sometimes, hazard value interacted with other attributes, for example, distance had stronger effects on susceptibility for larger than smaller hazard values. For all combined maps, PBH explained about the same amount of variance in susceptibility as did attributes. Modeled PBH may have utility for studying the influence of proximity to mapped hazards on risk beliefs, protective behavior, and other dependent variables. Further work is needed to examine these influences for more realistic maps and representative study samples.     

Risk assessment of major hazards and its application in urban planning: a case study

With the rapid development of industry in China, the number of establishments that are proposed or under construction is increasing year by year, and many are industries that handle flammable, explosive, toxic, harmful, and dangerous substances. Accidents such as fire, explosion, and toxic diffusion inevitably happen. Accidents resulting from these major hazards in cities cause a large number of casualties and property losses. It is increasingly important to analyze the risk of major hazards in cities realistically and to suitably plan and utilize the surrounding land based on the risk analysis results, thereby reducing the hazards. A theoretical system for risk assessment of major hazards in cities is proposed in this article, and the major hazard risk for the entire city is analyzed quantitatively. Risks of various major accidents are considered together, superposition effect is analyzed, individual risk contours of the entire city are drawn out, and the level of risk in the city is assessed using "as low as reasonably practicable" guidelines. After the entire city's individual risk distribution is obtained, risk zones are divided according to corresponding individual risk value of HSE, and land-use planning suggestions are proposed. Finally, a city in China is used as an example to illustrate the risk assessment process of the city's major hazard and its application in urban land-use planning. The proposed method has a certain theoretical and practical significance in establishing and improving risk analysis of major hazard and urban land-use planning. On the one hand, major urban public risk is avoided; further, the land is utilized in the best possible way in order to obtain the maximum benefit from its use.     

Weight of Evidence: A Review of Concept and Methods

"Weight of evidence" (WOE) is a common term in the published scientific and policy-making literature, most often seen in the context of risk assessment (RA). Its definition, however, is unclear. A systematic review of the scientific literature was undertaken to characterize the concept. For the years 1994 through 2004, PubMed was searched for publications in which "weight of evidence" appeared in the abstract and/or title. Of the 276 papers that met these criteria, 92 were selected for review: 71 papers published in 2003 and 2004 (WOE appeared in abstract/title) and 21 from 1994 through 2002 (WOE appeared in title). WOE has three characteristic uses in this literature: (1) metaphorical, where WOE refers to a collection of studies or to an unspecified methodological approach; (2) methodological, where WOE points to established interpretative methodologies (e.g., systematic narrative review, meta-analysis, causal criteria, and/or quality criteria for toxicological studies) or where WOE means that "all" rather than some subset of the evidence is examined, or rarely, where WOE points to methods using quantitative weights for evidence; and (3) theoretical, where WOE serves as a label for a conceptual framework. Several problems are identified: the frequent lack of definition of the term "weight of evidence," multiple uses of the term and a lack of consensus about its meaning, and the many different kinds of weights, both qualitative and quantitative, which can be used in RA. A practical recommendation emerges: the WOE concept and its associated methods should be fully described when used. A research agenda should examine the advantages of quantitative versus qualitative weighting schemes, how best to improve existing methods, and how best to combine those methods (e.g., epidemiology's causal criteria with toxicology's quality criteria).     

Presenting Uncertainty in Health Risk Assessment: Initial Studies of Its Effects on Risk Perception and Trust

Some analysts suggest that discussing uncertainties in health risk assessments might reduce citizens'perceptions of risk and increase their respect for the risk-assessing agency. We tested this assumption with simulated news stories varying simple displays of uncertainty (e.g., a range of risk estimates, with and without graphics). Subjects from Eugene, Oregon, read one story each, and then answered a questionnaire. Three studies tested between 180 and 272 subjects each. Two focus groups obtained more detailed responses to these stories. The results suggested that (1) people are unfamiliar with uncertainty in risk assessments and in science; (2) people may recognize uncertainty when it is presented simply; (3) graphics may help people recognize uncertainty; (4) reactions to the environmental problems in the stories seemed affected less by presentation of uncertainty than by general risk attitudes and perceptions; (5) agency discussion of uncertainty in risk estimates may signal agency honesty and agency incompetence for some people; and (6) people seem to see lower risk estimates (10^-6, as opposed to 10^-3) as less credible. These findings, if confirmed, would have important implications for risk communication.     

The Media and Genetically Modified Foods: Evidence in Support of Social Amplification of Risk

Empirical examinations of the "social amplification of risk" framework are rare, partly because of the difficulties in predicting when conditions likely to result in amplification effects will occur. This means that it is difficult to examine changes in risk perception that are contemporaneous with increases and/or decreases in social or media discussion of the risks associated with a particular risk event. However, the collection of attitude data before, during, and after the increased reporting of the risks of genetically modified food in the United Kingdom (spring 1999) has demonstrated that people's risk perceptions do increase and decrease in line with what might be expected upon examination of the amplification and attenuation mechanisms integral to the framework. Perceptions of benefit, however, appeared to be permanently depressed by negative reporting about genetically modified food. Trust in regulatory institutions with responsibility for protecting the public was not affected. It was concluded that the social amplification of risk framework is a useful framework for beginning to explain the potential impact on risk perceptions of a risk event, particularly if that risk event is presented to the public as a new hazard occurring in a crisis context.     

Food Safety in the Domestic Environment: The Effect of Consumer Risk Information on Human Disease Risks

The improvement of food safety in the domestic environment requires a transdisciplinary approach, involving interaction between both the social and natural sciences. This approach is applied in a study on risks associated with Campylobacter on broiler meat. First, some web-based information interventions were designed and tested on participant motivation and intentions to cook more safely. Based on these self-reported measures, the intervention supported by the emotion "disgust" was selected as the most promising information intervention. Its effect on microbial cross-contamination was tested by recruiting a set of participants who prepared a salad with chicken breast fillet carrying a known amount of tracer bacteria. The amount of tracer that could be recovered from the salad revealed the transfer and survival of Campylobacter and was used as a measure of hygiene. This was introduced into an existing risk model on Campylobacter in the Netherlands to assess the effect of the information intervention both at the level of exposure and the level of human disease risk. We showed that the information intervention supported by the emotion "disgust" alone had no measurable effect on the health risk. However, when a behavioral cue was embedded within the instruction for the salad preparation, the risk decreased sharply. It is shown that a transdisciplinary approach, involving research on risk perception, microbiology, and risk assessment, is successful in evaluating the efficacy of an information intervention in terms of human health risks. The approach offers a novel tool for science-based risk management in the area of food safety.     

An emotion-based model of risk perception and stigma susceptibility: cognitive appraisals of emotion, affective reactivity, worldviews, and risk perceptions in the generation of technological stigma.

A study (N=198) was conducted to examine hypotheses derived from an emotion-based model of stigma responses to radiation sources. A model of stigma susceptibility is proposed in which affective reactions and cognitive worldviews activate predispositions to appraise and experience events in systematic ways that result in the generation of negative emotion, risk perceptions, and stigma responses. Results of structural equation modeling supported the hypotheses. Radiation sources that scored higher on a measure of stigma were included in the analyses (i.e., nuclear power plants, radioactive waste from nuclear power plants, radiation from nuclear weapons testing). Individual differences in negative reactivity and worldviews were associated with the strength of emotional appraisals that were associated, in turn, with negative emotion toward stigmatized radiation sources. As hypothesized, the model fit better with perceived risk as a function of negative emotion rather than vice versa. Finally, a measure of stigma was associated with negative emotion and, to a lesser extent, with risk perceptions. Risk communication about stigmatized objects may benefit from a more complete understanding of how affective and emotional reactions are constructed and the routes through which they affect responses and behaviors.     

Evaluating the Mental Models Approach to Developing a Risk Communication: A Scoping Review of the Evidence

Risk communication is fundamental in ensuring people are equipped with the knowledge needed to navigate varied risks. One generally well‐regarded framework for the development of such communications is the mental models approach to risk communication (MMARC). Developed during the 1990s, the MMARC has been applied to a range of health, technological, and environmental risks. However, as yet, we know of no attempt to collate and review articles that evaluated communications developed using the MMARC. This article took a first step at addressing this gap by conducting a scoping review that aimed to begin to explore the fidelity with which the approach has been applied, explore whether there appeared to be sufficient studies to warrant a future systematic review, and identify future research questions. Although the initial search found over 100 articles explicitly applying the MMARC, only 12 of these developed a risk‐related communication that was tested against a control (and thus included in the current review). All studies reported a positive effect of the MMARC versus control communication for at least some of the outcome measures (knowledge being the most prevalent). However, there was wide variation between studies including type of control, outcomes assessed, and only five studies reported adopting a randomized design. The review highlights both the need for greater fidelity in the way future studies operationalize the MMARC approach, and suggests that a full‐scale systematic review of the MMARC literature appears justified, especially given the possibility of a large gray literature in this area.     

Quantitative analyses of misclassification rates in the hazard classification of environmental chemicals: Evaluation of procedures for deriving predicted no-effect concentrations in the Chemical Substance Control Law in Japan

The quality of chemical management depends more or less on practical procedures used to assess chemicals. This study quantitatively assessed the efficacy of a derivation procedure for calculating no-effect concentrations for screening assessment of environmental hazards under the Chemical Substance Control Law in Japan. We first evaluated the derivation procedure by applying a series of test ecotoxicity datasets to the procedure and calculating the resulting misclassification rates of the hazardous class of chemicals. In this study, a chemical was deemed to have been misclassified if its classification differed from its classification based on the full dataset (chronic toxicity data for three trophic levels), which was defined as the correct assignment. We also calculated the effects of additional uncertainty factors to decrease the variance (i.e., to improve the consistency) of the misclassification rates among cases with different data availability in the derivation procedure. The results showed that the derivation procedure resulted in very high rates of misclassification when only particular sets of ecotoxicity data were available (e.g., only chronic toxicity data of algae were available). Our analyses also showed that the use of additional uncertainty factors improved the consistency of the misclassification rates within the derivation procedure. Our study presents a broadly applicable calculation framework for quantifying error rates in assessment procedures and serves as a case study for future development and reforms of chemical assessment processes and policies, while additional analyses using more extensive ecotoxicity data with various modes of actions are needed in the future.     

“The Dose Makes the Poison”: Informing Consumers About the Scientific Risk Assessment of Food Additives

Intensive risk assessment is required before the approval of food additives. During this process, based on the toxicological principle of “the dose makes the poison,? maximum usage doses are assessed. However, most consumers are not aware of these efforts to ensure the safety of food additives and are therefore sceptical, even though food additives bring certain benefits to consumers. This study investigated the effect of a short video, which explains the scientific risk assessment and regulation of food additives, on consumers’ perceptions and acceptance of food additives. The primary goal of this study was to inform consumers and enable them to construct their own risk‐benefit assessment and make informed decisions about food additives. The secondary goal was to investigate whether people have different perceptions of food additives of artificial (i.e., aspartame) or natural origin (i.e., steviolglycoside). To attain these research goals, an online experiment was conducted on 185 Swiss consumers. Participants were randomly assigned to either the experimental group, which was shown a video about the scientific risk assessment of food additives, or the control group, which was shown a video about a topic irrelevant to the study. After watching the video, the respondents knew significantly more, expressed more positive thoughts and feelings, had less risk perception, and more acceptance than prior to watching the video. Thus, it appears that informing consumers about complex food safety topics, such as the scientific risk assessment of food additives, is possible, and using a carefully developed information video is a successful strategy for informing consumers.     

Flood Hazard and Flood Risk Assessment Using a Time Series of Satellite Images: A Case Study in Namibia

In this article, the use of time series of satellite imagery to flood hazard mapping and flood risk assessment is presented. Flooded areas are extracted from satellite images for the flood‐prone territory, and a maximum flood extent image for each flood event is produced. These maps are further fused to determine relative frequency of inundation (RFI). The study shows that RFI values and relative water depth exhibit the same probabilistic distribution, which is confirmed by Kolmogorov‐Smirnov test. The produced RFI map can be used as a flood hazard map, especially in cases when flood modeling is complicated by lack of available data and high uncertainties. The derived RFI map is further used for flood risk assessment. Efficiency of the presented approach is demonstrated for the Katima Mulilo region (Namibia). A time series of Landsat‐5/7 satellite images acquired from 1989 to 2012 is processed to derive RFI map using the presented approach. The following direct damage categories are considered in the study for flood risk assessment: dwelling units, roads, health facilities, and schools. The produced flood risk map shows that the risk is distributed uniformly all over the region. The cities and villages with the highest risk are identified. The proposed approach has minimum data requirements, and RFI maps can be generated rapidly to assist rescuers and decisionmakers in case of emergencies. On the other hand, limitations include: strong dependence on the available data sets, and limitations in simulations with extrapolated water depth values.     

Context in the Risk Assessment of Digital Systems

As the use of digital computers for instrumentation and control of safety-critical systems has increased, there has been a growing debate over the issue of whether probabilistic risk assessment techniques can be applied to these systems. This debate has centered on the issue of whether software failures can be modeled probabilistically. This paper describes a context-based approach to software risk assessment that explicitly recognizes the fact that the behavior of software is not probabilistic. The source of the perceived uncertainty in its behavior results from both the input to the software as well as the application and environment in which the software is operating. Failures occur as the result of encountering some context for which the software was not properly designed, as opposed to the software simply failing randomly. The paper elaborates on the concept of error-forcing context as it applies to software. It also illustrates a methodology which utilizes event trees, fault trees, and the Dynamic Flowgraph Methodology (DFM) to identify error-forcing contexts for software in the form of fault tree prime implicants.     

Incorporating Ethnographic Methods in Multidisciplinary Approaches to Risk Assessment and Communication: Cultural and Religious Uses of Mercury in Latino and Caribbean Communities

The potential risk from cultural and religious practices involving mercury in Latino and Caribbean communities raises central methodological and ethical questions for risk assessment and risk communication. Here, specific cultural practices unfamiliar to most risk professionals carry significant inherent risks in the eyes of those professionals but not necessarily in the eyes of practitioners. Practitioners' past experience and history as targets of religious suppression and anti-immigrant sentiment create a reluctance to engage with researchers or public health officials in risk assessment or preventive risk communication efforts. The potential for the risk--in this case mercury contamination in dwellings--to extend beyond the practicing community to future occupants adds to public health concern. Understanding the risks of these practices requires both an understanding of the cultural and political context, beliefs, and behaviors of mercury users and an understanding of the fate and transport of mercury in typical use scenarios. In this study, we employed ethnographic methods (interviews and participant observation) to understand beliefs and behavior about mercury use as well as quantitative modeling and measurement to estimate and assess potential exposures. This represents a new methodology tailored to situations in which traditional activities or observances that are integral components of cultural identity pose risks in and of themselves. Our findings indicate that there are different types of mercury use stemming from different cultural and religious traditions that result in different levels of exposure. Many of the mercury uses that can result in the highest exposures to mercury vapors have previously been attributed to the religious tradition of Santeria, but appear instead to have their roots outside of the religion.     

Effect of Providing the Uncertainty Information About a Tornado Occurrence on the Weather Recipients’ Cognition and Protective Action: Probabilistic Hazard Information Versus Deterministic Warnings

Currently, a binary alarm system is used in the United States to issue deterministic warning polygons in case of tornado events. To enhance the effectiveness of the weather information, a likelihood alarm system, which uses a tool called probabilistic hazard information (PHI), is being developed at National Severe Storms Laboratory to issue probabilistic information about the threat. This study aims to investigate the effects of providing the uncertainty information about a tornado occurrence through the PHI's graphical swath on laypeople's concern, fear, and protective action, as compared with providing the warning information with the deterministic polygon. The displays of color‐coded swaths and deterministic polygons were shown to subjects. Some displays had a blue background denoting the probability of any tornado formation in the general area. Participants were asked to report their levels of concern, fear, and protective action at randomly chosen locations within each of seven designated levels on each display. Analysis of a three‐stage nested design showed that providing the uncertainty information via the PHI would appropriately increase recipients’ levels of concern, fear, and protective action in highly dangerous scenarios, with a more than 60% chance of being affected by the threat, as compared with deterministic polygons. The blue background and the color‐coding type did not have a significant effect on the people's cognition of the threat and reaction to it. This study shows that using a likelihood alarm system leads to more conscious decision making by the weather information recipients and enhances the system safety.     

Comparative assessment of analytical approaches to quantify the risk for introduction of rare animal diseases: the example of avian influenza in Spain

Trade of animals and animal products imposes an uncertain and variable risk for exotic animal diseases introduction into importing countries. Risk analysis provides importing countries with an objective, transparent, and internationally accepted method for assessing that risk. Over the last decades, European Union countries have conducted probabilistic risk assessments quite frequently to quantify the risk for rare animal diseases introduction into their territories. Most probabilistic animal health risk assessments have been typically classified into one-level and multilevel binomial models. One-level models are more simple than multilevel models because they assume that animals or products originate from one single population. However, it is unknown whether such simplification may result in substantially different results compared to those obtained through the use of multilevel models. Here, data used on a probabilistic multilevel binomial model formulated to assess the risk for highly pathogenic avian influenza introduction into Spain were reanalyzed using a one-level binomial model and their outcomes were compared. An alternative ordinal model is also proposed here, which makes use of simpler assumptions and less information compared to those required by traditional one-level and multilevel approaches. Results suggest that, at least under certain circumstances, results of the one-level and ordinal approaches are similar to those obtained using multilevel models. Consequently, we argue that, when data are insufficient to run traditional probabilistic models, the ordinal approach presented here may be a suitable alternative to rank exporting countries in terms of the risk that they impose for the spread of rare animal diseases into disease-free countries.     

Application of FISK,an Invasiveness Screening Tool for Non‐Native Freshwater Fishes,in the Murray‐Darling Basin (Southeastern Australia)

The Fish Invasiveness Scoring Kit (FISK) is currently one of the most popular pre‐screening tools for freshwater fishes. A recent upgrade has ensured its wider climatic relevance to countries with subtropical regions. This enhancement is of particular importance to Australia, which encompasses tropical, arid, and temperate zones, and where the introduction of non‐native fish species poses a significant risk to biodiversity. In this study, 55 fish species previously evaluated in a U.K.‐based calibration of FISK are reassessed for their potential invasiveness in the Murray‐Darling Basin (MDB; southeastern Australia), the continent's largest catchment encompassing arid and temperate climates. Approximately half of the species were classed as “medium risk” and the other half as “high risk,” and the ≥19 threshold previously identified from the calibration study was confirmed. The three highest scoring species (common carp Cyprinus carpio carpio, goldfish Carassius auratus, and eastern mosquitofish Gambusia holbrooki) were those already present and invasive in the area, whereas nearly half of the tropical and subtropical species had lower scores compared to U.K. assessments, possibly because of climate change predictions of drier conditions across the MDB. There were some discordances between FISK and two Australian‐based assessment protocols, one of which is qualitative and the other represents a simplified version of FISK. Notably, the Australian origins of FISK should provide for an additional reason for further applications of the tool in other RA areas (i.e., drainage basins) of the continent, ultimately encouraging adoption as the country's reference screening tool for management and conservation purposes.