Decision Making for Risk Management: A Comparison of Graphical Methods for Presenting Quantitative Uncertainty

Previous research has shown that people err when making decisions aided by probability information. Surprisingly, there has been little exploration into the accuracy of decisions made based on many commonly used probabilistic display methods. Two experiments examined the ability of a comprehensive set of such methods to effectively communicate critical information to a decision maker and influence confidence in decision making. The second experiment investigated the performance of these methods under time pressure, a situational factor known to exacerbate judgmental errors. Ten commonly used graphical display methods were randomly assigned to participants. Across eight scenarios in which a probabilistic outcome was described, participants were asked questions regarding graph interpretation (e.g., mean) and made behavioral choices (i.e., act; do not act) based on the provided information indicated that decision‐maker accuracy differed by graphical method; error bars and boxplots led to greatest mean estimation and behavioral choice accuracy whereas complementary cumulative probability distribution functions were associated with the highest probability estimation accuracy. Under time pressure, participant performance decreased when making behavioral choices.     

Designing Graphs that Promote Both Risk Understanding and Behavior Change

Graphs show promise for improving communications about different types of risks, including health risks, financial risks, and climate risks. However, graph designs that are effective at meeting one important risk communication goal (promoting risk‐avoidant behaviors) can at the same time compromise another key goal (improving risk understanding). We developed and tested simple bar graphs aimed at accomplishing these two goals simultaneously. We manipulated two design features in graphs, namely, whether graphs depicted the number of people affected by a risk and those at risk of harm (“foreground+background”) versus only those affected (“foreground‐only”), and the presence versus absence of simple numerical labels above bars. Foreground‐only displays were associated with larger risk perceptions and risk‐avoidant behavior (i.e., willingness to take a drug for heart attack prevention) than foreground+background displays, regardless of the presence of labels. Foreground‐only graphs also hindered risk understanding when labels were not present. However, the presence of labels significantly improved understanding, eliminating the detrimental effect of foreground‐only displays. Labels also led to more positive user evaluations of the graphs, but did not affect risk‐avoidant behavior. Using process modeling we identified mediators (risk perceptions, understanding, user evaluations) that explained the effect of display type on risk‐avoidant behavior. Our findings contribute new evidence to the graph design literature: unlike what was previously feared, we demonstrate that it is possible to design foreground‐only graphs that promote intentions for behavior change without a detrimental effect on risk understanding. Implications for the design of graphical risk communications and decision support are discussed.     

Effect of Providing the Uncertainty Information About a Tornado Occurrence on the Weather Recipients’ Cognition and Protective Action: Probabilistic Hazard Information Versus Deterministic Warnings

Currently, a binary alarm system is used in the United States to issue deterministic warning polygons in case of tornado events. To enhance the effectiveness of the weather information, a likelihood alarm system, which uses a tool called probabilistic hazard information (PHI), is being developed at National Severe Storms Laboratory to issue probabilistic information about the threat. This study aims to investigate the effects of providing the uncertainty information about a tornado occurrence through the PHI's graphical swath on laypeople's concern, fear, and protective action, as compared with providing the warning information with the deterministic polygon. The displays of color‐coded swaths and deterministic polygons were shown to subjects. Some displays had a blue background denoting the probability of any tornado formation in the general area. Participants were asked to report their levels of concern, fear, and protective action at randomly chosen locations within each of seven designated levels on each display. Analysis of a three‐stage nested design showed that providing the uncertainty information via the PHI would appropriately increase recipients’ levels of concern, fear, and protective action in highly dangerous scenarios, with a more than 60% chance of being affected by the threat, as compared with deterministic polygons. The blue background and the color‐coding type did not have a significant effect on the people's cognition of the threat and reaction to it. This study shows that using a likelihood alarm system leads to more conscious decision making by the weather information recipients and enhances the system safety.     

Understanding the Positive Effects of Graphical Risk Information on Comprehension: Measuring Attention Directed to Written,Tabular, and Graphical Risk Information

Risk communications are an integral aspect of health education and promotion. However, the commonly used textual risk information is relatively difficult to understand for the average recipient. Consequently, researchers and health promoters have started to focus on so‐called decision aids, such as tables and graphs. Although tabular and graphical risk information more effectively communicate risks than textual risk information, the cognitive mechanisms responsible for this enhancement are unclear. This study aimed to examine two possible mechanisms (i.e., cognitive workload and attention). Cognitive workload (mean pupil size and peak pupil dilation) and attention directed to the risk information (viewing time, number of eye fixations, and eye fixation durations) were both measured in a between‐subjects experimental design. The results suggest that graphical risk information facilitates comprehension of that information because it attracts and holds attention for a longer period of time than textual risk information. Graphs are thus a valuable asset to risk communication practice for two reasons: first, they tend to attract attention and, second, when attended to, they elicit information extraction with relatively little cognitive effort, and finally result in better comprehension.     

Effects of Track and Threat Information on Judgments of Hurricane Strike Probability

Although evacuation is one of the best strategies for protecting citizens from hurricane threat, the ways that local elected officials use hurricane data in deciding whether to issue hurricane evacuation orders is not well understood. To begin to address this problem, we examined the effects of hurricane track and intensity information in a laboratory setting where participants judged the probability that hypothetical hurricanes with a constant bearing (i.e., straight line forecast track) would make landfall in each of eight 45 degree sectors around the Gulf of Mexico. The results from 162 participants in a student sample showed that the judged strike probability distributions over the eight sectors within each scenario were, unsurprisingly, unimodal and centered on the sector toward which the forecast track pointed. More significantly, although strike probability judgments for the sector in the direction of the forecast track were generally higher than the corresponding judgments for the other sectors, the latter were not zero. Most significantly, there were no appreciable differences in the patterns of strike probability judgments for hurricane tracks represented by a forecast track only, an uncertainty cone only, or forecast track with an uncertainty cone—a result consistent with a recent survey of coastal residents threatened by Hurricane Charley. The study results suggest that people are able to correctly process basic information about hurricane tracks but they do make some errors. More research is needed to understand the sources of these errors and to identify better methods of displaying uncertainty about hurricane parameters.     

Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature*

A considerable amount of research has been conducted over a long period of time into the effects of graphical and tabular representations on decision-making performance. To date, however, the literature appears to have arrived at few conclusions with regard to the performance of the two representations. This paper addresses these issues by presenting a theory, based on information processing theory, to explain under what circumstances one representation outperforms the other. The fundamental aspects of the theory are: (1) although graphical and tabular representations may contain the same information, they present that information in fundamentally different ways; graphical representations emphasize spatial information, while tables emphasize symbolic information; (2) tasks can be divided into two types, spatial and symbolic, based on the type of information that facilitates their solution; (3) performance on a task will be enhanced when there is a cognitive fit (match) between the information emphasized in the representation type and that required by the task type; that is, when graphs support spatial tasks and when tables support symbolic tasks; (4) the processes or strategies problem solvers use are the crucial elements of cognitive fit since they provide the link between representation and task; the processes identified here are perceptual and analytical; (5) so long as there is a complete fit of representation, processes, and task type, each representation will lead to both quicker and more accurate problem solving. The theory is validated by its success in explaining the results of published studies that examine the performance of graphical and tabular representations in decision making.     

Probability Size Matters: The Effect of Foreground-Only versus Foreground+Background Graphs on Risk Aversion Diminishes with Larger Probabilities

Graphs are increasingly recommended for improving decision-making and promoting risk-avoidant behaviors. Graphs that depict only the number of people affected by a risk (“foreground-only” displays) tend to increase perceived risk and risk aversion (e.g., willingness to get vaccinated), as compared to graphs that also depict the number of people at risk for harm (“foreground+background” displays). However, previous research examining these “foreground-only effects” has focused on relatively low-probability risks (<10%), limiting generalizability to communications about larger risks. In two experiments, we systematically investigated the moderating role of probability size on foreground-only effects, using a wide range of probability sizes (from 0.1% to 40%). Additionally, we examined the moderating role of the size of the risk reduction, that is, the extent to which a protective behavior reduces the risk. Across both experiments, foreground-only effects on perceived risk and risk aversion were weaker for larger probabilities. Experiment 2 also revealed that foreground-only effects were weaker for smaller risk reductions, while foreground-only displays decreased understanding of absolute risk magnitudes independently of probability size. These findings suggest that the greater effectiveness of foreground-only versus foreground+background displays for increasing perceived risk and risk aversion diminishes with larger probability sizes and smaller risk reductions. Moreover, if the goal is to promote understanding of absolute risk magnitudes, foreground+background displays should be used rather than foreground-only displays regardless of probability size. Our findings also help to refine and extend existing theoretical accounts of foreground-only effects to situations involving a wide range of probability sizes.     

The Effects of an Interactive Graphics-Based DSS to Support Problem Structuring

Graphical, interactive, problem-structuring tools were devised and incorporated into a DSS. These tools were based on concepts of structural modeling and motivated by propositions and findings in memory theory and imagery theory. A laboratory study was then conducted to investigate the relationship between decision-maker characteristics and use of the problem-structuring aid. The experiment was designed to determine whether use of the tool to assist in problem formulation led to a better understanding of problem structure. A moderately complex management simulation and its companion DSS were used in the experiment. Eighty-four subjects were divided into two groups: a control group that did not use the Graphical Interactive Structural Modeling Option (GISMO), and an experimental group that did. The results showed a statistically significant effect between cognitive ability and use of GISMO. Field independents (i.e., high cognitive ability) effectively combined GISMO use with image processing skills to understand the structure of the problem domain, but field dependents (i.e., low cognitive ability) were ineffective. There was very little difference in problem understanding in field dependents who used the tool and those who did not.     

Contractor Accreditation: A Probabilistic Model

The selection of competent contractors is a critical function in all business organizations. In contrast to other types of vendors (e.g., distributors, manufacturers, etc.), contractors are typically accredited before any business transaction takes place. In such situations, there is often a considerable amount of uncertainty associated with the accreditation process. This research presents a probabilistic model for accrediting contractors. We discuss a methodology in which probability measures are used to capture the uncertainty inherent in the decision process. These probabilities are estimated from data on (i) past applicants and (ii) their eventual performance, if accredited. Furthermore, these probabilities are used to determine when additional information about an applicant should be collected, as well as what kind of information would be most relevant for the vendor under consideration.     

On Some Recent Definitions and Analysis Frameworks for Risk,Vulnerability, and Resilience

Recently, considerable attention has been paid to a systems‐based approach to risk, vulnerability, and resilience analysis. It is argued that risk, vulnerability, and resilience are inherently and fundamentally functions of the states of the system and its environment. Vulnerability is defined as the manifestation of the inherent states of the system that can be subjected to a natural hazard or be exploited to adversely affect that system, whereas resilience is defined as the ability of the system to withstand a major disruption within acceptable degradation parameters and to recover within an acceptable time, and composite costs, and risks. Risk, on the other hand, is probability based, defined by the probability and severity of adverse effects (i.e., the consequences). In this article, we look more closely into this approach. It is observed that the key concepts are inconsistent in the sense that the uncertainty (probability) dimension is included for the risk definition but not for vulnerability and resilience. In the article, we question the rationale for this inconsistency. The suggested approach is compared with an alternative framework that provides a logically defined structure for risk, vulnerability, and resilience, where all three concepts are incorporating the uncertainty (probability) dimension.     

Graphical Communication of Uncertain Quantities to Nontechnical People

Nine pictorial displays for communicating quantitative information about the value of an uncertain quantity, x , were evaluated for their ability to communicate , p ( x > a ) and p ( b > x > a ) to well-educated semi-and nontechnical subjects. Different displays performed best in different applications. Cumulative distribution functions alone can severely mislead some subjects in estimating the mean. A "rusty" knowledge of statistics did not improve performance, and even people with a good basic knowledge of statistics did not perform as well as one would like. Until further experiments are performed, the authors recommend the use of a cumulative distribution function plotted directly above a probability density function with the same horizontal scale, and with the location of the mean clearly marked on both curves.     

Analysis of Uncertainties in Interactive Probabilistic Safety Analysis (PSA) Models

Probabilistic safety analysis (PSA) has been used in nuclear, chemical, petrochemical, and several other industries. The probability and/or frequency results of most PSAs are based on average component unavailabilities during the mission of interest. While these average results are useful, they provide no indication of the significance of the facility's current status when one or more components are known to be out of service. Recently, several interactive computational models have been developed for nuclear power plants to allow the user to specify the plant's status at a particular time (i.e., to specify equipment known to be out of service) and then to receive updated PSA information. As with conventional PSA results, there are uncertainties associated with the numerical updated results. These uncertainties stem from a number of sources, including parameter uncertainty (uncertainty in equipment failure rates and human error probabilities). This paper presents an analysis of the impact of parameter uncertainty on updated PSA results.     

Influence of Distributional Shape of Substance Parameters on Exposure Model Output

Uncertainty of environmental concentrations is calculated with the regional multimedia exposure model of EUSES 1.0 by considering probability input distributions for aqueous solubility, vapor pressure, and octanol-water partition coefficient, K(ow). Only reliable experimentally determined data are selected from available literature for eight reference chemicals representing a wide substance property spectrum. Monte Carlo simulations are performed with uniform, triangular, and log-normal input distributions to assess the influence of the choice of input distribution type on the predicted concentration distributions. The impact of input distribution shapes on output variance exceeds the effect on the output mean by one order of magnitude. Both are affected by influence and uncertainty (i.e., variance) of the input variable as well. Distributional shape has no influence when the sensitivity function of the respective parameter is perfectly linear. For nonlinear relationships, overlap of probability mass of input distribution with influential ranges of the parameter space is important. Differences in computed output distribution are greatest when input distributions differ in the most influential parameter range.     

Hybrid Processing of Stochastic and Subjective Uncertainty Data

Uncertainty analyses typically recognize separate stochastic and subjective sources of uncertainty, but do not systematically combine the two, although a large amount of data used in analyses is partly stochastic and partly subjective. We have developed methodology for mathematically combining stochastic and subjective sources of data uncertainty, based on new "hybrid number" approaches. The methodology can be utilized in conjunction with various traditional techniques, such as PRA (probabilistic risk assessment) and risk analysis decision support. Hybrid numbers have been previously examined as a potential method to represent combinations of stochastic and subjective information, but mathematical processing has been impeded by the requirements inherent in the structure of the numbers, e.g., there was no known way to multiply hybrids. In this paper, we will demonstrate methods for calculating with hybrid numbers that avoid the difficulties. By formulating a hybrid number as a probability distribution that is only fuzzily known, or alternatively as a random distribution of fuzzy numbers, methods are demonstrated for the full suite of arithmetic operations, permitting complex mathematical calculations. It will be shown how information about relative subjectivity (the ratio of subjective to stochastic knowledge about a particular datum) can be incorporated. Techniques are also developed for conveying uncertainty information visually, so that the stochastic and subjective components of the uncertainty, as well as the ratio of knowledge about the two, are readily apparent. The techniques demonstrated have the capability to process uncertainty information for independent, uncorrelated data, and for some types of dependent and correlated data. Example applications are suggested, illustrative problems are shown, and graphical results are given.     

Exemplifying “Us”: Integrating social identity theory of leadership with cognitive models of categorization

Identity leadership theorizing suggests that leadership effectiveness derives from a potential leader’s perceived ability to create, embody, promote, and embed a shared group identity. However, little is known about how people integrate this information to form a judgment of a leader. We use cognitive modeling to operationalize leadership judgments as exemplar-and prototype-based categorization processes. Analysis of attribute rating data for 80 highly recognizable Americans revealed that leadership judgments were well-characterized by an exemplar-based model. Judgments were based overwhelmingly on promoting shared collective interests and embedding group identity. The pattern of attribute weightings was consistent for judgments of a general leadership role (i.e., as a competent leader) as well as judgments for a specific leadership role (i.e., as an effective US president). We discuss the implications of these findings for our understanding of identity leadership as well as for integrated social-cognitive models of individuals’ judgements of and responses to leaders.     

At Home on the Range? Lay Interpretations of Numerical Uncertainty Ranges

Numerical uncertainty ranges are often used to convey the precision of a forecast. In three studies, we examined how users perceive the distribution underlying numerical ranges and test specific hypotheses about the display characteristics that affect these perceptions. We discuss five primary conclusions from these studies: (1) substantial variation exists in how people perceive the distribution underlying numerical ranges; (2) distributional perceptions appear similar whether the uncertain variable is a probability or an outcome; (3) the variation in distributional perceptions is due in part to individual differences in numeracy, with more numerate individuals more likely to perceive the distribution as roughly normal; (4) the variation is also due in part to the presence versus absence of common cues used to convey the correct interpretation (e.g., including a best estimate increases perceptions that the distribution is roughly normal); and (5) simple graphical representations can decrease the variance in distributional perceptions. These results point toward significant opportunities to improve uncertainty communication in climate change and other domains.     

A Critical Discussion and Practical Recommendations on Some Issues Relevant to the Nonprobabilistic Treatment of Uncertainty in Engineering Risk Assessment

Models for the assessment of the risk of complex engineering systems are affected by uncertainties due to the randomness of several phenomena involved and the incomplete knowledge about some of the characteristics of the system. The objective of this article is to provide operative guidelines to handle some conceptual and technical issues related to the treatment of uncertainty in risk assessment for engineering practice. In particular, the following issues are addressed: (1) quantitative modeling and representation of uncertainty coherently with the information available on the system of interest; (2) propagation of the uncertainty from the input(s) to the output(s) of the system model; (3) (Bayesian) updating as new information on the system becomes available; and (4) modeling and representation of dependences among the input variables and parameters of the system model. Different approaches and methods are recommended for efficiently tackling each of issues (1)?(4) above; the tools considered are derived from both classical probability theory as well as alternative, nonfully probabilistic uncertainty representation frameworks (e.g., possibility theory). The recommendations drawn are supported by the results obtained in illustrative applications of literature.     

Communicating uncertainty in national security intelligence: Expert and nonexpert interpretations of and preferences for verbal and numeric formats

Organizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst's confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N = 481 and 624, respectively), uncertainty communication preferences of expert (n = 41 intelligence analysts in Experiment 1) and nonexpert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated verbal probability and confidence terms as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that although most nonexperts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative. Experts and nonexperts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately one-fourth of experts and over one-half of nonexperts provided incoherent numeric probability translations for the terms likely and unlikely when the elicitation of best estimates and lower and upper bounds were briefly spaced by intervening tasks.     

Uncertainty Evaluation of Human Risk Analysis (HRA) of Chemicals by Multiple Exposure Routes

The application of an ISO standard procedure (Guide to the Expression of Uncertainty in Measurement (GUM)) is here discussed to quantify uncertainty in human risk estimation under chronic exposure to hazardous chemical compounds. The procedure was previously applied to a simple model; in this article a much more complex model is used, i.e., multiple compound and multiple exposure pathways. Risk was evaluated using the usual methodologies: the deterministic reasonable maximum exposure (RME) and the statistical Monte Carlo method. In both cases, the procedures to evaluate uncertainty on risk values are detailed. Uncertainties were evaluated by different methodologies to account for the peculiarity of information about the single variable. The GUM procedure enables the ranking of variables by their contribution to uncertainty; it provides a criterion for choosing variables for deeper analysis. The obtained results show that the application of GUM procedure is easy and straightforward to quantify uncertainty and variability of risk estimation. Health risk estimation is based on literature data on a water table contaminated by three volatile organic compounds. Daily intake was considered by either ingestion of water or inhalation during showering. The results indicate one of the substances as the main contaminant, and give a criterion to identify the key component on which the treatment selection may be performed and the treatment process may be designed in order to reduce risk.