An Integrated Scenario Ensemble-Based Framework for Hurricane Evacuation Modeling: Part 2—Hazard Modeling

Hurricane track and intensity can change rapidly in unexpected ways, thus making predictions of hurricanes and related hazards uncertain. This inherent uncertainty often translates into suboptimal decision-making outcomes, such as unnecessary evacuation. Representing this uncertainty is thus critical in evacuation planning and related activities. We describe a physics-based hazard modeling approach that (1) dynamically accounts for the physical interactions among hazard components and (2) captures hurricane evolution uncertainty using an ensemble method. This loosely coupled model system provides a framework for probabilistic water inundation and wind speed levels for a new, risk-based approach to evacuation modeling, described in a companion article in this issue. It combines the Weather Research and Forecasting (WRF) meteorological model, the Coupled Routing and Excess STorage (CREST) hydrologic model, and the ADvanced CIRCulation (ADCIRC) storm surge, tide, and wind-wave model to compute inundation levels and wind speeds for an ensemble of hurricane predictions. Perturbations to WRF's initial and boundary conditions and different model physics/parameterizations generate an ensemble of storm solutions, which are then used to drive the coupled hydrologic + hydrodynamic models. Hurricane Isabel (2003) is used as a case study to illustrate the ensemble-based approach. The inundation, river runoff, and wind hazard results are strongly dependent on the accuracy of the mesoscale meteorological simulations, which improves with decreasing lead time to hurricane landfall. The ensemble envelope brackets the observed behavior while providing “best-case” and “worst-case” scenarios for the subsequent risk-based evacuation model.     

The Effects of Presenting Imprecise Probabilities in Intelligence Forecasts

How to assess and present analytic uncertainty to policymakers has emerged as an important topic in risk and policy analysis. Due to the complexity and deep uncertainty present in many forecasting domains, these reports are often fraught with analytic uncertainty. In three studies, we explore the effect of presenting probability assessments and analytic uncertainty through probability ranges. Participants were presented with mock intelligence forecasts that include narrative evidence as well as numerical probability assessments. Participants were sensitive to the ambiguity communicated through the confidence range. The narrative appeared to have a smaller effect on judgments when accompanied by a probability range as opposed to a point assessment. In one study, participants also thought that the probability range was more useful for decision making at a higher probability whereas the point estimate was more useful at a lower probability. When evaluating a forecast in hindsight, decisionmakers tended to report lower levels of blame and higher levels of source credibility for forecasts that reported ranges as compared to point assessments. These findings suggest that decisionmakers are not necessarily “ambiguity averse” in the forecasting context. Presenting ranges of probability may have distinct advantages as a way to communicate probability and analytic confidence to decisionmakers.     

Factors Affecting Hurricane Evacuation Intentions

Protective actions for hurricane threats are a function of the environmental and information context; individual and household characteristics, including cultural worldviews, past hurricane experiences, and risk perceptions; and motivations and barriers to actions. Using survey data from the Miami‐Dade and Houston‐Galveston areas, we regress individuals’ stated evacuation intentions on these factors in two information conditions: (1) seeing a forecast that a hurricane will hit one's area, and (2) receiving an evacuation order. In both information conditions having an evacuation plan, wanting to keep one's family safe, and viewing one's home as vulnerable to wind damage predict increased evacuation intentions. Some predictors of evacuation intentions differ between locations; for example, Florida respondents with more egalitarian worldviews are more likely to evacuate under both information conditions, and Florida respondents with more individualist worldviews are less likely to evacuate under an evacuation order, but worldview was not significantly associated with evacuation intention for Texas respondents. Differences by information condition also emerge, including: (1) evacuation intentions decrease with age in the evacuation order condition but increase with age in the saw forecast condition, and (2) evacuation intention in the evacuation order condition increases among those who rely on public sources of information on hurricane threats, whereas in the saw forecast condition evacuation intention increases among those who rely on personal sources. Results reinforce the value of focusing hurricane information efforts on evacuation plans and residential vulnerability and suggest avenues for future research on how hurricane contexts shape decision making.     

Communicating uncertainty in national security intelligence: Expert and nonexpert interpretations of and preferences for verbal and numeric formats

Organizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst's confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N = 481 and 624, respectively), uncertainty communication preferences of expert (n = 41 intelligence analysts in Experiment 1) and nonexpert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated verbal probability and confidence terms as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that although most nonexperts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative. Experts and nonexperts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately one-fourth of experts and over one-half of nonexperts provided incoherent numeric probability translations for the terms likely and unlikely when the elicitation of best estimates and lower and upper bounds were briefly spaced by intervening tasks.     

Process Tracing Analysis of Hurricane Information Displays

To study people's processing of hurricane forecast advisories, we conducted a computer‐based experiment that examined 11 research questions about the information seeking patterns of students assuming the role of a county emergency manager in a sequence of six hurricane forecast advisories for each of four different hurricanes. The results show that participants considered a variety of different sources of information—textual, graphic, and numeric—when tracking hurricanes. Click counts and click durations generally gave the same results but there were some significant differences. Moreover, participants’ information search strategies became more efficient over forecast advisories and with increased experience tracking the four hurricanes. These changes in the search patterns from the first to the fourth hurricane suggest that the presentation of abstract principles in a training manual was not sufficient for them to learn how to track hurricanes efficiently but they were able to significantly improve their search efficiency with a modest amount (roughly an hour) of practice. Overall, these data indicate that information search patterns are complex and deserve greater attention in studies of dynamic decision tasks.     

Public perceptions of hurricane modification

If hurricane modification were to become a feasible strategy for potentially reducing hurricane damages, it would likely generate public discourse about whether to support its implementation. To facilitate an informed and constructive discourse, policymakers need to understand how people perceive hurricane modification. Here, we examine Florida residents' perceptions of hurricane modification techniques that aim to alter path and wind speed. Following the mental models approach, we conducted a survey study about public perceptions of hurricane modification that was guided by formative interviews on the topic. We report a set of four primary findings. First, hurricane modification was perceived as a relatively ineffective strategy for damage reduction, compared to other strategies for damage reduction. Second, hurricane modification was expected to lead to changes in projected hurricane path, but not necessarily to the successful reduction of projected hurricane strength. Third, more anger was evoked when a hurricane was described as having changed from the initially forecasted path or strength after an attempted modification. Fourth, unlike what we expected, participants who more strongly agreed with statements that recognized the uncertainty inherent in forecasts reported more rather than less anger at scientists across hurricane modification scenarios. If the efficacy of intensity-reduction techniques can be increased, people may be willing to support hurricane modification. However, such an effort would need to be combined with open and honest communications to members of the general public.     

Dynamic Simulation as an Approach to Understanding Hurricane Risk Response: Insights from the Stormview Lab

This article investigates the use of dynamic laboratory simulations as a tool for studying decisions to prepare for hurricane threats. A prototype web‐based simulation named Stormview is described that allows individuals to experience the approach of a hurricane in a computer‐based environment. In Stormview participants can gather storm information through various media, hear the opinions of neighbors, and indicate intentions to take protective action. We illustrate how the ability to exert experimental control over the information viewed by participants can be used to provide insights into decision making that would be difficult to gain from field studies, such as how preparedness decisions are affected by the nature of news coverage of prior storms, how a storm's movement is depicted in graphics, and the content of word‐of‐mouth communications. Data from an initial application involving a sample of Florida residents reveal a number of unexpected findings about hurricane risk response. Participants who viewed forecast graphics, which contained track lines depicting the most likely path of the storm, for example, had higher levels of preparation than those who saw graphics that showed only uncertainty cones—even among those living far from the predicted center path. Similarly, the participants who were most likely to express worry about an approaching storm and fastest to undertake preparatory action were those who, ironically, had never experienced one. Finally, external validity is evidenced by a close rank‐order correspondence between patterns of information use revealed in the lab and that found in previous cross‐sectional field studies.     

An Integrated Scenario Ensemble-Based Framework for Hurricane Evacuation Modeling: Part 1—Decision Support System

This article introduces a new integrated scenario-based evacuation (ISE) framework to support hurricane evacuation decision making. It explicitly captures the dynamics, uncertainty, and human–natural system interactions that are fundamental to the challenge of hurricane evacuation, but have not been fully captured in previous formal evacuation models. The hazard is represented with an ensemble of probabilistic scenarios, population behavior with a dynamic decision model, and traffic with a dynamic user equilibrium model. The components are integrated in a multistage stochastic programming model that minimizes risk and travel times to provide a tree of evacuation order recommendations and an evaluation of the risk and travel time performance for that solution. The ISE framework recommendations offer an advance in the state of the art because they: (1) are based on an integrated hazard assessment (designed to ultimately include inland flooding), (2) explicitly balance the sometimes competing objectives of minimizing risk and minimizing travel time, (3) offer a well-hedged solution that is robust under the range of ways the hurricane might evolve, and (4) leverage the substantial value of increasing information (or decreasing degree of uncertainty) over the course of a hurricane event. A case study for Hurricane Isabel (2003) in eastern North Carolina is presented to demonstrate how the framework is applied, the type of results it can provide, and how it compares to available methods of a single scenario deterministic analysis and a two-stage stochastic program.     

Effects of Acknowledging Uncertainty about Earthquake Risk Estimates on San Francisco Bay Area Residents’ Beliefs,Attitudes, and Intentions

We test here the risk communication proposition that explicit expert acknowledgment of uncertainty in risk estimates can enhance trust and other reactions. We manipulated such a scientific uncertainty message, accompanied by probabilities (20%, 70%, implicit [“will occur”] 100%) and time periods (10 or 30 years) in major (≥magnitude 8) earthquake risk estimates to test potential effects on residents potentially affected by seismic activity on the San Andreas fault in the San Francisco Bay Area (n = 750). The uncertainty acknowledgment increased belief that these specific experts were more honest and open, and led to statistically (but not substantively) significant increases in trust in seismic experts generally only for the 20% probability (vs. certainty) and shorter versus longer time period. The acknowledgment did not change judged risk, preparedness intentions, or mitigation policy support. Probability effects independent of the explicit admission of expert uncertainty were also insignificant except for judged risk, which rose or fell slightly depending upon the measure of judged risk used. Overall, both qualitative expressions of uncertainty and quantitative probabilities had limited effects on public reaction. These results imply that both theoretical arguments for positive effects, and practitioners’ potential concerns for negative effects, of uncertainty expression may have been overblown. There may be good reasons to still acknowledge experts’ uncertainties, but those merit separate justification and their own empirical tests.     

Sizing Up a Superstorm: Exploring the Role of Recalled Experience and Attribution of Responsibility in Judgments of Future Hurricane Risk

Research suggests that hurricane‐related risk perception is a critical predictor of behavioral response, such as evacuation. Less is known, however, about the precursors of these subjective risk judgments, especially when time has elapsed from a focal event. Drawing broadly from the risk communication, social psychology, and natural hazards literature, and specifically from concepts adapted from the risk information seeking and processing model and the protective action decision model, we examine how individuals’ distant recollections, including attribution of responsibility for the effects of a storm, attitude toward relevant information, and past hurricane experience, relate to risk judgment for a future, similar event. The present study reports on a survey involving U.S. residents in Connecticut, New Jersey, and New York (n = 619) impacted by Hurricane Sandy. While some results confirm past findings, such as that hurricane experience increases risk judgment, others suggest additional complexity, such as how various types of experience (e.g., having evacuated vs. having experienced losses) may heighten or attenuate individual‐level judgments of responsibility. We suggest avenues for future research, as well as implications for federal agencies involved in severe weather/natural hazard forecasting and communication with public audiences.     

Accounting for Both Random Errors and Systematic Errors in Uncertainty Propagation Analysis of Computer Models Involving Experimental Measurements with Monte Carlo Methods

A Monte Carlo method is presented to study the effect of systematic and random errors on computer models mainly dealing with experimental data. It is a common assumption in this type of models (linear and nonlinear regression, and nonregression computer models) involving experimental measurements that the error sources are mainly random and independent with no constant background errors (systematic errors). However, from comparisons of different experimental data sources evidence is often found of significant bias or calibration errors. The uncertainty analysis approach presented in this work is based on the analysis of cumulative probability distributions for output variables of the models involved taking into account the effect of both types of errors. The probability distributions are obtained by performing Monte Carlo simulation coupled with appropriate definitions for the random and systematic errors. The main objectives are to detect the error source with stochastic dominance on the uncertainty propagation and the combined effect on output variables of the models. The results from the case studies analyzed show that the approach is able to distinguish which error type has a more significant effect on the performance of the model. Also, it was found that systematic or calibration errors, if present, cannot be neglected in uncertainty analysis of models dependent on experimental measurements such as chemical and physical properties. The approach can be used to facilitate decision making in fields related to safety factors selection, modeling, experimental data measurement, and experimental design.     

Decision Making for Risk Management: A Comparison of Graphical Methods for Presenting Quantitative Uncertainty

Previous research has shown that people err when making decisions aided by probability information. Surprisingly, there has been little exploration into the accuracy of decisions made based on many commonly used probabilistic display methods. Two experiments examined the ability of a comprehensive set of such methods to effectively communicate critical information to a decision maker and influence confidence in decision making. The second experiment investigated the performance of these methods under time pressure, a situational factor known to exacerbate judgmental errors. Ten commonly used graphical display methods were randomly assigned to participants. Across eight scenarios in which a probabilistic outcome was described, participants were asked questions regarding graph interpretation (e.g., mean) and made behavioral choices (i.e., act; do not act) based on the provided information indicated that decision‐maker accuracy differed by graphical method; error bars and boxplots led to greatest mean estimation and behavioral choice accuracy whereas complementary cumulative probability distribution functions were associated with the highest probability estimation accuracy. Under time pressure, participant performance decreased when making behavioral choices.     

Perceptions and Expected Immediate Reactions to Severe Storm Displays

The National Weather Service has adopted warning polygons that more specifically indicate the risk area than its previous county‐wide warnings. However, these polygons are not defined in terms of numerical strike probabilities (p_s). To better understand people's interpretations of warning polygons, 167 participants were shown 23 hypothetical scenarios in one of three information conditions—polygon‐only (Condition A), polygon + tornadic storm cell (Condition B), and polygon + tornadic storm cell + flanking nontornadic storm cells (Condition C). Participants judged each polygon's p_s and reported the likelihood of taking nine different response actions. The polygon‐only condition replicated the results of previous studies; p_s was highest at the polygon's centroid and declined in all directions from there. The two conditions displaying storm cells differed from the polygon‐only condition only in having p_s just as high at the polygon's edge nearest the storm cell as at its centroid. Overall, p_s values were positively correlated with expectations of continuing normal activities, seeking information from social sources, seeking shelter, and evacuating by car. These results indicate that participants make more appropriate p_s judgments when polygons are presented in their natural context of radar displays than when they are presented in isolation. However, the fact that p_s judgments had moderately positive correlations with both sheltering (a generally appropriate response) and evacuation (a generally inappropriate response) suggests that experiment participants experience the same ambivalence about these two protective actions as people threatened by actual tornadoes.     

What Number is “Fifty‐Fifty”?: Redistributing Excessive 50% Responses in Elicited Probabilities

Studies using open-ended response modes to elicit probabilistic beliefs have sometimes found an elevated frequency (or blip) at 50 in their response distributions. Our previous research suggests that this is caused by intrusion of the phrase "fifty-fifty," which represents epistemic uncertainty, rather than a true numeric probability of 50%. Such inappropriate responses pose a problem for decision analysts and others relying on probabilistic judgments. Using an explicit numeric probability scale (ranging from 0-100%) reduces thinking about uncertain events in verbal terms like "fifty-fifty," and, with it, exaggerated use of the 50 response. Here, we present two procedures for adjusting response distributions for data already collected with open-ended response modes and hence vulnerable to an exaggerated presence of 50%. Each procedure infers the prevalence of 50s had a numeric probability scale been used, then redistributes the excess. The two procedures are validated on some of our own existing data and then applied to judgments elicited from experts in groundwater pollution and bioremediation.     

The Effect of Cognitive Load on Decision Making with Graphically Displayed Uncertainty Information

An experiment examined the ability of five graphical displays to communicate uncertainty information when end users were under cognitive load (i.e., remembering an eight‐digit number). The extent to which people could accurately derive information from the graphs and the adequacy of decisions about optimal behaviors based on the graphs were assessed across eight scenarios in which probabilistic outcomes were described. Results indicated that the load manipulation did not have an overall effect on derivation of information from the graphs (i.e., mean and probability estimation) but did suppress the ability to optimize behavioral choices based on the graph. Cognitive load affected people's use of some graphical displays (basic probability distribution function) more than others. Overall, the research suggests that interpreting basic characteristics of uncertainty data is unharmed under conditions of limited cognitive resources, whereas more deliberative processing is negatively affected.     

Cross-Cultural Differences in Risk Perception: A Model-Based Approach

The present study assessed cross-cultural differences in the perception of financial risks. Students at large universities in Hong Kong, Taiwan, the Netherlands, and the U.S., as well as a group of Taiwanese security analysts rated the riskiness of a set of monetary lotteries. Risk judgments differed with nationality, but not with occupation (students vs. security analysts) and were modeled by the Conjoint Expected Risk (CER) model.⁽¹⁾ Consistent with cultural differences in country uncertainty avoidance,⁽²⁾ CER model parameters of respondents from the two Western countries differed from those of respondents from the two countries with Chinese cultural roots: The risk judgments of respondents from Hong Kong and Taiwan were more sensitive to the magnitude of potential losses and less mitigated by the probability of positive outcomes.     

A Cognitive‐Affective Scale for Hurricane Risk Perception

The aim of this study was to develop a reliable and valid measure of hurricane risk perception. The utility of such a measure lies in the need to understand how people make decisions when facing an evacuation order. This study included participants located within a 15‐mile buffer of the Gulf and southeast Atlantic U.S. coasts. The study was executed as a three‐wave panel with mail surveys in 2010–2012 (T₀ baseline N = 629, 56%; T₁ retention N = 427, 75%; T₂ retention N = 350, 89%). An inventory based on the psychometric model was developed to discriminate cognitive and affective perceptions of hurricane risk, and included open‐ended responses to solicit additional concepts in the T₀ survey. Analysis of the T₀ data modified the inventory and this revised item set was fielded at T₁ and then replicated at T₂. The resulting scales were assessed for validity against existing measures for perception of hurricane risk, dispositional optimism, and locus of control. A measure of evacuation expectation was also examined as a dependent variable, which was significantly predicted by the new measures. The resulting scale was found to be reliable, stable, and largely valid against the comparison measures. Despite limitations involving sample size, bias, and the strength of some reliabilities, it was concluded that the measure has potential to inform approaches to hurricane preparedness efforts and advance planning for evacuation messages, and that the measure has good promise to generalize to other contexts in natural hazards as well as other domains of risk.     

Risk-Adjusted Mission Value: Trading Off Mission Risk for Mission Value

This article develops a decision-theoretic methodology for the risk-adjusted mission value (RAMV) for selecting between alternative missions in the presence of uncertainty in the outcomes of the missions. This methodology permits trading off mission risk for mission value, something that probabilistic risk analysis cannot do unless it explicitly incorporates both mission value and risk aversion of the project management. The methodology, in its complete implementation, is consistent with the decision theory known as expected utility theory, although it differs from conventional decision theory in that the probabilities and all but one of the utilities are not those of the decision maker. The article also introduces a new interpretation of risk aversion. The methodology is consistent with the elementary management concept concerning division of labor. An example is presented for selecting between discrete alternatives-four landing sites on Mars. A second example is presented for selecting among a set of continuous alternatives-a comet flyby distance. The methodology is developed within the context of scientific missions, but the methodology is equally applicable to any situation requiring outcome value judgments, probability judgments, and risk aversion judgments by different constituencies.     

Bayesian Forecasting via Deterministic Model

Rational decision making requires that the total uncertainty about a variate of interest (a predictand) be quantified in terms of a probability distribution, conditional on all available information and knowledge. Suppose the state-of-knowledge is embodied in a deterministic model, which is imperfect and outputs only an estimate of the predictand. Fundamentals are presented of two Bayesian methods for producing a probabilistic forecast via any deterministic model. The Bayesian Processor of Forecast (BPF) quantifies the total uncertainty in terms of a posterior distribution, conditional on model output. The Bayesian Forecasting System (BFS) decomposes the total uncertainty into input uncertainty and model uncertainty, which are characterized independently and then integrated into a predictive distribution. The BFS is compared with Monte Carlo simulation and ensemble forecasting technique, none of which can alone produce a probabilistic forecast that quantifies the total uncertainty, but each can serve as a component of the BFS.     

Using Uncertainty Analysis to Improve Consistency in Regulatory Assessments of Criteria Pollutant Standards

Regulatory impact analyses (RIAs), required for new major federal regulations, are often criticized for not incorporating epistemic uncertainties into their quantitative estimates of benefits and costs. “Integrated uncertainty analysis,” which relies on subjective judgments about epistemic uncertainty to quantitatively combine epistemic and statistical uncertainties, is often prescribed. This article identifies an additional source for subjective judgment regarding a key epistemic uncertainty in RIAs for National Ambient Air Quality Standards (NAAQS)—the regulator's degree of confidence in continuation of the relationship between pollutant concentration and health effects at varying concentration levels. An illustrative example is provided based on the 2013 decision on the NAAQS for fine particulate matter (PM_2.5). It shows how the regulator's justification for setting that NAAQS was structured around the regulator's subjective confidence in the continuation of health risks at different concentration levels, and it illustrates how such expressions of uncertainty might be directly incorporated into the risk reduction calculations used in the rule's RIA. The resulting confidence-weighted quantitative risk estimates are found to be substantially different from those in the RIA for that rule. This approach for accounting for an important source of subjective uncertainty also offers the advantage of establishing consistency between the scientific assumptions underlying RIA risk and benefit estimates and the science-based judgments developed when deciding on the relevant standards for important air pollutants such as PM_2.5.