The Use of Narrative Evidence and Explicit Likelihood by Decisionmakers Varying in Numeracy

Decisionmakers are often presented with explicit likelihood assessments (e.g., there is a 10% chance that an attack will occur over the next three months) and supporting narrative evidence in forecasting and risk communication domains. Decisionmakers are thought to rely on both numerical and narrative information to the extent that they perceive the information to be diagnostic, accurate, and trustworthy. In two studies, we explored how lay decisionmakers varying in numeracy evaluated and used likelihood assessments and narrative evidence in forecasts. Overall, the less numerate reported higher risk and likelihood perceptions. In simple probabilistic forecasts without narrative evidence, decisionmakers at all levels of numeracy were able to use the stated likelihood information, although risk perceptions of the less numerate were more affected by likelihood format. When a forecast includes narrative evidence, decisionmakers were better able to use stated likelihood in a percentage as compared to frequency or verbal formats. The more numerate used stated likelihood more in their evaluations whereas the less numerate focused more on the narrative evidence. These results have important implications for risk analysts and forecasters who need to report the results of their analyses to decisionmakers. Decisionmakers varying in numerical ability may evaluate forecasts in different ways depending on the types of information they find easiest to evaluate.     

A Framework for Understanding Uncertainty in Seismic Risk Assessment

A better understanding of the uncertainty that exists in models used for seismic risk assessment is critical to improving risk-based decisions pertaining to earthquake safety. Current models estimating the probability of collapse of a building do not consider comprehensively the nature and impact of uncertainty. This article presents a model framework to enhance seismic risk assessment and thus gives decisionmakers a fuller understanding of the nature and limitations of the estimates. This can help ensure that risks are not over- or underestimated and the value of acquiring accurate data is appreciated fully. The methodology presented provides a novel treatment of uncertainties in input variables, their propagation through the model, and their effect on the results. The study presents ranges of possible annual collapse probabilities for different case studies on buildings in different parts of the world, exposed to different levels of seismicity, and with different vulnerabilities. A global sensitivity analysis was conducted to determine the significance of uncertain variables. Two key outcomes are (1) that the uncertainty in ground-motion conversion equations has the largest effect on the uncertainty in the calculation of annual collapse probability; and (2) the vulnerability of a building appears to have an effect on the range of annual collapse probabilities produced, i.e., the level of uncertainty in the estimate of annual collapse probability, with less vulnerable buildings having a smaller uncertainty.     

A Method for Computing the Fraction of Attributable Risk Related to Climate Damages

The recent decision of the U.S. Supreme Court on the regulation of CO₂ emissions from new motor vehicles^{( 1 )} shows the need for a robust methodology to evaluate the fraction of attributable risk from such emissions. The methodology must enable decisionmakers to reach practically relevant conclusions on the basis of expert assessments the decisionmakers see as an expression of research in progress, rather than as knowledge consolidated beyond any reasonable doubt.^{( 2,3,4 )} This article presents such a methodology and demonstrates its use for the Alpine heat wave of 2003. In a Bayesian setting, different expert assessments on temperature trends and volatility can be formalized as probability distributions, with initial weights (priors) attached to them. By Bayesian learning, these weights can be adjusted in the light of data. The fraction of heat wave risk attributable to anthropogenic climate change can then be computed from the posterior distribution. We show that very different priors consistently lead to the result that anthropogenic climate change has contributed more than 90% to the probability of the Alpine summer heat wave in 2003. The present method can be extended to a wide range of applications where conclusions must be drawn from divergent assessments under uncertainty.     

Approaches to displaying information to assist decisions under uncertainty

The estimation of the costs of a product or project and the decisions based on these forecasts are subject to much uncertainty relating to factors like unknown future developments. This has been addressed repeatedly in research studies focusing on different aspects of uncertainty; unfortunately, this interest has not yet been adopted in practice. One reason can be found in the inadequate representation of uncertainty. This paper introduces an experiment, which engages different approaches to displaying cost forecasting information to gauge the consideration of uncertainty in the subsequent decision-making process. Three different approaches of displaying cost-forecasting information including the uncertainty involved in the data were tested, namely a three point trend forecast, a bar chart, and a FAN-diagram. Furthermore, the effects of using different levels of contextual information about the decision problem were examined. The results show that decision makers tend to simplify the level of uncertainty from a possible range of future outcomes to the limited form of a point estimate. Furthermore, the contextual information made the participants more aware of uncertainty. In addition, the fan-diagram prompted 75.0% of the participants to consider uncertainty even if they had not used this type of diagram before; it was therefore identified as the most suitable method of graphical information display for encouraging decision makers to consider the uncertainty in cost forecasting.     

ON THE IMPLICATIONS OF SPECIFICATION UNCERTAINTY IN FORECASTING*

Forecasters typically select a statistical forecasting model from among a set of alternative models. Subsequently, forecasts are generated with the chosen model and reported to management (forecast consumers) as if specification uncertainty did not exist (i.e., as if the chosen model were the “true” model of the forecast variable). In this note, a well-known Bayesian model-comparison procedure is used to illustrate some of the ambiguities and distortions of forecasts that do not reflect specification uncertainty. It is shown that a single selected forecasting model (however chosen) will generally misstate measures of forecast risk and lead to point and interval forecasts that are misplaced from a decision-theoretic point of view.     

Structure of Uncertainty and Decision Making: An Experimental Investigation*

Decisions in the real world usually involve imprecise information or uncertainty about the precesses by which outcomes may be determined. This research reports the results of a laboratory experiment which examined whether the structure of uncertainty, namely, both the center and the range of the probability distribution describing the uncertainty, is an important determinant of choice. Specifically, it examines how the uncertainty of audit by the Internal Revenue Service of income tax returns affects taxpayers' decisions about intentional noncompliance. The context is relevant as almost nothing is known about how taxpayers assess detection risks using the probability information they have. The study focuses on intentional noncompliance. The factors affecting it are distinct and separate from those affecting unintentional noncompliance. Other factors that affect intentional tax noncompliance, such as risk, tax rates, and penalty rates, were controlled in the experiment. It was hypothesized that the lower the mean and the lesser the range (ambiguity) of the perceived audit probability, the greater the international noncompliance. As hypothesized, the analysis indicates that both the mean and the range of the perceived audit probability rate affect intentional noncompliance, though the effect of ambiguity is greater at a relatively higher level of mean. This result suggests that the strength of the information describing an uncertain event is captured better by both the mean and the range of the uncertainty than either of those components singly.     

Provider–user differences in perceived usefulness of forecasting formats

This paper aims to examine potential differences in perceived usefulness of various forecasting formats from the perspectives of providers and users of predictions. Experimental procedure consists of asking participants to assume the role of forecast providers and to construct forecasts using different formats, followed by requesting usefulness ratings for these formats (Phase 1). Usefulness of the formats are rated again in hindsight after receiving individualized performance feedback (Phase 2). In the ensuing role switch exercise, given new series and external predictions, participants are required to assign usefulness ratings as forecast users (Phase 3). In the last phase, participants are given performance feedback and asked to rate the usefulness in hindsight as users of predictions (Phase 4). Results reveal that regardless of the forecasting role, 95% prediction intervals are considered to be the most useful format, followed by directional predictions, 50% interval forecasts, and lastly, point forecasts. Finally, for all formats and for both roles, usefulness in hindsight is found to be lower than usefulness prior to performance feedback presentation.     

Quantifying Water Pathogen Risk in an Epidemiological Framework

Traditionally, microbial risk assessors have used point estimates to evaluate the probability that an individual will become infected. We developed a quantitative approach that shifts the risk characterization perspective from point estimate to distributional estimate, and from individual to population. To this end, we first designed and implemented a dynamic model that tracks traditional epidemiological variables such as the number of susceptible, infected, diseased, and immune, and environmental variables such as pathogen density. Second, we used a simulation methodology that explicitly acknowledges the uncertainty and variability associated with the data. Specifically, the approach consists of assigning probability distributions to each parameter, sampling from these distributions for Monte Carlo simulations, and using a binary classification to assess the output of each simulation. A case study is presented that explores the uncertainties in assessing the risk of giardiasis when swimming in a recreational impoundment using reclaimed water. Using literature-based information to assign parameters ranges, our analysis demonstrated that the parameter describing the shedding of pathogens by infected swimmers was the factor that contributed most to the uncertainty in risk. The importance of other parameters was dependent on reducing the a priori range of this shedding parameter. By constraining the shedding parameter to its lower subrange, treatment efficiency was the parameter most important in predicting whether a simulation resulted in prevalences above or below non outbreak levels. Whereas parameters associated with human exposure were important when the shedding parameter was constrained to a higher subrange. This Monte Carlo simulation technique identified conditions in which outbreaks and/or nonoutbreaks are likely and identified the parameters that most contributed to the uncertainty associated with a risk prediction.     

On the Treatment of Uncertainty and Variability in Making Decisions About Risk

Much attention has been paid to the treatment of dependence and to the characterization of uncertainty and variability (including the issue of dependence among inputs) in performing risk assessments to avoid misleading results. However, with relatively little progress in communicating about the effects and implications of dependence, the effort involved in performing relatively sophisticated risk analyses (e.g., two‐dimensional Monte Carlo analyses that separate variability from uncertainty) may be largely wasted, if the implications of those analyses are not clearly understood by decisionmakers. This article emphasizes that epistemic uncertainty can introduce dependence among related risks (e.g., risks to different individuals, or at different facilities), and illustrates the potential importance of such dependence in the context of two important types of decisions—evaluations of risk acceptability for a single technology, and comparisons of the risks for two or more technologies. We also present some preliminary ideas on how to communicate the effects of dependence to decisionmakers in a clear and easily comprehensible manner, and suggest future research directions in this area.     

The futility of forecasting

Long-range forecasting is an integral part of planning, but relying on its accuracy may be a mistake. The landscape is strewn with often wildly inaccurate forecasts. This article studies performances of some forecasts, analyses factors contributing to forecast error, and suggests ways in which management may deal with the uncertainty resulting from faulty forecasting performances.     

Uncertain Numbers and Uncertainty in the Selection of Input Distributions—Consequences for a Probabilistic Risk Assessment of Contaminated Land

Risks from exposure to contaminated land are often assessed with the aid of mathematical models. The current probabilistic approach is a considerable improvement on previous deterministic risk assessment practices, in that it attempts to characterize uncertainty and variability. However, some inputs continue to be assigned as precise numbers, while others are characterized as precise probability distributions. Such precision is hard to justify, and we show in this article how rounding errors and distribution assumptions can affect an exposure assessment. The outcome of traditional deterministic point estimates and Monte Carlo simulations were compared to probability bounds analyses. Assigning all scalars as imprecise numbers (intervals prescribed by significant digits) added uncertainty to the deterministic point estimate of about one order of magnitude. Similarly, representing probability distributions as probability boxes added several orders of magnitude to the uncertainty of the probabilistic estimate. This indicates that the size of the uncertainty in such assessments is actually much greater than currently reported. The article suggests that full disclosure of the uncertainty may facilitate decision making in opening up a negotiation window. In the risk analysis process, it is also an ethical obligation to clarify the boundary between the scientific and social domains.     

Deterrence and Risk Preferences in Sequential Attacker–Defender Games with Continuous Efforts

Most attacker–defender games consider players as risk neutral, whereas in reality attackers and defenders may be risk seeking or risk averse. This article studies the impact of players' risk preferences on their equilibrium behavior and its effect on the notion of deterrence. In particular, we study the effects of risk preferences in a single‐period, sequential game where a defender has a continuous range of investment levels that could be strategically chosen to potentially deter an attack. This article presents analytic results related to the effect of attacker and defender risk preferences on the optimal defense effort level and their impact on the deterrence level. Numerical illustrations and some discussion of the effect of risk preferences on deterrence and the utility of using such a model are provided, as well as sensitivity analysis of continuous attack investment levels and uncertainty in the defender's beliefs about the attacker's risk preference. A key contribution of this article is the identification of specific scenarios in which the defender using a model that takes into account risk preferences would be better off than a defender using a traditional risk‐neutral model. This study provides insights that could be used by policy analysts and decisionmakers involved in investment decisions in security and safety.     

Assessing Uncertainty in Simulation-Based Maritime Risk Assessment

Recent work in the assessment of risk in maritime transportation systems has used simulation-based probabilistic risk assessment techniques. In the Prince William Sound and Washington State Ferries risk assessments, the studies' recommendations were backed up by estimates of their impact made using such techniques and all recommendations were implemented. However, the level of uncertainty about these estimates was not available, leaving the decisionmakers unsure whether the evidence was sufficient to assess specific risks and benefits. The first step toward assessing the impact of uncertainty in maritime risk assessments is to model the uncertainty in the simulation models used. In this article, a study of the impact of proposed ferry service expansions in San Francisco Bay is used as a case study to demonstrate the use of Bayesian simulation techniques to propagate uncertainty throughout the analysis. The conclusions drawn in the original study are shown, in this case, to be robust to the inherent uncertainties. The main intellectual merit of this work is the development of Bayesian simulation technique to model uncertainty in the assessment of maritime risk. However, Bayesian simulations have been implemented only as theoretical demonstrations. Their use in a large, complex system may be considered state of the art in the field of computational sciences.     

我国A股市场的模糊性溢价——基于日内高频数据的分析

基于Izhakian(2020)不确定性概率分布下的预期效用理论框架,本文实证研究了我国A股市场的模糊性溢价问题。利用上证综指的日内高频收益数据估计日收益率分布,以月度内日收益率分布的波动性衡量月度市场模糊性,并对模糊性-风险-收益的权衡关系进行实证检验。结果发现:(1)我国A股市场存在时变的模糊性,其模糊性溢价不总是正数,而是依赖于投资者对市场的预期;(2)模糊性溢价和风险溢价受到市场预期的影响,好的市场预期有正的模糊性溢价(模糊厌恶)和负的风险溢价(风险爱好),不好的市场预期有负的模糊性溢价(模糊爱好)和正的风险溢价(风险厌恶)。这些发现不同于当前文献中的许多结果,可以很好地解释市场中的"追涨杀跌"现象。     

How Much Uncertainty is Too Much and How Do We Know? A Case Example of the Assessment of Ozone Monitor Network Options

Limited time and resources usually characterize environmental decision making at policy organizations such as the U.S. Environmental Protection Agency. In these climates, addressing uncertainty, usually considered a flaw in scientific analyses, is often avoided. However, ignoring uncertainties can result in unpleasant policy surprises. Furthermore, it is important for decisionmakers to know how defensible a chosen policy option is over other options when the uncertainties of the data are considered. The purpose of this article is to suggest an approach that is unique from other approaches in that it considers uncertainty in two specific ways-the uncertainty of stakeholder values within a particular decision context and data uncertainty in the light of the decision-contextual data-values relationship. It is the premise of this article that the interaction between data and stakeholder values is critical to how the decision options are viewed and determines the effect of data uncertainty on the relative acceptability of the decision options, making the understanding of this interaction important to decisionmakers and other stakeholders. This approach utilizes the recently developed decision analysis framework and process, multi-criteria integrated resource assessment (MIRA). This article will specifically address how MIRA can be used to help decisionmakers better understand the importance of uncertainty on the specific (i.e., decision contextual) environmental policy options that they are deliberating.     

How to Model a Negligible Probability Under the WTO Sanitary and Phytosanitary Agreement?

Since the 1997 EC – Hormones decision, World Trade Organization (WTO) Dispute Settlement Panels have wrestled with the question of what constitutes a negligible risk under the Sanitary and Phytosanitary Agreement. More recently, the 2010 WTO Australia – Apples Panel focused considerable attention on the appropriate quantitative model for a negligible probability in a risk assessment. The 2006 Australian Import Risk Analysis for Apples from New Zealand translated narrative probability statements into quantitative ranges. The uncertainty about a “negligible” probability was characterized as a uniform distribution with a minimum value of zero and a maximum value of 10^?6. The Australia – Apples Panel found that the use of this distribution would tend to overestimate the likelihood of “negligible” events and indicated that a triangular distribution with a most probable value of zero and a maximum value of 10^?6 would correct the bias. The Panel observed that the midpoint of the uniform distribution is 5 × 10^?7 but did not consider that the triangular distribution has an expected value of 3.3 × 10^?7. Therefore, if this triangular distribution is the appropriate correction, the magnitude of the bias found by the Panel appears modest. The Panel's detailed critique of the Australian risk assessment, and the conclusions of the WTO Appellate Body about the materiality of flaws found by the Panel, may have important implications for the standard of review for risk assessments under the WTO SPS Agreement.     

Model Uncertainty and Model Averaging in the Estimation of Infectious Doses for Microbial Pathogens

Food‐borne infection is caused by intake of foods or beverages contaminated with microbial pathogens. Dose‐response modeling is used to estimate exposure levels of pathogens associated with specific risks of infection or illness. When a single dose‐response model is used and confidence limits on infectious doses are calculated, only data uncertainty is captured. We propose a method to estimate the lower confidence limit on an infectious dose by including model uncertainty and separating it from data uncertainty. The infectious dose is estimated by a weighted average of effective dose estimates from a set of dose‐response models via a Kullback information criterion. The confidence interval for the infectious dose is constructed by the delta method, where data uncertainty is addressed by a bootstrap method. To evaluate the actual coverage probabilities of the lower confidence limit, a Monte Carlo simulation study is conducted under sublinear, linear, and superlinear dose‐response shapes that can be commonly found in real data sets. Our model‐averaging method achieves coverage close to nominal in almost all cases, thus providing a useful and efficient tool for accurate calculation of lower confidence limits on infectious doses.     

Mlonte Carlo Techniques for Quantitative Uncertainty Analysis in Public Health Risk Assessments

Most public health risk assessments assume and combine a series of average, conservative, and worst-case values to derive a conservative point estimate of risk. This procedure has major limitations. This paper demonstrates a new methodology for extended uncertainty analyses in public health risk assessments using Monte Carlo techniques. The extended method begins as do some conventional methods--with the preparation of a spreadsheet to estimate exposure and risk. This method, however, continues by modeling key inputs as random variables described by probability density functions (PDFs). Overall, the technique provides a quantitative way to estimate the probability distributions for exposure and health risks within the validity of the model used. As an example, this paper presents a simplified case study for children playing in soils contaminated with benzene and benzo(a)pyrene (BaP).     

Risk-Based Design of Aseptic Processing of Heterogeneous Food Products

A risk assessment was performed to incorporate uncertainty in food processing conditions to develop a risk-based sterilization process design. The focus of this analysis was uncertainty associated with heterogeneous food products. Quartered button mushrooms were the chosen food product because it represents the most typical type. A model for sterilization of spherical particles was utilized, and each parameter's uncertainty was characterized for use under Monte Carlo simulation. Various particle distributions and fluid types were compared. The output of the model was the required sterilization time to achieve the target sterilization conditions with 95% probability. This value was then used to determine the mean fluid velocity for a given tube length. Finally, the output from the model was analyzed to determine the confidence in output based on uncertainty in the input parameters. The model was more sensitive to variation in particle size distribution than fluid type for power-law fluids. The 90% confidence interval included a holding time range of 1 min. With a 95% confidence level that only 8% of the data will be below the target sterilization conditions, a maximum of 9% of the data were expected to achieve double the target level. The results of such an analysis would be useful for management decisions concerning the design of aseptic food processing operations.