What to Do at Low Doses: A Bounding Approach for Economic Analysis

To quantify the health benefits of environmental policies, economists generally require estimates of the reduced probability of illness or death. For policies that reduce exposure to carcinogenic substances, these estimates traditionally have been obtained through the linear extrapolation of experimental dose-response data to low-exposure scenarios as described in the U.S. Environmental Protection Agency's Guidelines for Carcinogen Risk Assessment (1986). In response to evolving scientific knowledge, EPA proposed revisions to the guidelines in 1996. Under the proposed revisions, dose-response relationships would not be estimated for carcinogens thought to exhibit nonlinear modes of action. Such a change in cancer-risk assessment methods and outputs will likely have serious consequences for how benefit-cost analyses of policies aimed at reducing cancer risks are conducted. Any tendency for reduced quantification of effects in environmental risk assessments, such as those contemplated in the revisions to EPA's cancer-risk assessment guidelines, impedes the ability of economic analysts to respond to increasing calls for benefit-cost analysis. This article examines the implications for benefit-cost analysis of carcinogenic exposures of the proposed changes to the 1986 Guidelines and proposes an approach for bounding dose-response relationships when no biologically based models are available. In spite of the more limited quantitative information provided in a carcinogen risk assessment under the proposed revisions to the guidelines, we argue that reasonable bounds on dose-response relationships can be estimated for low-level exposures to nonlinear carcinogens. This approach yields estimates of reduced illness for use in a benefit-cost analysis while incorporating evidence of nonlinearities in the dose-response relationship. As an illustration, the bounding approach is applied to the case of chloroform exposure.     

A Distributional Approach to Characterizing Low-Dose Cancer Risk

Since cancer risk at very low doses cannot be directly measured in humans or animals, mathematical extrapolation models and scientific judgment are required. This article demonstrates a probabilistic approach to carcinogen risk assessment that employs probability trees, subjective probabilities, and standard bootstrapping procedures. The probabilistic approach is applied to the carcinogenic risk of formaldehyde in environmental and occupational settings. Sensitivity analyses illustrate conditional estimates of risk for each path in the probability tree. Fundamental mechanistic uncertainties are characterized. A strength of the analysis is the explicit treatment of alternative beliefs about pharmacokinetics and pharmacodynamics. The resulting probability distributions on cancer risk are compared with the point estimates reported by federal agencies. Limitations of the approach are discussed as well as future research directions.     

Estimating Conditional Probabilities of Terrorist Attacks: Modeling Adversaries with Uncertain Value Tradeoffs

Many analyses conducted to inform security decisions depend on estimates of the conditional probabilities of different attack alternatives. These probabilities are difficult to estimate since analysts have limited access to the adversary and limited knowledge of the adversary’s utility function, so subject matter experts often provide the estimates through direct elicitation. In this article, we describe a method of using uncertainty in utility function value tradeoffs to model the adversary’s decision process and solve for the conditional probabilities of different attacks in closed form. The conditional probabilities are suitable to be used as inputs to probabilistic risk assessments and other decision support techniques. The process we describe is an extension of value‐focused thinking and is broadly applicable, including in general business decision making. We demonstrate the use of this technique with simple examples.     

Quantifying Flood Risks in the Netherlands

The Flood Risk in the Netherlands project (Dutch acronym: VNK2) is a large‐scale probabilistic risk assessment for all major levee systems in the Netherlands. This article provides an overview of the methods and techniques used in the VNK2 project. It also discusses two examples that illustrate the potential of quantitative flood risk assessments such as VNK2 to improve flood risk management processes: (i) informing political debates about the risks of flooding and the effectiveness of risk management actions, and (ii) (re)directing research efforts towards important sources of uncertainty.     

Bounding Analysis as an Inadequately Specified Methodology

The bounding analysis methodology described by Ha-Duong et al. (this issue) is logically incomplete and invites serious misuse and misinterpretation, as their own example and interpretation illustrate. A key issue is the extent to which these problems are inherent in their methodology, and resolvable by a logically complete assessment (such as Monte Carlo or Bayesian risk assessment), as opposed to being general problems in any risk-assessment methodology. I here attempt to apportion the problems between those inherent in the proposed bounding analysis and those that are more general, such as reliance on questionable expert elicitations. I conclude that the specific methodology of Ha-Duong et al. suffers from logical gaps in the definition and construction of inputs, and hence should not be used in the form proposed. Furthermore, the labor required to do a sound bounding analysis is great enough so that one may as well skip that analysis and carry out a more logically complete probabilistic analysis, one that will better inform the consumer of the appropriate level uncertainty. If analysts insist on carrying out a bounding analysis in place of more thorough assessments, extensive analyses of sensitivity to inputs and assumptions will be essential to display uncertainties, arguably more essential than it would be in full probabilistic analyses.     

Uncertain Numbers and Uncertainty in the Selection of Input Distributions—Consequences for a Probabilistic Risk Assessment of Contaminated Land

Risks from exposure to contaminated land are often assessed with the aid of mathematical models. The current probabilistic approach is a considerable improvement on previous deterministic risk assessment practices, in that it attempts to characterize uncertainty and variability. However, some inputs continue to be assigned as precise numbers, while others are characterized as precise probability distributions. Such precision is hard to justify, and we show in this article how rounding errors and distribution assumptions can affect an exposure assessment. The outcome of traditional deterministic point estimates and Monte Carlo simulations were compared to probability bounds analyses. Assigning all scalars as imprecise numbers (intervals prescribed by significant digits) added uncertainty to the deterministic point estimate of about one order of magnitude. Similarly, representing probability distributions as probability boxes added several orders of magnitude to the uncertainty of the probabilistic estimate. This indicates that the size of the uncertainty in such assessments is actually much greater than currently reported. The article suggests that full disclosure of the uncertainty may facilitate decision making in opening up a negotiation window. In the risk analysis process, it is also an ethical obligation to clarify the boundary between the scientific and social domains.     

Assessing Uncertainty in Simulation-Based Maritime Risk Assessment

Recent work in the assessment of risk in maritime transportation systems has used simulation-based probabilistic risk assessment techniques. In the Prince William Sound and Washington State Ferries risk assessments, the studies' recommendations were backed up by estimates of their impact made using such techniques and all recommendations were implemented. However, the level of uncertainty about these estimates was not available, leaving the decisionmakers unsure whether the evidence was sufficient to assess specific risks and benefits. The first step toward assessing the impact of uncertainty in maritime risk assessments is to model the uncertainty in the simulation models used. In this article, a study of the impact of proposed ferry service expansions in San Francisco Bay is used as a case study to demonstrate the use of Bayesian simulation techniques to propagate uncertainty throughout the analysis. The conclusions drawn in the original study are shown, in this case, to be robust to the inherent uncertainties. The main intellectual merit of this work is the development of Bayesian simulation technique to model uncertainty in the assessment of maritime risk. However, Bayesian simulations have been implemented only as theoretical demonstrations. Their use in a large, complex system may be considered state of the art in the field of computational sciences.     

A Discussion of Findings and Their Possible Implications from a Workshop on Bioterrorism Threat Assessment and Risk Management

In November 2001, the Monterey Institute of International Studies convened a workshop on bioterrorism threat assessment and risk management. Risk assessment practitioners from various disciplines, but without specialized knowledge of terrorism, were brought together with security and intelligence threat analysts to stimulate an exchange that could be useful to both communities. This article, prepared by a subset of the participants, comments on the workshop's findings and their implications and makes three recommendations, two short term (use of threat assessment methodologies and vulnerability analysis) and one long term (application of quantitative risk assessment and modeling), regarding the practical application of risk assessment methods to bioterrorism issues.     

Quantifying Uncertainty in a Risk Assessment Using Human Data

A call for risk assessment approaches that better characterize and quantify uncertainty has been made by the scientific and regulatory community. This paper responds to that call by demonstrating a distributional approach that draws upon human data to derive potency estimates and to identify and quantify important sources of uncertainty. The approach is rooted in the science of decision analysis and employs an influence diagram, a decision tree, probabilistic weights, and a distribution of point estimates of carcinogenic potency. Its results estimate the likelihood of different carcinogenic risks (potencies) for a chemical under a specific scenario. For this exercise, human data on formaldehyde were employed to demonstrate the approach. Sensitivity analyses were performed to determine the relative impact of specific levels and alternatives on the potency distribution. The resulting potency estimates are compared with the results of an exercise using animal data on formaldehyde. The paper demonstrates that distributional risk assessment is readily adapted to situations in which epidemiologic data serve as the basis for potency estimates. Strengths and weaknesses of the distributional approach are discussed. Areas for further application and research are recommended.     

Historical Relationship Between Performance Assessment for Radioactive Waste Disposal and Other Types of Risk Assessment

This article describes the evolution of the process for assessing the hazards of a geologic disposal system for radioactive waste and, similarly, nuclear power reactors, and the relationship of this process with other assessments of risk, particularly assessments of hazards from manufactured carcinogenic chemicals during use and disposal. This perspective reviews the common history of scientific concepts for risk assessment developed until the 1950s. Computational tools and techniques developed in the late 1950s and early 1960s to analyze the reliability of nuclear weapon delivery systems were adopted in the early 1970s for probabilistic risk assessment of nuclear power reactors, a technology for which behavior was unknown. In turn, these analyses became an important foundation for performance assessment of nuclear waste disposal in the late 1970s. The evaluation of risk to human health and the environment from chemical hazards is built on methods for assessing the dose response of radionuclides in the 1950s. Despite a shared background, however, societal events, often in the form of legislation, have affected the development path for risk assessment for human health, producing dissimilarities between these risk assessments and those for nuclear facilities. An important difference is the regulator's interest in accounting for uncertainty.     

Risk Framework for an Organizational System With Major Components

Many large organizations accomplish their various functions through interactions across their major components. Components refers to functional entities within a large complex organization, such as business sectors, academic departments, or regional divisions. The dependency between the various components can cause risk to propagate through their overall system. This article presents a risk assessment framework that integrates risk across a diverse set of components to the overall organization functions. This project addresses three major challenges: aggregating risk, estimating component interdependencies including cycles of dependencies, and propagating risk across components. The framework aggregates risk assessments through a value function for severity that is evaluated at the expected outcome of accomplishing planned goals in terms of performance, schedule, and resources. The value function, which represents risk tolerance, scales between defined points corresponding to failure and success. Different risk assessment may be aggregated together. This article presents a novel approach to establishing relationships between the various components. This article develops and compares three network risk propagation models that characterize the overall organizational risk. The U.S. Air Force has applied this risk framework to evaluate success in hypothetical future wars. The analysts employing this risk framework have informed billions of dollars of strategic investment decisions.     

The [R]Evolving Relationship Between Risk Assessment and Risk Management

In Science and Decisions: Advancing Risk Assessment, the National Research Council recommends improvements in the U.S. Environmental Protection Agency's approach to risk assessment. The recommendations aim to increase the utility of these assessments, embedding them within a new risk‐based decision‐making framework. The framework involves first identifying the problem and possible options for addressing it, conducting related analyses, then reviewing the results and making the risk management decision. Experience with longstanding requirements for regulatory impact analysis provides insights into the implementation of this framework. First, neither the Science and Decisions framework nor the framework for regulatory impact analysis should be viewed as a static or linear process, where each step is completed before moving on to the next. Risk management options are best evaluated through an iterative and integrative procedure. The extent to which a hazard has been previously studied will strongly influence analysts’ ability to identify options prior to conducting formal analyses, and these options will be altered and refined as the analysis progresses. Second, experience with regulatory impact analysis suggests that legal and political constraints may limit the range of options assessed, contrary to both existing guidance for regulatory impact analysis and the Science and Decisions recommendations. Analysts will need to work creatively to broaden the range of options considered. Finally, the usefulness of regulatory impact analysis has been significantly hampered by the inability to quantify many health impacts of concern, suggesting that the scientific improvements offered within Science and Decisions will fill an crucial research gap.     

Variability and Uncertainty Meet Risk Management and Risk Communication

In the past decade, the use of probabilistic risk analysis techniques to quantitatively address variability and uncertainty in risks increased in popularity as recommended by the 1994 National Research Council that wrote Science and Judgment in Risk Assessment. Under the 1996 Food Quality Protection Act, for example, the U.S. EPA supported the development of tools that produce distributions of risk demonstrating the variability and/or uncertainty in the results. This paradigm shift away from the use of point estimates creates new challenges for risk managers, who now struggle with decisions about how to use distributions in decision making. The challenges for risk communication, however, have only been minimally explored. This presentation uses the case studies of variability in the risks of dying on the ground from a crashing airplane and from the deployment of motor vehicle airbags to demonstrate how better characterization of variability and uncertainty in the risk assessment lead to better risk communication. Analogies to food safety and environmental risks are also discussed. This presentation demonstrates that probabilistic risk assessment has an impact on both risk management and risk communication, and highlights remaining research issues associated with using improved sensitivity and uncertainty analyses in risk assessment.     

How Probabilistic Risk Assessment Can Mislead Terrorism Risk Analysts

Traditional probabilistic risk assessment (PRA), of the type originally developed for engineered systems, is still proposed for terrorism risk analysis. We show that such PRA applications are unjustified in general. The capacity of terrorists to seek and use information and to actively research different attack options before deciding what to do raises unique features of terrorism risk assessment that are not adequately addressed by conventional PRA for natural and engineered systems—in part because decisions based on such PRA estimates do not adequately hedge against the different probabilities that attackers may eventually act upon. These probabilities may differ from the defender's (even if the defender's experts are thoroughly trained, well calibrated, unbiased probability assessors) because they may be conditioned on different information. We illustrate the fundamental differences between PRA and terrorism risk analysis, and suggest use of robust decision analysis for risk management when attackers may know more about some attack options than we do.     

The Benefits of Probabilistic Exposure Assessment: Three Case Studies Involving Contaminated Air,Water, and Soil1

Probabilistic risk assessments are enjoying increasing popularity as a tool to characterize the health hazards associated with exposure to chemicals in the environment. Because probabilistic analyses provide much more information to the risk manager than standard “point” risk estimates, this approach has generally been heralded as one which could significantly improve the conduct of health risk assessments. The primary obstacles to replacing point estimates with probabilistic techniques include a general lack of familiarity with the approach and a lack of regulatory policy and guidance. This paper discusses some of the advantages and disadvantages of the point estimate vs. probabilistic approach. Three case studies are presented which contrast and compare the results of each. The first addresses the risks associated with household exposure to volatile chemicals in tapwater. The second evaluates airborne dioxin emissions which can enter the food-chain. The third illustrates how to derive health-based cleanup levels for dioxin in soil. It is shown that, based on the results of Monte Carlo analyses of probability density functions (PDFs), the point estimate approach required by most regulatory agencies will nearly always overpredict the risk for the 95^th percentile person by a factor of up to 5. When the assessment requires consideration of 10 or more exposure variables, the point estimate approach will often predict risks representative of the 99.9^th percentile person rather than the 50^th or 95^th percentile person. This paper recommends a number of data distributions for various exposure variables that we believe are now sufficiently well understood to be used with confidence in most exposure assessments. A list of exposure variables that may require additional research before adequate data distributions can be developed are also discussed.     

Behavioral Economics and Regulatory Analysis

Behavioral economics has captured the interest of scholars and the general public by demonstrating ways in which individuals make decisions that appear irrational. While increasing attention is being focused on the implications of this research for the design of risk‐reducing policies, less attention has been paid to how it affects the economic valuation of policy consequences. This article considers the latter issue, reviewing the behavioral economics literature and discussing its implications for the conduct of benefit‐cost analysis, particularly in the context of environmental, health, and safety regulations. We explore three concerns: using estimates of willingness to pay or willingness to accept compensation for valuation, considering the psychological aspects of risk when valuing mortality‐risk reductions, and discounting future consequences. In each case, we take the perspective that analysts should avoid making judgments about whether values are “rational” or “irrational.” Instead, they should make every effort to rely on well‐designed studies, using ranges, sensitivity analysis, or probabilistic modeling to reflect uncertainty. More generally, behavioral research has led some to argue for a more paternalistic approach to policy analysis. We argue instead for continued focus on describing the preferences of those affected, while working to ensure that these preferences are based on knowledge and careful reflection.     

Elusive Critical Elements of Transformative Risk Assessment Practice and Interpretation: Is Alternatives Analysis the Next Step?

This article argues that “game‐changing” approaches to risk analysis must focus on “democratizing” risk analysis in the same way that information technologies have democratized access to, and production of, knowledge. This argument is motivated by the author's reading of Goble and Bier's analysis, “Risk Assessment Can Be a Game‐Changing Information Technology—But Too Often It Isn't” (Risk Analysis, 2013; 33: 1942–1951), in which living risk assessments are shown to be “game changing” in probabilistic risk analysis. In this author's opinion, Goble and Bier's article focuses on living risk assessment's potential for transforming risk analysis from the perspective of risk professionals—yet, the game‐changing nature of information technologies has typically achieved a much broader reach. Specifically, information technologies change who has access to, and who can produce, information. From this perspective, the author argues that risk assessment is not a game‐changing technology in the same way as the printing press or the Internet because transformative information technologies reduce the cost of production of, and access to, privileged knowledge bases. The author argues that risk analysis does not reduce these costs. The author applies Goble and Bier's metaphor to the chemical risk analysis context, and in doing so proposes key features that transformative risk analysis technology should possess. The author also discusses the challenges and opportunities facing risk analysis in this context. These key features include: clarity in information structure and problem representation, economical information dissemination, increased transparency to nonspecialists, democratized manufacture and transmission of knowledge, and democratic ownership, control, and interpretation of knowledge. The chemical safety decision‐making context illustrates the impact of changing the way information is produced and accessed in the risk context. Ultimately, the author concludes that although new chemical safety regulations do transform access to risk information, they do not transform the costs of producing this information—rather, they change the bearer of these costs. The need for further risk assessment transformation continues to motivate new practical and theoretical developments in risk analysis and management.     

Stepping Out of Your Own Shadow: A Didactic Example of How Facing Uncertainty Can Improve Decision-Making

Advocates of quantitative uncertainty analysis (QUA) have invested substantial effort in explaining why uncertainty is a crucial aspect of risk and yet have devoted much less effort to explaining how QUA can improve the risk manager's performance. This paper develops a teaching example, using a personal decision problem with subtle parallels to societal risk management, to show how choices made with increasing appreciation of uncertainty are superior ones. In the hypothetical, five analysts explain the same uncertain prospect (whether to invest in a volatile stock issue), with increasing attention to the nuances of uncertainty. The path through these different perspectives on the decision demonstrates four general points applicable to environmental risk management: (1) Various point estimates with equal claim to being "best estimates" can differ markedly from each other and lead to diametrically different choices; (2) "conservatism" has both relative and absolute meanings, with different implications for decision-making; (3) both inattention to and fixation on "outliers" in the uncertainty distribution can lead the manager astray; and (4) the best QUA is one that helps discriminate among real options, that points to optimum pathways toward new information, and that spurs on the iterative search for new decision options that may outperform any of the initial ones offered.     

Improving Health Risk Assessment as a Basis for Public Health Decisions in the 21st Century

One-fifth of the way through the 21st century, a commonality of factors with those of the last 50 years may offer the opportunity to address unfinished business and current challenges. The recommendations include: (1) Resisting the tendency to oversimplify scientific assessments by reliance on single disciplines in lieu of clear weight-of-evidence expressions, and on single quantitative point estimates of health protective values for policy decisions; (2) Improving the separation of science and judgment in risk assessment through the use of clear expressions of the range of judgments that bracket protective quantitative levels for public health protection; (3) Use of comparative risk to achieve the greatest gains in health and the environment; and (4) Where applicable, reversal of the risk assessment and risk management steps to facilitate timely and substantive improvements in public health and the environment. Lessons learned and improvements in the risk assessment process are applied to the unprecedented challenges of the 21st century such as, pandemics and climate change. The beneficial application of the risk assessment and risk management paradigm to ensure timely research with consistency and transparency of assessments is presented. Institutions with mandated stability and leadership roles at the national and international levels are essential to ensure timely interdisciplinary scientific assessment at the interface with public policy as a basis for organized policy decisions, to meet time sensitive goals, and to inform the public.