The Repeatability of Uncertainty and Sensitivity Analyses for Complex Probabilistic Risk Assessments

The performance of a probabilistic risk assessment (PRA) for a nuclear power plant is a complex undertaking, involving the assembly of an accident frequency analysis, an accident progression analysis, a source term analysis, and a consequence analysis. Each of these analyses is, in itself, quite complex. Uncertainties enter into a PRA from each of these analyses. An important focus in recent PRAs has been to incorporate these uncertainties at each stage of the analysis, propagate the subsequent uncertainties through the entire analysis, and include uncertainty in the final results. Monte Carlo procedures based on Latin hypercube sampling provide one way to perform propagations of this type. In this paper, the results of two complete and independent Monte Carlo calculations for a recently completed PRA for a nuclear power plant are compared as a means of providing empirical evidence on the repeatability of uncertainty and sensitivity analyses for large-scale PRA calculations. These calculations use the same variables and analysis structure with two independently generated Latin hypercube samples. The results of the two calculations show a high degree of repeatability for the analysis of a very complex system.     

Risk and Uncertainty Analysis in Government Safety Decisions

Probabilistic risk analysis (PRA) can be an effective tool to assess risks and uncertainties and to set priorities among safety policy options. Based on systems analysis and Bayesian probability, PRA has been applied to a wide range of cases, three of which are briefly presented here: the maintenance of the tiles of the space shuttle, the management of patient risk in anesthesia, and the choice of seismic provisions of building codes for the San Francisco Bay Area. In the quantification of a risk, a number of problems arise in the public sector where multiple stakeholders are involved. In this article, I describe different approaches to the treatments of uncertainties in risk analysis, their implications for risk ranking, and the role of risk analysis results in the context of a safety decision process. I also discuss the implications of adopting conservative hypotheses before proceeding to what is, in essence, a conditional uncertainty analysis, and I explore some implications of different levels of "conservatism" for the ranking of risk mitigation measures.     

A Nuclear Utility's Views on the Use of Probabilistic Risk Assessment

Probabilistic risk assessment (PRA) is a relatively new tool in the nuclear industry. The Reactor Safety Study started the present trend of conducting PRAs for nuclear power plants when it was published in 1975. Now, nine years later, those in the industry currently using PRA techniques are frequently asked the same question: Why should the nuclear utility industry, with so many accepted analytical tools already available, invest the time and manpower to develop a new technique with so many uncertainties?     

Probabilistic Risk Analysis for a High-Level Radioactive Waste Repository

A probabilistic risk analysis (PRA) for a high-level radioactive waste repository is very important since it gives an estimate of its health impacts, allowing comparisons to be made with the health impacts of competing technologies. However, it is extremely difficult to develop a credible PRA for a specific repository site because of large uncertainties in future climate, hydrology, geological processes, etc. At best, such a PRA would not be understandable to the public. An alternative proposed here is to develop a PRA for an average U.S. site, taking all properties of the site to be the U.S. average. The results are equivalent to the average results for numerous randomly selected sites. Such a PRA is presented here; it is easy to understand, and it is not susceptible to substantial uncertainty. Applying the results to a specific repository site then requires only a simple, intuitively acceptable "leap of faith" in assuming that with large expenditures of effort and money, experts can select a site that would be at least as secure as a randomly selected site.     

Analysis of Uncertainties in Interactive Probabilistic Safety Analysis (PSA) Models

Probabilistic safety analysis (PSA) has been used in nuclear, chemical, petrochemical, and several other industries. The probability and/or frequency results of most PSAs are based on average component unavailabilities during the mission of interest. While these average results are useful, they provide no indication of the significance of the facility's current status when one or more components are known to be out of service. Recently, several interactive computational models have been developed for nuclear power plants to allow the user to specify the plant's status at a particular time (i.e., to specify equipment known to be out of service) and then to receive updated PSA information. As with conventional PSA results, there are uncertainties associated with the numerical updated results. These uncertainties stem from a number of sources, including parameter uncertainty (uncertainty in equipment failure rates and human error probabilities). This paper presents an analysis of the impact of parameter uncertainty on updated PSA results.     

Expert Elicitation of the Value per Statistical Life in an Air Pollution Context

The monetized value of avoided premature mortality typically dominates the calculated benefits of air pollution regulations; therefore, characterization of the uncertainty surrounding these estimates is key to good policymaking. Formal expert judgment elicitation methods are one means of characterizing this uncertainty. They have been applied to characterize uncertainty in the mortality concentration‐response function, but have yet to be used to characterize uncertainty in the economic values placed on avoided mortality. We report the findings of a pilot expert judgment study for Health Canada designed to elicit quantitative probabilistic judgments of uncertainties in Value‐per‐Statistical‐Life (VSL) estimates for use in an air pollution context. The two‐stage elicitation addressed uncertainties in both a base case VSL for a reduction in mortality risk from traumatic accidents and in benefits transfer‐related adjustments to the base case for an air quality application (e.g., adjustments for age, income, and health status). Results for each expert were integrated to develop example quantitative probabilistic uncertainty distributions for VSL that could be incorporated into air quality models.     

A Combined Monte Carlo and Possibilistic Approach to Uncertainty Propagation in Event Tree Analysis

In risk analysis, the treatment of the epistemic uncertainty associated to the probability of occurrence of an event is fundamental. Traditionally, probabilistic distributions have been used to characterize the epistemic uncertainty due to imprecise knowledge of the parameters in risk models. On the other hand, it has been argued that in certain instances such uncertainty may be best accounted for by fuzzy or possibilistic distributions. This seems the case in particular for parameters for which the information available is scarce and of qualitative nature. In practice, it is to be expected that a risk model contains some parameters affected by uncertainties that may be best represented by probability distributions and some other parameters that may be more properly described in terms of fuzzy or possibilistic distributions. In this article, a hybrid method that jointly propagates probabilistic and possibilistic uncertainties is considered and compared with pure probabilistic and pure fuzzy methods for uncertainty propagation. The analyses are carried out on a case study concerning the uncertainties in the probabilities of occurrence of accident sequences in an event tree analysis of a nuclear power plant.     

Managing Safety‐Related Disruptions: Evidence from the U.S. Nuclear Power Industry

Low‐probability, high‐impact events are difficult to manage. Firms may underinvest in risk assessments for low‐probability, high‐impact events because it is not easy to link the direct and indirect benefits of doing so. Scholarly research on the effectiveness of programs aimed at reducing such events faces the same challenge. In this article, we draw on comprehensive industry‐wide data from the U.S. nuclear power industry to explore the impact of conducting probabilistic risk assessment (PRA) on preventing safety‐related disruptions. We examine this using data from over 25,000 monthly event reports across 101 U.S. nuclear reactors from 1985 to 1998. Using Poisson fixed effects models with time trends, we find that the number of safety‐related disruptions reduced between 8% and 27% per month in periods after operators submitted their PRA in response to the Nuclear Regulatory Commission's Generic Letter 88‐20, which required all operators to conduct a PRA. One possible mechanism for this is that the adoption of PRA may have increased learning rates, lowering the rate of recurring events by 42%. We find that operators that completed their PRA before Generic Letter 88‐20 continued to experience safety improvements during 1990–1995. This suggests that revisiting PRA or conducting it again can be beneficial. Our results suggest that even in a highly safety‐conscious industry as nuclear utilities, a more formal approach to quantifying risk has its benefits.     

Uncertainty Analysis Based on Probability Bounds (P-Box) Approach in Probabilistic Safety Assessment

A wide range of uncertainties will be introduced inevitably during the process of performing a safety assessment of engineering systems. The impact of all these uncertainties must be addressed if the analysis is to serve as a tool in the decision-making process. Uncertainties present in the components (input parameters of model or basic events) of model output are propagated to quantify its impact in the final results. There are several methods available in the literature, namely, method of moments, discrete probability analysis, Monte Carlo simulation, fuzzy arithmetic, and Dempster-Shafer theory. All the methods are different in terms of characterizing at the component level and also in propagating to the system level. All these methods have different desirable and undesirable features, making them more or less useful in different situations. In the probabilistic framework, which is most widely used, probability distribution is used to characterize uncertainty. However, in situations in which one cannot specify (1) parameter values for input distributions, (2) precise probability distributions (shape), and (3) dependencies between input parameters, these methods have limitations and are found to be not effective. In order to address some of these limitations, the article presents uncertainty analysis in the context of level-1 probabilistic safety assessment (PSA) based on a probability bounds (PB) approach. PB analysis combines probability theory and interval arithmetic to produce probability boxes (p-boxes), structures that allow the comprehensive propagation through calculation in a rigorous way. A practical case study is also carried out with the developed code based on the PB approach and compared with the two-phase Monte Carlo simulation results.     

Site-Specific Applications of Probabilistic Health Risk Assessment: Review of the Literature Since 2000

Whether and to what extent contaminated sites harm ecologic and human health are topics of considerable interest, but also considerable uncertainty. Several federal and state agencies have approved the use of some or many aspects of probabilistic risk assessment (PRA), but its site-specific application has often been limited to high-profile sites and large projects. Nonetheless, times are changing: newly developed software tools, and recent federal and state guidance documents formalizing PRA procedures, now make PRA a readily available method of analysis for even small-scale projects. This article presents and discusses a broad review of PRA literature published since 2000.     

Accommodating Uncertainty in Comparative Risk

Comparative risk projects can provide broad policy guidance but they rarely have adequate scientific foundations to support precise risk rankings. Many extant projects report rankings anyway, with limited attention to uncertainty. Stochastic uncertainty, structural uncertainty, and ignorance are types of incertitude that afflict risk comparisons. The recently completed New Jersey Comparative Risk Project was innovative in trying to acknowledge and accommodate some historically ignored uncertainties in a substantive manner. This article examines the methods used and lessons learned from the New Jersey project. Monte Carlo techniques were used to characterize stochastic uncertainty, and sensitivity analysis helped to manage structural uncertainty. A deliberative process and a sorting technique helped manage ignorance. Key findings are that stochastic rankings can be calculated but they reveal such an alarming degree of imprecision that the rankings are no longer useful, whereas sorting techniques are helpful in spite of uncertainty. A deliberative process is helpful to counter analytical overreaching.     

Nuclear Plant PRA: How Far Has It Come?

Nearly ten years have passed since the publication in August 1974 of the draft Reactor Safety Study (WASH 1400), the first detailed attempt to apply probabilistic risk assessment (PRA) techniques to estimate the public risks posed by commercial nuclear power plants. Now is an opportune time to look back and see how PRA has fared over these ten years. We will not attempt to pass judgement on how the Reactor Safety Study report itself has withstood the test of time, as that task is best left to others less directly involved in preparing the report. Instead, we will examine advances in the understanding, acceptance, and utilization of PRA techniques, as well as technical advances in PRA methods. Some of the significant insights gained from PRAs will be discussed. Finally, some observations on the future of PRA will be offered.     

A Comparative Analysis of PRA and Intelligent Adversary Methods for Counterterrorism Risk Management

In counterterrorism risk management decisions, the analyst can choose to represent terrorist decisions as defender uncertainties or as attacker decisions. We perform a comparative analysis of probabilistic risk analysis (PRA) methods including event trees, influence diagrams, Bayesian networks, decision trees, game theory, and combined methods on the same illustrative examples (container screening for radiological materials) to get insights into the significant differences in assumptions and results. A key tenent of PRA and decision analysis is the use of subjective probability to assess the likelihood of possible outcomes. For each technique, we compare the assumptions, probability assessment requirements, risk levels, and potential insights for risk managers. We find that assessing the distribution of potential attacker decisions is a complex judgment task, particularly considering the adaptation of the attacker to defender decisions. Intelligent adversary risk analysis and adversarial risk analysis are extensions of decision analysis and sequential game theory that help to decompose such judgments. These techniques explicitly show the adaptation of the attacker and the resulting shift in risk based on defender decisions.     

Uncertainties in the Link Between Global Climate Change and Predicted Health Risks from Pollution: Hexachlorobenzene (HCB) Case Study Using a Fugacity Model

Industrial societies have altered the earth's environment in ways that could have important, long-term ecological, economic, and health implications. In this paper, we examine the extent to which uncertainty about global climate change could impact the precision of predictions of secondary outcomes such as health impacts of pollution. Using a model that links global climate change with predictions of chemical exposure and human health risk in the Western region of the United States of America (U.S.), we define parameter variabilities and uncertainties and we characterize the resulting outcome variance. As a case study, we consider the public health consequences from releases of hexachlorobenzene (HCB), a ubiquitous multimedia pollutant. By constructing a matrix that links global environmental change both directly and indirectly to potential human-health effects attributable to HCB released into air, soil, and water, we define critical parameter variances in the health risk estimation process. We employ a combined uncertainty/sensitivity analysis to investigate how HCB releases are affected by increasing atmospheric temperature and the accompanying climate alterations that are anticipated. We examine how such uncertainty impacts both the expected magnitude and calculational precision of potential human exposures and health effects. This assessment reveals that uncertain temperature increases of up to 5°C have little impact on either the magnitude or precision of the public-health consequences estimated under existing climate variations for HCB released into air and water in the Western region of the U.S.     

Sourcing Decisions with Stochastic Supplier Reliability and Stochastic Demand

Supplier sourcing strategies are a crucial factor driving supply chain success. In this paper, we investigate the implications of uncertain supplier reliability on a firm's sourcing decisions in an environment with stochastic demand. In particular, we characterize specific conditions under which a firm should choose a single versus multiple supplier sourcing strategy. In an environment with both uncertain demand and supply, we characterize the total order quantity, the number of suppliers selected for order placement, and the allocation of the total order quantity among these selected suppliers. For deeper managerial insight, we also examine the sensitivity of the optimal sourcing decisions to interactions between uncertainties in product demand and supply reliability. We show that sourcing from a single supplier is an optimal strategy for environments characterized by high levels of demand uncertainty or high salvage values. A numerical analysis based on data obtained from an office products retailer further reinforces our analytical results. In addition, we also find that when minimal order quantities are imposed, there are situations where it is not optimal to place an order with the lowest cost supplier.     

On the Need for Restricting the Probabilistic Analysis in Risk Assessments to Variability

It is common perspective in risk analysis that there are two kinds of uncertainties: i) variability as resulting from heterogeneity and stochasticity (aleatory uncertainty) and ii) partial ignorance or epistemic uncertainties resulting from systematic measurement error and lack of knowledge. Probability theory is recognized as the proper tool for treating the aleatory uncertainties, but there are different views on what is the best approach for describing partial ignorance and epistemic uncertainties. Subjective probabilities are often used for representing this type of ignorance and uncertainties, but several alternative approaches have been suggested, including interval analysis, probability bound analysis, and bounds based on evidence theory. It is argued that probability theory generates too precise results when the background knowledge of the probabilities is poor. In this article, we look more closely into this issue. We argue that this critique of probability theory is based on a conception of risk assessment being a tool to objectively report on the true risk and variabilities. If risk assessment is seen instead as a method for describing the analysts’ (and possibly other stakeholders’) uncertainties about unknown quantities, the alternative approaches (such as the interval analysis) often fail in providing the necessary decision support.     

How Much Uncertainty is Too Much and How Do We Know? A Case Example of the Assessment of Ozone Monitor Network Options

Limited time and resources usually characterize environmental decision making at policy organizations such as the U.S. Environmental Protection Agency. In these climates, addressing uncertainty, usually considered a flaw in scientific analyses, is often avoided. However, ignoring uncertainties can result in unpleasant policy surprises. Furthermore, it is important for decisionmakers to know how defensible a chosen policy option is over other options when the uncertainties of the data are considered. The purpose of this article is to suggest an approach that is unique from other approaches in that it considers uncertainty in two specific ways-the uncertainty of stakeholder values within a particular decision context and data uncertainty in the light of the decision-contextual data-values relationship. It is the premise of this article that the interaction between data and stakeholder values is critical to how the decision options are viewed and determines the effect of data uncertainty on the relative acceptability of the decision options, making the understanding of this interaction important to decisionmakers and other stakeholders. This approach utilizes the recently developed decision analysis framework and process, multi-criteria integrated resource assessment (MIRA). This article will specifically address how MIRA can be used to help decisionmakers better understand the importance of uncertainty on the specific (i.e., decision contextual) environmental policy options that they are deliberating.     

Critical Review and Uncertainty Analysis of Factors Influencing Influenza Transmission

Influenza remains a significant threat to public health, yet there is significant uncertainty about the routes of influenza transmission from an infectious source through the environment to a receptor, and their relative risks. Herein, data pertaining to factors that influence the environmental mediation of influenza transmission are critically reviewed, including: frequency, magnitude and size distribution and virus expiration, inactivation rates, environmental and self‐contact rates, and viral transfer efficiencies during contacts. Where appropriate, two‐stage Monte Carlo uncertainty analysis is used to characterize variability and uncertainty in the reported data. Significant uncertainties are present in most factors, due to: limitations in instrumentation or study realism; lack of documentation of data variability; or lack of study. These analyses, and future experimental work, will improve parameterization of influenza transmission and risk models, facilitating more robust characterization of the magnitude and uncertainty in infection risk.     

A Probabilistic Approach for Deriving Acceptable Human Intake Limits and Human Health Risks from Toxicological Studies: General Framework

The use of uncertainty factors in the standard method for deriving acceptable intake or exposure limits for humans, such as the Reference Dose (RfD), may be viewed as a conservative method of taking various uncertainties into account. As an obvious alternative, the use of uncertainty distributions instead of uncertainty factors is gaining attention. This paper presents a comprehensive discussion of a general framework that quantifies both the uncertainties in the no-adverse-effect level in the animal (using a benchmark-like approach) and the uncertainties in the various extrapolation steps involved (using uncertainty distributions). This approach results in an uncertainty distribution for the no-adverse-effect level in the sensitive human subpopulation, reflecting the overall scientific uncertainty associated with that level. A lower percentile of this distribution may be regarded as an acceptable exposure limit (e.g., RfD) that takes account of the various uncertainties in a nonconservative fashion. The same methodology may also be used as a tool to derive a distribution for possible human health effects at a given exposure level. We argue that in a probabilistic approach the uncertainty in the estimated no-adverse-effect-level in the animal should be explicitly taken into account. Not only is this source of uncertainty too large to be ignored, it also has repercussions for the quantification of the other uncertainty distributions.     

Sensitivity and Uncertainty Studies of the CRAC2 Computer Code

We have studied the sensitivity of health impacts from nuclear reactor accidents, as predicted by the CRAC2 computer code, to the following sources of uncertainty: (1) the model for plume rise, (2) the model for wet deposition, (3) the meteorological bin-sampling procedure for selecting weather sequences with rain, (4) the dose conversion factors for inhalation as affected by uncertainties in the particle size of the carrier aerosol and the clearance rates of radionuclides from the respiratory tract, (5) the weathering half-time for external ground-surface exposure, and (6) the transfer coefficients for terrestrial foodchain pathways. Predicted health impacts usually showed little sensitivity to use of an alternative plume-rise model or a modified rain-bin structure in bin-sampling. Health impacts often were quite sensitive to use of an alternative wet-deposition model in single-trial runs with rain during plume passage, but were less sensitive to the model in bin-sampling runs. Uncertainties in the inhalation dose conversion factors had important effects on early injuries in single-trial runs. Latent cancer fatalities were moderately sensitive to uncertainties in the weathering half-time for ground-surface exposure, but showed little sensitivity to the transfer coefficients for terrestrial foodchain pathways. Sensitivities of CRAC2 predictions to uncertainties in the models and parameters also depended on the magnitude of the source term, and some of the effects on early health effects were comparable to those that were due only to selection of different sets of weather sequences in bin-sampling.