Principles of Good Practice for the Use of Monte Carlo Techniques in Human Health and Ecological Risk Assessments

We propose 14 principles of good practice to assist people in performing and reviewing probabilistic or Monte Carlo risk assessments, especially in the context of the federal and state statutes concerning chemicals in the environment. Monte Carlo risk assessments for hazardous waste sites that follow these principles will be easier to understand, will explicitly distinguish assumptions from data, and will consider and quantify effects that could otherwise lead to misinterpretation of the results. The proposed principles are neither mutually exclusive nor collectively exhaustive. We think and hope that these principles will evolve as new ideas arise and come into practice.     

Integrating Software into PRA: A Test-Based Approach

Probabilistic risk assessment (PRA) is a methodology to assess the probability of failure or success of a system's operation. PRA has been proved to be a systematic, logical, and comprehensive technique for risk assessment. Software plays an increasing role in modern safety critical systems. A significant number of failures can be attributed to software failures. Unfortunately, current probabilistic risk assessment concentrates on representing the behavior of hardware systems, humans, and their contributions (to a limited extent) to risk but neglects the contributions of software due to a lack of understanding of software failure phenomena. It is thus imperative to consider and model the impact of software to reflect the risk in current and future systems. The objective of our research is to develop a methodology to account for the impact of software on system failure that can be used in the classical PRA analysis process. A test-based approach for integrating software into PRA is discussed in this article. This approach includes identification of software functions to be modeled in the PRA, modeling of the software contributions in the ESD, and fault tree. The approach also introduces the concepts of input tree and output tree and proposes a quantification strategy that uses a software safety testing technique. The method is applied to an example system, PACS.     

Nuclear Plant Systems Analysis Research at EPRI

A synopsis is given of research projects on probabilistic systems analysis conducted by the Risk Assessment Program of the Electric Power Research Institute. A brief outline of the value of systems analysis both within PRA and as a stand-alone discipline is used to show how the objectives of the research program are related to industry needs. Research projects and their deliverables are discussed in relation to three objectives: (1) to improve the credibility of the methods and results, (2) to provide computer tools and technology transfer, and (3) to develop use of the methods to improve safety and availability in operation. Recent achievements and future plans are briefly described and an up to date list of relevant EPRI publications is provided.     

Applications of probabilistic risk assessments: the selection of appropriate tools

Probabilistic risk assessment (PRA) is an important methodology for assessing the risks of complex technologies. This paper discusses the strengths and weaknesses of PRA. Its application is explored in three different settings: adversarial policy processes, regulatory/licensing procedures, and plant safety audits. It is concluded that PRA is a valuable tool for auditing safety precautions of existing or planned technologies, especially when it is carried out as an interactive process involving designers and plant personnel who are familiar with actual, everyday operations. PRA has not proven to be as well-suited in providing absolute risk estimates in public-policy debates concerning the acceptability of a technology, or for the licensing and regulatory procedures. The reasons for this are discussed.     

A Nuclear Utility's Views on the Use of Probabilistic Risk Assessment

Probabilistic risk assessment (PRA) is a relatively new tool in the nuclear industry. The Reactor Safety Study started the present trend of conducting PRAs for nuclear power plants when it was published in 1975. Now, nine years later, those in the industry currently using PRA techniques are frequently asked the same question: Why should the nuclear utility industry, with so many accepted analytical tools already available, invest the time and manpower to develop a new technique with so many uncertainties?     

Nuclear Plant PRA: How Far Has It Come?

Nearly ten years have passed since the publication in August 1974 of the draft Reactor Safety Study (WASH 1400), the first detailed attempt to apply probabilistic risk assessment (PRA) techniques to estimate the public risks posed by commercial nuclear power plants. Now is an opportune time to look back and see how PRA has fared over these ten years. We will not attempt to pass judgement on how the Reactor Safety Study report itself has withstood the test of time, as that task is best left to others less directly involved in preparing the report. Instead, we will examine advances in the understanding, acceptance, and utilization of PRA techniques, as well as technical advances in PRA methods. Some of the significant insights gained from PRAs will be discussed. Finally, some observations on the future of PRA will be offered.     

Managing Safety‐Related Disruptions: Evidence from the U.S. Nuclear Power Industry

Low‐probability, high‐impact events are difficult to manage. Firms may underinvest in risk assessments for low‐probability, high‐impact events because it is not easy to link the direct and indirect benefits of doing so. Scholarly research on the effectiveness of programs aimed at reducing such events faces the same challenge. In this article, we draw on comprehensive industry‐wide data from the U.S. nuclear power industry to explore the impact of conducting probabilistic risk assessment (PRA) on preventing safety‐related disruptions. We examine this using data from over 25,000 monthly event reports across 101 U.S. nuclear reactors from 1985 to 1998. Using Poisson fixed effects models with time trends, we find that the number of safety‐related disruptions reduced between 8% and 27% per month in periods after operators submitted their PRA in response to the Nuclear Regulatory Commission's Generic Letter 88‐20, which required all operators to conduct a PRA. One possible mechanism for this is that the adoption of PRA may have increased learning rates, lowering the rate of recurring events by 42%. We find that operators that completed their PRA before Generic Letter 88‐20 continued to experience safety improvements during 1990–1995. This suggests that revisiting PRA or conducting it again can be beneficial. Our results suggest that even in a highly safety‐conscious industry as nuclear utilities, a more formal approach to quantifying risk has its benefits.     

Probabilistic Risk Assessment of Wastes Buried in the Ground

The differences between probabilistic risk assessment (PRA) and safety analysis (SA) are discussed, and it is shown that PRA is more suitable than SA for determining the acceptability of a technology. Since a PRA by the fault tree-event tree analysis method used for reactor safety studies does not seem to be practical for buried waste, an alternative approach is suggested using geochemical analogs. This method is illustrated for the cases of high-level and low-level radioactive waste and for chemical carcinogens released in coal burning.     

A Discourse on the Incorporation of Organizational Factors into Probabilistic Risk Assessment: Key Questions and Categorical Review

This article presents a discourse on the incorporation of organizational factors into probabilistic risk assessment (PRA)/probabilistic safety assessment (PSA), a topic of debate since the 1980s that has spurred discussions among industry, regulatory agencies, and the research community. The main contributions of this article include (1) identifying the four key open questions associated with this topic; (2) framing ongoing debates by considering differing perspectives around each question; (3) offering a categorical review of existing studies on this topic to justify the selection of each question and to analyze the challenges related to each perspective; and (4) highlighting the directions of research required to reach a final resolution for each question. The four key questions are: (I) How significant is the contribution of organizational factors to accidents and incidents? (II) How critical, with respect to improving risk assessment, is the explicit incorporation of organizational factors into PRA? (III) What theoretical bases are needed for explicit incorporation of organizational factors into PRA? (IV) What methodological bases are needed for the explicit incorporation of organizational factors into PRA? Questions I and II mainly analyze PRA literature from the nuclear domain. For Questions III and IV, a broader review and categorization is conducted of those existing cross-disciplinary studies that have evaluated the effects of organizational factors on safety (not solely PRA-based) to shed more light on future research needs.     

The Role of Risk and Future Land Use in Cleanup Decisions at the Department of Energy

As a result of the legacy of the Cold War, the Departments of Energy and Defense are involved in massive cleanup and remediation projects. While health risk to humans and ecological receptors is perceived to be the basis for remediation, this assumption is rarely examined. In this article, we examine the role of risk and future land-use designations in cleanup decisions, using the Department of Energy's self-assessment of 36 sites. We then discuss the risk-related tools that might be required to address the cleanup challenge. Much of the current cleanup program is driven by compliance with federal and state statutes and regulations, presumably to protect human health and the environment. Compliance, however, is not synonymous with cleanup. Although some of these laws and regulations take risk into account, the lack of site-specific data on exposures and risk scenarios, and the lack of attention to future land use or end states, has often resulted in disconnects between risk and cleanup goals, risk and final end states, and cleanup levels and end state or subsequent land use. Partly, these disconnects result from the need for a range of technical, economic, sociological, and public policy tools to address the issues. A better transfer of information among and within Department of Energy facilities, operations offices, and DOE headquarters is required. Further, linking cleanup decisions and goals with the final end state involves a number of risk tradeoffs, including (1) ecological versus human health, (2) worker versus public health, (3) among competing contaminated areas, (4) among temporal patterns of cleanup, (5) among different ecological receptors (plants vs. animals, one animal vs. another), and (6) among the sites across the DOE complex. For the nation, balancing among risks is essential within sites and among Department of Energy sites, as well as among other remediation sites (such as those of Department of Defense and Superfund sites).     

GIS‐Based Integration of Social Vulnerability and Level 3 Probabilistic Risk Assessment to Advance Emergency Preparedness,Planning, and Response for Severe Nuclear Power Plant Accidents

In the nuclear power industry, Level 3 probabilistic risk assessment (PRA) is used to estimate damage to public health and the environment if a severe accident leads to large radiological release. Current Level 3 PRA does not have an explicit inclusion of social factors and, therefore, it is not possible to perform importance ranking of social factors for risk‐informing emergency preparedness, planning, and response (EPPR). This article offers a methodology for adapting the concept of social vulnerability, commonly used in natural hazard research, in the context of a severe nuclear power plant accident. The methodology has four steps: (1) calculating a hazard‐independent social vulnerability index for the local population; (2) developing a location‐specific representation of the maximum radiological hazard estimated from current Level 3 PRA, in a geographic information system (GIS) environment; (3) developing a GIS‐based socio‐technical risk map by combining the social vulnerability index and the location‐specific radiological hazard; and (4) conducting a risk importance measure analysis to rank the criticality of social factors based on their contribution to the socio‐technical risk. The methodology is applied using results from the 2012 Surry Power Station state‐of‐the‐art reactor consequence analysis. A radiological hazard model is generated from MELCOR accident consequence code system, translated into a GIS environment, and combined with the Center for Disease Control social vulnerability index (SVI). This research creates an opportunity to explicitly consider and rank the criticality of location‐specific SVI themes based on their influence on risk, providing input for EPPR.     

Eliciting and Combining Decision Criteria Using a Limited Palette of Utility Functions and Uncertainty Distributions: Illustrated by Application to Pest Risk Analysis

Utility functions in the form of tables or matrices have often been used to combine discretely rated decision‐making criteria. Matrix elements are usually specified individually, so no one rule or principle can be easily stated for the utility function as a whole. A series of five matrices are presented that aggregate criteria two at a time using simple rules that express a varying degree of constraint of the lower rating over the higher. A further nine possible matrices were obtained by using a different rule either side of the main axis of the matrix to describe situations where the criteria have a differential influence on the outcome. Uncertainties in the criteria are represented by three alternative frequency distributions from which the assessors select the most appropriate. The output of the utility function is a distribution of rating frequencies that is dependent on the distributions of the input criteria. In pest risk analysis (PRA), seven of these utility functions were required to mimic the logic by which assessors for the European and Mediterranean Plant Protection Organization arrive at an overall rating of pest risk. The framework enables the development of PRAs that are consistent and easy to understand, criticize, compare, and change. When tested in workshops, PRA practitioners thought that the approach accorded with both the logic and the level of resolution that they used in the risk assessments.     

Probabilistic Risk Analysis for a High-Level Radioactive Waste Repository

A probabilistic risk analysis (PRA) for a high-level radioactive waste repository is very important since it gives an estimate of its health impacts, allowing comparisons to be made with the health impacts of competing technologies. However, it is extremely difficult to develop a credible PRA for a specific repository site because of large uncertainties in future climate, hydrology, geological processes, etc. At best, such a PRA would not be understandable to the public. An alternative proposed here is to develop a PRA for an average U.S. site, taking all properties of the site to be the U.S. average. The results are equivalent to the average results for numerous randomly selected sites. Such a PRA is presented here; it is easy to understand, and it is not susceptible to substantial uncertainty. Applying the results to a specific repository site then requires only a simple, intuitively acceptable "leap of faith" in assuming that with large expenditures of effort and money, experts can select a site that would be at least as secure as a randomly selected site.     

Probabilistic Risk Analyses: NRC Programs and Perspectives

The historical basis and more recent activities and products of probabilistic risk analysis (PRA) in the Atomic Energy Commission and Nuclear Regulatory Commission (NRC) are reviewed. Current NRC program activities and objectives are described. The author's opinions on the best uses of PRA are presented.     

Consideration of Background Exposures in the Management of Hazardous Waste Sites: A New Approach to Risk Assessment

The current approach to health risk assessment of toxic waste sites in the U.S. may lead to considerable expenditure of resources without any meaningful reduction in population exposure. Risk assessment methods used generally ignore background exposures and consider only incremental risk estimates for maximally exposed individuals. Such risk estimates do not address true public health risks to which background exposures also contribute. The purpose of this paper is to recommend a new approach to risk assessment and risk management concerning toxic waste sites. Under this new approach, which we have called public health risk assessment, chemical substances would be classified into a level of concern based on the potential health risks associated with typical national and regional background exposures. Site assessment would then be based on the level of concern for the particular pollutants involved and the potential contribution of site contaminants to typical background human exposures. While various problems can be foreseen with this approach, the key advantage is that resources would be allocated to reduce the most important sources of human exposure, and site remediation decisions could be simplified by focussing on exposure assessment rather than questionable risk extrapolations.     

Chemical and Radiation Environmental Risk Management: Differences, Commonalities, and Challenges

Driven by differing statutory mandates and programmatic separation of regulatory responsibilities between federal, state, and tribal agencies, distinct chemical and radiation risk management strategies have evolved. In the field this separation poses real challenges since many of the major environmental risk management decisions we face today require the evaluation of both types of risks. Over the last decade, federal, state, and tribal agencies have continued to discuss their different approaches and explore areas where their activities could be harmonized. The current framework for managing public exposures to chemical carcinogens has been referred to as a "bottom up approach." Risk between 10(-4) and 10(-6) is established as an upper bound goal. In contrast, a "top down" approach that sets an upper bound dose limit and couples with site specific As Low As Reasonably Achievable Principle (ALARA), is in place to manage individual exposure to radiation. While radiation risk are typically managed on a cumulative basis, exposure to chemicals is generally managed on a chemical-by-chemical, medium-by-medium basis. There are also differences in the nature and size of sites where chemical and radiation contamination is found. Such differences result in divergent management concerns. In spite of these differences, there are several common and practical concerns among radiation and chemical risk managers. They include 1) the issue of cost for site redevelopment and long-term stewardship, 2) public acceptance and involvement, and 3) the need for flexible risk management framework to address the first two issues. This article attempts to synthesize key differences, opportunities for harmonization, and challenges ahead.     

Scheduling Updates of Probabilistic Risk Assessments: The Arkansas Nuclear One-Unit 1 Experience

This paper presents the results of a study that identified how often a probabilistic risk assessment (PRA)should be updated to accommodate the changes that take place at nuclear power plants. Based on a 7-year analysis of design and procedural changes at one plant, we consider 5 years to be the maximum interval for updating PRAs. This conclusion is preliminary because it is based on the review of changes that occurred at a single plant, and it addresses only PRAs that involve a Level 1 analysis (i.e., a PRA including calculation of core damage frequency only). Nevertheless, this conclusion indicates that maintaining a useful PRA requires periodic updating efforts. However, the need for this periodic update stems only partly from the number of changes that can be expected to take place at nuclear power plants–changes that individually have only a moderate to minor impact on the PRA, but whose combined impact is substantial and necessitates a PRA update. Additionally, a comparison of two generations of PRAs performed about 5 years apart indicates that PRAs must be periodically updated to reflect the evolution of PRA methods. The most desirable updating interval depends on these two technical considerations as well as the cost of updating the PRA. (Cost considerations, however, were beyond the scope of this study.)     

Multicriteria Decision Analysis: A Comprehensive Decision Approach for Management of Contaminated Sediments

Contaminated sediments and other sites present a difficult challenge for environmental decisionmakers. They are typically slow to recover or attenuate naturally, may involve multiple regulatory agencies and stakeholder groups, and engender multiple toxicological and ecotoxicological risks. While environmental decision-making strategies over the last several decades have evolved into increasingly more sophisticated, information-intensive, and complex approaches, there remains considerable dissatisfaction among business, industry, and the public with existing management strategies. Consequently, contaminated sediments and materials are the subject of intense technology development, such as beneficial reuse or in situ treatment. However, current decision analysis approaches, such as comparative risk assessment, benefit-cost analysis, and life cycle assessment, do not offer a comprehensive approach for incorporating the varied types of information and multiple stakeholder and public views that must typically be brought to bear when new technologies are under consideration. Alternatively, multicriteria decision analysis (MCDA) offers a scientifically sound decision framework for management of contaminated materials or sites where stakeholder participation is of crucial concern and criteria such as economics, environmental impacts, safety, and risk cannot be easily condensed into simple monetary expressions. This article brings together a multidisciplinary review of existing decision-making approaches at regulatory agencies in the United States and Europe and synthesizes state-of-the-art research in MCDA methods applicable to the assessment of contaminated sediment management technologies. Additionally, it tests an MCDA approach for coupling expert judgment and stakeholder values in a hypothetical contaminated sediments management case study wherein MCDA is used as a tool for testing stakeholder responses to and improving expert assessment of innovative contaminated sediments technologies.     

Risk,Process Maturity,and Project Performance: An Empirical Analysis of US Federal Government Technology Projects

In recent years, there has been increasing pressure on the US federal government to reduce spending and improve the management of its technology projects. Mitigating the adverse impact of risks on the performance of these projects presents a significant challenge for its stakeholders. Our research examines this challenge in two steps. First, we identify and define a set of salient risks in federal technology projects—specifically, complexity risk and contracting risk in the planning process, and execution risk in the execution process. Next, we investigate whether higher levels of process maturity, assessed by the Capability Maturity Model Integration (CMMI) framework, mitigate the negative effect of project risks on project performance. The analysis of time‐series data collected from 82 federal technology projects across 519 quarterly time periods indicates that each of the three types of risks has a significant negative effect on project performance. This finding highlights the practical significance of managing these risks in the federal technology project context. Further, we find that increasing levels of process maturity attenuate the negative effect of project risks on the performance of federal technology projects. However, the attenuation effects are consequential only at high levels of project risks; at low levels of project risk, increasing levels of process maturity can adversely affect project performance. To demonstrate the financial implications of increasing process maturity levels in federal technology projects, we examine the magnitude of project cost savings (and overruns) across different levels of CMMI and project risks. In summary, our study contributes to the sparse literature on public sector operations by addressing the understudied context of federal technology projects, and provides a nuanced examination of the implications of process maturity in managing the risk to performance relationship in such projects.     

Ecological Risk Assessment,Prediction, and Assessing Risk Predictions

Ecological risk assessment embodied in an adaptive management framework is becoming the global standard approach for formally assessing and managing the ecological risks of technology and development. Ensuring the continual improvement of ecological risk assessment approaches is partly achieved through the dissemination of not only the types of risk assessment approaches used, but also their efficacy. While there is an increasing body of literature describing the results of general comparisons between alternate risk assessment methods and models, there is a paucity of literature that post hoc assesses the performance of specific predictions based on an assessment of risk and the effectiveness of the particular model used to predict the risk. This is especially the case where risk assessments have been used to grant consent or approval for the construction of major infrastructure projects. While postconstruction environmental monitoring is increasingly commonplace, it is not common for a postconstruction assessment of the accuracy and performance of the ecological risk assessment and underpinning model to be undertaken. Without this “assessment of the assessment,” it is difficult for other practitioners to gain insight into the performance of the approach and models used and therefore, as argued here, this limits the rate of improvement of risk assessment approaches.