Assessing Uncertainty in Simulation-Based Maritime Risk Assessment

Recent work in the assessment of risk in maritime transportation systems has used simulation-based probabilistic risk assessment techniques. In the Prince William Sound and Washington State Ferries risk assessments, the studies' recommendations were backed up by estimates of their impact made using such techniques and all recommendations were implemented. However, the level of uncertainty about these estimates was not available, leaving the decisionmakers unsure whether the evidence was sufficient to assess specific risks and benefits. The first step toward assessing the impact of uncertainty in maritime risk assessments is to model the uncertainty in the simulation models used. In this article, a study of the impact of proposed ferry service expansions in San Francisco Bay is used as a case study to demonstrate the use of Bayesian simulation techniques to propagate uncertainty throughout the analysis. The conclusions drawn in the original study are shown, in this case, to be robust to the inherent uncertainties. The main intellectual merit of this work is the development of Bayesian simulation technique to model uncertainty in the assessment of maritime risk. However, Bayesian simulations have been implemented only as theoretical demonstrations. Their use in a large, complex system may be considered state of the art in the field of computational sciences.     

Maritime Transportation Risk Assessment of Tianjin Port with Bayesian Belief Networks

This article develops a Bayesian belief network model for the prediction of accident consequences in the Tianjin port. The study starts with a statistical analysis of historical accident data of six years from 2008 to 2013. Then a Bayesian belief network is constructed to express the dependencies between the indicator variables and accident consequences. The statistics and expert knowledge are synthesized in the Bayesian belief network model to obtain the probability distribution of the consequences. By a sensitivity analysis, several indicator variables that have influence on the consequences are identified, including navigational area, ship type and time of the day. The results indicate that the consequences are most sensitive to the position where the accidents occurred, followed by time of day and ship length. The results also reflect that the navigational risk of the Tianjin port is at the acceptable level, despite that there is more room of improvement. These results can be used by the Maritime Safety Administration to take effective measures to enhance maritime safety in the Tianjin port.     

Risk-Based Ranking of Dominant Contributors to Maritime Pollution Events

This report describes a conceptual approach for identifying dominant contributors to risk from maritime shipping of hazardous materials. Maritime transportation accidents are relatively common occurrences compared to more frequently analyzed contributors to public risk. Yet research on maritime safety and pollution incidents has not been guided by a systematic, risk-based approach. Maritime shipping accidents can be analyzed using event trees to group the accidents into "bins," or groups, of similar characteristics such as type of cargo, location of accident (e.g., harbor, inland waterway), type of accident (e.g., fire, collision, grounding), and size of release. The importance of specific types of events to each accident bin can be quantified. Then the overall importance of accident events to risk can be estimated by weighting the events' individual bin importance measures by the risk associated with each accident bin.     

Exploring the Uncertainties in Cancer Risk Assessment Using the Integrated Probabilistic Risk Assessment (IPRA) Approach

Current methods for cancer risk assessment result in single values, without any quantitative information on the uncertainties in these values. Therefore, single risk values could easily be overinterpreted. In this study, we discuss a full probabilistic cancer risk assessment approach in which all the generally recognized uncertainties in both exposure and hazard assessment are quantitatively characterized and probabilistically evaluated, resulting in a confidence interval for the final risk estimate. The methodology is applied to three example chemicals (aflatoxin, N‐nitrosodimethylamine, and methyleugenol). These examples illustrate that the uncertainty in a cancer risk estimate may be huge, making single value estimates of cancer risk meaningless. Further, a risk based on linear extrapolation tends to be lower than the upper 95% confidence limit of a probabilistic risk estimate, and in that sense it is not conservative. Our conceptual analysis showed that there are two possible basic approaches for cancer risk assessment, depending on the interpretation of the dose‐incidence data measured in animals. However, it remains unclear which of the two interpretations is the more adequate one, adding an additional uncertainty to the already huge confidence intervals for cancer risk estimates.     

Use of Advances in Technology for Maritime Risk Assessment

The maritime industry is moving toward a "goal-setting" risk-based regime. This opens the way to safety engineers to explore and exploit flexible and advanced risk modeling and decision-making approaches in the design and operation processes. In this article, following a brief review of the current status of maritime risk assessment, a design/operation selection framework and a design/operation optimization framework are outlined. A general discussion of control engineering techniques and their application to risk modeling and decision making is given. Four novel risk modeling and decision-making approaches are then outlined with illustrative examples to demonstrate their use. Such approaches may be used as alternatives to facilitate risk modeling and decision making in situations where conventional techniques cannot be appropriately applied. Finally, recommendations on further exploitation of advances in general engineering and technology are suggested with respect to risk modeling and decision making.     

Quantifying Uncertainty in a Risk Assessment Using Human Data

A call for risk assessment approaches that better characterize and quantify uncertainty has been made by the scientific and regulatory community. This paper responds to that call by demonstrating a distributional approach that draws upon human data to derive potency estimates and to identify and quantify important sources of uncertainty. The approach is rooted in the science of decision analysis and employs an influence diagram, a decision tree, probabilistic weights, and a distribution of point estimates of carcinogenic potency. Its results estimate the likelihood of different carcinogenic risks (potencies) for a chemical under a specific scenario. For this exercise, human data on formaldehyde were employed to demonstrate the approach. Sensitivity analyses were performed to determine the relative impact of specific levels and alternatives on the potency distribution. The resulting potency estimates are compared with the results of an exercise using animal data on formaldehyde. The paper demonstrates that distributional risk assessment is readily adapted to situations in which epidemiologic data serve as the basis for potency estimates. Strengths and weaknesses of the distributional approach are discussed. Areas for further application and research are recommended.     

Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment

The US Department of Transportation was interested in the risks associated with transporting Hydrazine in tanks with and without relief devices. Hydrazine is both highly toxic and flammable, as well as corrosive. Consequently, there was a conflict as to whether a relief device should be used or not. Data were not available on the impact of relief devices on release probabilities or the impact of Hydrazine on the likelihood of fires and explosions. In this paper, a Monte Carlo sensitivity analysis of the unknown parameters was used to assess the risks associated with highway transport of Hydrazine. To help determine whether or not relief devices should be used, fault trees and event trees were used to model the sequences of events that could lead to adverse consequences during transport of Hydrazine. The event probabilities in the event trees were derived as functions of the parameters whose effects were not known. The impacts of these parameters on the risk of toxic exposures, fires, and explosions were analyzed through a Monte Carlo sensitivity analysis and analyzed statistically through an analysis of variance. The analysis allowed the determination of which of the unknown parameters had a significant impact on the risks. It also provided the necessary support to a critical transportation decision even though the values of several key parameters were not known.     

Uncertainty Assessment in the Quantification of Risk Rates of Occupational Accidents

Occupational risk rates per hour of exposure have been quantified for 63 occupational accident types for the Dutch working population. Data were obtained from the analysis of more than 9,000 accidents that occurred over a period of six years in the Netherlands and resulted in three types of reportable consequences under Dutch law: (a) fatal injury, (b) permanent injury, and (c) serious recoverable injury requiring at least one day of hospitalization. A Bayesian uncertainty assessment on the value of the risk rates has been performed. Annual risks for each of the 63 occupational accident types have been calculated, including the variability in the annual exposure of the working population to the corresponding hazards. The suitability of three risk measures—individual risk rates, individual annual risk, and number of accidents—is examined and discussed.     

Integrated Uncertainty Analysis for Ambient Pollutant Health Risk Assessment: A Case Study of Ozone Mortality Risk

The U.S. Environmental Protection Agency (EPA) uses health risk assessment to help inform its decisions in setting national ambient air quality standards (NAAQS). EPA's standard approach is to make epidemiologically‐based risk estimates based on a single statistical model selected from the scientific literature, called the “core” model. The uncertainty presented for “core” risk estimates reflects only the statistical uncertainty associated with that one model's concentration‐response function parameter estimate(s). However, epidemiologically‐based risk estimates are also subject to “model uncertainty,” which is a lack of knowledge about which of many plausible model specifications and data sets best reflects the true relationship between health and ambient pollutant concentrations. In 2002, a National Academies of Sciences (NAS) committee recommended that model uncertainty be integrated into EPA's standard risk analysis approach. This article discusses how model uncertainty can be taken into account with an integrated uncertainty analysis (IUA) of health risk estimates. It provides an illustrative numerical example based on risk of premature death from respiratory mortality due to long‐term exposures to ambient ozone, which is a health risk considered in the 2015 ozone NAAQS decision. This example demonstrates that use of IUA to quantitatively incorporate key model uncertainties into risk estimates produces a substantially altered understanding of the potential public health gain of a NAAQS policy decision, and that IUA can also produce more helpful insights to guide that decision, such as evidence of decreasing incremental health gains from progressive tightening of a NAAQS.     

Uncertainty and Sensitivity Analysis in Assessment of the Thyroid Cancer Risk Related to Chernobyl Fallout in Eastern France

The increase in the thyroid cancer incidence in France observed over the last 20 years has raised public concern about its association with the 1986 nuclear power plant accident at Chernobyl. At the request of French authorities, a first study sought to quantify the possible risk of thyroid cancer associated with the Chernobyl fallout in France. This study suffered from two limitations. The first involved the lack of knowledge of spontaneous thyroid cancer incidence rates (in the absence of exposure), which was especially necessary to take their trends into account for projections over time; the second was the failure to consider the uncertainties. The aim of this article is to enhance the initial thyroid cancer risk assessment for the period 1991-2007 in the area of France most exposed to the fallout (i.e., eastern France) and thereby mitigate these limitations. We consider the changes over time in the incidence of spontaneous thyroid cancer and conduct both uncertainty and sensitivity analyses. The number of spontaneous thyroid cancers was estimated from French cancer registries on the basis of two scenarios: one with a constant incidence, the other using the trend observed. Thyroid doses were estimated from all available data about contamination in France from Chernobyl fallout. Results from a 1995 pooled analysis published by Ron et al. were used to determine the dose-response relation. Depending on the scenario, the number of spontaneous thyroid cancer cases ranges from 894 (90% CI: 869-920) to 1,716 (90% CI: 1,691-1,741). The number of excess thyroid cancer cases predicted ranges from 5 (90% UI: 1-15) to 63 (90% UI: 12-180). All of the assumptions underlying the thyroid cancer risk assessment are discussed.     

Presenting Uncertainty in Health Risk Assessment: Initial Studies of Its Effects on Risk Perception and Trust

Some analysts suggest that discussing uncertainties in health risk assessments might reduce citizens'perceptions of risk and increase their respect for the risk-assessing agency. We tested this assumption with simulated news stories varying simple displays of uncertainty (e.g., a range of risk estimates, with and without graphics). Subjects from Eugene, Oregon, read one story each, and then answered a questionnaire. Three studies tested between 180 and 272 subjects each. Two focus groups obtained more detailed responses to these stories. The results suggested that (1) people are unfamiliar with uncertainty in risk assessments and in science; (2) people may recognize uncertainty when it is presented simply; (3) graphics may help people recognize uncertainty; (4) reactions to the environmental problems in the stories seemed affected less by presentation of uncertainty than by general risk attitudes and perceptions; (5) agency discussion of uncertainty in risk estimates may signal agency honesty and agency incompetence for some people; and (6) people seem to see lower risk estimates (10^-6, as opposed to 10^-3) as less credible. These findings, if confirmed, would have important implications for risk communication.     

A Framework for Understanding Uncertainty in Seismic Risk Assessment

A better understanding of the uncertainty that exists in models used for seismic risk assessment is critical to improving risk-based decisions pertaining to earthquake safety. Current models estimating the probability of collapse of a building do not consider comprehensively the nature and impact of uncertainty. This article presents a model framework to enhance seismic risk assessment and thus gives decisionmakers a fuller understanding of the nature and limitations of the estimates. This can help ensure that risks are not over- or underestimated and the value of acquiring accurate data is appreciated fully. The methodology presented provides a novel treatment of uncertainties in input variables, their propagation through the model, and their effect on the results. The study presents ranges of possible annual collapse probabilities for different case studies on buildings in different parts of the world, exposed to different levels of seismicity, and with different vulnerabilities. A global sensitivity analysis was conducted to determine the significance of uncertain variables. Two key outcomes are (1) that the uncertainty in ground-motion conversion equations has the largest effect on the uncertainty in the calculation of annual collapse probability; and (2) the vulnerability of a building appears to have an effect on the range of annual collapse probabilities produced, i.e., the level of uncertainty in the estimate of annual collapse probability, with less vulnerable buildings having a smaller uncertainty.     

Quantitative Uncertainty Analysis for a Weed Risk Assessment System

Weed risk assessments (WRA) are used to identify plant invaders before introduction. Unfortunately, very few incorporate uncertainty ratings or evaluate the effects of uncertainty, a fundamental risk component. We developed a probabilistic model to quantitatively evaluate the effects of uncertainty on the outcomes of a question‐based WRA tool for the United States. In our tool, the uncertainty of each response is rated as Negligible, Low, Moderate, or High. We developed the model by specifying the likelihood of a response changing for each uncertainty rating. The simulations determine if responses change, select new responses, and sum the scores to determine the risk rating. The simulated scores reveal potential variation in WRA risk ratings. In testing with 204 species assessments, the ranges of simulated risk scores increased with greater uncertainty, and analyses for most species produced simulated risk ratings that differed from the baseline WRA rating. Still, the most frequent simulated rating matched the baseline rating for every High Risk species, and for 87% of all tested species. The remaining 13% primarily involved ambiguous Low Risk results. Changing final ratings based on the uncertainty analysis results was not justified here because accuracy (match between WRA tool and known risk rating) did not improve. Detailed analyses of three species assessments indicate that assessment uncertainty may be best reduced by obtaining evidence for unanswered questions, rather than obtaining additional evidence for questions with responses. This analysis represents an advance in interpreting WRA results, and has enhanced our regulation and management of potential weed species.     

Science and Decisions: Advancing Risk Assessment

At the request of the U.S. Environmental Protection Agency (EPA), the National Research Council (NRC) recently completed a major report, Science and Decisions: Advancing Risk Assessment, that is intended to strengthen the scientific basis, credibility, and effectiveness of risk assessment practices and subsequent risk management decisions. The report describes the challenges faced by risk assessment and the need to consider improvements in both the technical analyses of risk assessments (i.e., the development and use of scientific information to improve risk characterization) and the utility of risk assessments (i.e., making assessments more relevant and useful for risk management decisions). The report tackles a number of topics relating to improvements in the process, including the design and framing of risk assessments, uncertainty and variability characterization, selection and use of defaults, unification of cancer and noncancer dose‐response assessment, cumulative risk assessment, and the need to increase EPA's capacity to address these improvements. This article describes and summarizes the NRC report, with an eye toward its implications for risk assessment practices at EPA.     

Sensitivity Analysis of a Two‐Dimensional Quantitative Microbiological Risk Assessment: Keeping Variability and Uncertainty Separated

The aim of quantitative microbiological risk assessment is to estimate the risk of illness caused by the presence of a pathogen in a food type, and to study the impact of interventions. Because of inherent variability and uncertainty, risk assessments are generally conducted stochastically, and if possible it is advised to characterize variability separately from uncertainty. Sensitivity analysis allows to indicate to which of the input variables the outcome of a quantitative microbiological risk assessment is most sensitive. Although a number of methods exist to apply sensitivity analysis to a risk assessment with probabilistic input variables (such as contamination, storage temperature, storage duration, etc.), it is challenging to perform sensitivity analysis in the case where a risk assessment includes a separate characterization of variability and uncertainty of input variables. A procedure is proposed that focuses on the relation between risk estimates obtained by Monte Carlo simulation and the location of pseudo‐randomly sampled input variables within the uncertainty and variability distributions. Within this procedure, two methods are used—that is, an ANOVA‐like model and Sobol sensitivity indices—to obtain and compare the impact of variability and of uncertainty of all input variables, and of model uncertainty and scenario uncertainty. As a case study, this methodology is applied to a risk assessment to estimate the risk of contracting listeriosis due to consumption of deli meats.     

How Probabilistic Risk Assessment Can Mislead Terrorism Risk Analysts

Traditional probabilistic risk assessment (PRA), of the type originally developed for engineered systems, is still proposed for terrorism risk analysis. We show that such PRA applications are unjustified in general. The capacity of terrorists to seek and use information and to actively research different attack options before deciding what to do raises unique features of terrorism risk assessment that are not adequately addressed by conventional PRA for natural and engineered systems—in part because decisions based on such PRA estimates do not adequately hedge against the different probabilities that attackers may eventually act upon. These probabilities may differ from the defender's (even if the defender's experts are thoroughly trained, well calibrated, unbiased probability assessors) because they may be conditioned on different information. We illustrate the fundamental differences between PRA and terrorism risk analysis, and suggest use of robust decision analysis for risk management when attackers may know more about some attack options than we do.     

Sensitivity Analysis Using Risk Measures

In a quantitative model with uncertain inputs, the uncertainty of the output can be summarized by a risk measure. We propose a sensitivity analysis method based on derivatives of the output risk measure, in the direction of model inputs. This produces a global sensitivity measure, explicitly linking sensitivity and uncertainty analyses. We focus on the case of distortion risk measures, defined as weighted averages of output percentiles, and prove a representation of the sensitivity measure that can be evaluated on a Monte Carlo sample, as a weighted average of gradients over the input space. When the analytical model is unknown or hard to work with, nonparametric techniques are used for gradient estimation. This process is demonstrated through the example of a nonlinear insurance loss model. Furthermore, the proposed framework is extended in order to measure sensitivity to constant model parameters, uncertain statistical parameters, and random factors driving dependence between model inputs.     

Updating Uncertainty in an Integrated Risk Assessment: Conceptual Framework and Methods

Bayesian methods are presented for updating the uncertainty in the predictions of an integrated Environmental Health Risk Assessment (EHRA) model. The methods allow the estimation of posterior uncertainty distributions based on the observation of different model outputs along the chain of the linked assessment framework. Analytical equations are derived for the case of the multiplicative lognormal risk model where the sequential log outputs (log ambient concentration, log applied dose, log delivered dose, and log risk) are each normally distributed. Given observations of a log output made with a normally distributed measurement error, the posterior distributions of the log outputs remain normal, but with modified means and variances, and induced correlations between successive log outputs and log inputs. The analytical equations for forward and backward propagation of the updates are generally applicable to sums of normally distributed variables. The Bayesian Monte-Carlo (BMC) procedure is presented to provide an approximate, but more broadly applicable method for numerically updating uncertainty with concurrent backward and forward propagation. Illustrative examples, presented for the multiplicative lognormal model, demonstrate agreement between the analytical and BMC methods, and show how uncertainty updates can propagate through a linked EHRA. The Bayesian updating methods facilitate the pooling of knowledge encoded in predictive models with that transmitted by research outcomes (e.g., field measurements), and thereby support the practice of iterative risk assessment and value of information appraisals.     

A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment

The life cycle assessment (LCA) framework has established itself as the leading tool for the assessment of the environmental impact of products. Several works have established the need of integrating the LCA and risk analysis methodologies, due to the several common aspects. One of the ways to reach such integration is through guaranteeing that uncertainties in LCA modeling are carefully treated. It has been claimed that more attention should be paid to quantifying the uncertainties present in the various phases of LCA. Though the topic has been attracting increasing attention of practitioners and experts in LCA, there is still a lack of understanding and a limited use of the available statistical tools. In this work, we introduce a protocol to conduct global sensitivity analysis in LCA. The article focuses on the life cycle impact assessment (LCIA), and particularly on the relevance of global techniques for the development of trustable impact assessment models. We use a novel characterization model developed for the quantification of the impacts of noise on humans as a test case. We show that global SA is fundamental to guarantee that the modeler has a complete understanding of: (i) the structure of the model and (ii) the importance of uncertain model inputs and the interaction among them.     

Enhancing the Characterization of Epistemic Uncertainties in PM2.5 Risk Analyses

The Environmental Benefits Mapping and Analysis Program (BenMAP) is a software tool developed by the U.S. Environmental Protection Agency (EPA) that is widely used inside and outside of EPA to produce quantitative estimates of public health risks from fine particulate matter (PM_2.5). This article discusses the purpose and appropriate role of a risk analysis tool to support risk management deliberations, and evaluates the functions of BenMAP in this context. It highlights the importance in quantitative risk analyses of characterization of epistemic uncertainty, or outright lack of knowledge, about the true risk relationships being quantified. This article describes and quantitatively illustrates sensitivities of PM_2.5 risk estimates to several key forms of epistemic uncertainty that pervade those calculations: the risk coefficient, shape of the risk function, and the relative toxicity of individual PM_2.5 constituents. It also summarizes findings from a review of U.S.‐based epidemiological evidence regarding the PM_2.5 risk coefficient for mortality from long‐term exposure. That review shows that the set of risk coefficients embedded in BenMAP substantially understates the range in the literature. We conclude that BenMAP would more usefully fulfill its role as a risk analysis support tool if its functions were extended to better enable and prompt its users to characterize the epistemic uncertainties in their risk calculations. This requires expanded automatic sensitivity analysis functions and more recognition of the full range of uncertainty in risk coefficients.