Further Thoughts on the Utility of Risk Matrices

Risk matrices are commonly encountered devices for rating hazards in numerous areas of risk management. Part of their popularity is predicated on their apparent simplicity and transparency. Recent research, however, has identified serious mathematical defects and inconsistencies. This article further examines the reliability and utility of risk matrices for ranking hazards, specifically in the context of public leisure activities including travel. We find that (1) different risk assessors may assign vastly different ratings to the same hazard, (2) even following lengthy reflection and learning scatter remains high, and (3) the underlying drivers of disparate ratings relate to fundamentally different worldviews, beliefs, and a panoply of psychosocial factors that are seldom explicitly acknowledged. It appears that risk matrices when used in this context may be creating no more than an artificial and even untrustworthy picture of the relative importance of hazards, which may be of little or no benefit to those trying to manage risk effectively and rationally.     

How Probabilistic Risk Assessment Can Mislead Terrorism Risk Analysts

Traditional probabilistic risk assessment (PRA), of the type originally developed for engineered systems, is still proposed for terrorism risk analysis. We show that such PRA applications are unjustified in general. The capacity of terrorists to seek and use information and to actively research different attack options before deciding what to do raises unique features of terrorism risk assessment that are not adequately addressed by conventional PRA for natural and engineered systems—in part because decisions based on such PRA estimates do not adequately hedge against the different probabilities that attackers may eventually act upon. These probabilities may differ from the defender's (even if the defender's experts are thoroughly trained, well calibrated, unbiased probability assessors) because they may be conditioned on different information. We illustrate the fundamental differences between PRA and terrorism risk analysis, and suggest use of robust decision analysis for risk management when attackers may know more about some attack options than we do.     

Risk Matrix Integrating Risk Attitudes Based on Utility Theory

Recent studies indicate that absence of the consideration of risk attitudes of decisionmakers in the risk matrix establishment process has become a major limitation. In order to evaluate risk in a more comprehensive manner, an approach to establish risk matrices that integrates risk attitudes based on utility theory is proposed. There are three main steps within this approach: (1) describing risk attitudes of decisionmakers by utility functions, (2) bridging the gap between utility functions and the risk matrix by utility indifference curves, and (3) discretizing utility indifference curves. A complete risk matrix establishment process based on practical investigations is introduced. This process utilizes decisionmakers’ answers to questionnaires to formulate required boundary values for risk matrix establishment and utility functions that effectively quantify their respective risk attitudes.     

Principles for Conduct of Pest Risk Analyses: Report of an Expert Workshop

The North American Free Trade Agreement (NAFTA) and the General Agreement on Tariffs and Trade (GATT) have focused attention on risk assessment of potential insect, weed, and animal pests and diseases of livestock. These risks have traditionally been addressed through quarantine protocols ranging from limits on the geographical areas from which a product may originate, postharvest disinfestation procedures like fumigation, and inspections at points of export and import, to outright bans. To ensure that plant and animal protection measures are not used as nontariff trade barriers, GATT and NAFTA require pest risk analysis (PRA) to support quarantine decisions. The increased emphasis on PRA has spurred multiple efforts at the national and international level to design frameworks for the conduct of these analyses. As approaches to pest risk analysis proliferate, and the importance of the analyses grows, concerns have arisen about the scientific and technical conduct of pest risk analysis. In January of 1997, the Harvard Center for Risk Analysis (HCRA) held an invitation-only workshop in Washington, D.C. to bring experts in risk analysis and pest characterization together to develop general principles for pest risk analysis. Workshop participants examined current frameworks for PRA, discussed strengths and weaknesses of the approaches, and formulated principles, based on years of experience with risk analysis in other setting and knowledge of the issues specific to analysis of pests. The principles developed highlight the both the similarities of pest risk analysis to other forms of risk analysis, and its unique attributes.     

What's Wrong with Hazard‐Ranking Systems? An Expository Note

Two commonly recommended principles for allocating risk management resources to remediate uncertain hazards are: (1) select a subset to maximize risk-reduction benefits (e.g., maximize the von Neumann-Morgenstern expected utility of the selected risk-reducing activities), and (2) assign priorities to risk-reducing opportunities and then select activities from the top of the priority list down until no more can be afforded. When different activities create uncertain but correlated risk reductions, as is often the case in practice, then these principles are inconsistent: priority scoring and ranking fails to maximize risk-reduction benefits. Real-world risk priority scoring systems used in homeland security and terrorism risk assessment, environmental risk management, information system vulnerability rating, business risk matrices, and many other important applications do not exploit correlations among risk-reducing opportunities or optimally diversify risk-reducing investments. As a result, they generally make suboptimal risk management recommendations. Applying portfolio optimization methods instead of risk prioritization ranking, rating, or scoring methods can achieve greater risk-reduction value for resources spent.     

How to Design Rating Schemes of Risk Matrices: A Sequential Updating Approach

Risk matrices have been widely used as a risk evaluation tool in many fields due to their simplicity and intuitive nature. Designing a rating scheme, i.e., determining the number of ratings used in a risk matrix and assigning different ratings to different cells, is an essential part of risk matrix construction. However, most of the related literature has focused on applying a risk matrix to various fields, instead of researching how to design risk matrices. Based on the analysis of several current rules, we propose a new approach, namely, the sequential updating approach (SUA), to design the rating scheme of a risk matrix in a reliable way. In this article, we propose three principles and a rating algorithm based on these principles. The three principles, namely, adjusted weak consistency, consistent internality, and continuous screening, characterize a good rating scheme. The resulting rating scheme has been proven to be unique. A global rating algorithm is then proposed to create the design that satisfies the three principles. We then explore the performance of the SUA. An illustrative application is first given to explain the feasibility of our approach. The sensitivity analysis shows that our method captures a resolution‐reliability tradeoff for decisionmakers in choosing an appropriate rating scheme for a risk matrix. Finally, we compare the designs based on the SUA and Cox's axioms, highlighting the advantages of the SUA.     

Comparison of Scoring Systems for Invasive Pests Using ROC Analysis and Monte Carlo Simulations

Different international plant protection organisations advocate different schemes for conducting pest risk assessments. Most of these schemes use structured questionnaire in which experts are asked to score several items using an ordinal scale. The scores are then combined using a range of procedures, such as simple arithmetic mean, weighted averages, multiplication of scores, and cumulative sums. The most useful schemes will correctly identify harmful pests and identify ones that are not. As the quality of a pest risk assessment can depend on the characteristics of the scoring system used by the risk assessors (i.e., on the number of points of the scale and on the method used for combining the component scores), it is important to assess and compare the performance of different scoring systems. In this article, we proposed a new method for assessing scoring systems. Its principle is to simulate virtual data using a stochastic model and, then, to estimate sensitivity and specificity values from these data for different scoring systems. The interest of our approach was illustrated in a case study where several scoring systems were compared. Data for this analysis were generated using a probabilistic model describing the pest introduction process. The generated data were then used to simulate the outcome of scoring systems and to assess the accuracy of the decisions about positive and negative introduction. The results showed that ordinal scales with at most 5 or 6 points were sufficient and that the multiplication‐based scoring systems performed better than their sum‐based counterparts. The proposed method could be used in the future to assess a great diversity of scoring systems.     

A Nuclear Utility's Views on the Use of Probabilistic Risk Assessment

Probabilistic risk assessment (PRA) is a relatively new tool in the nuclear industry. The Reactor Safety Study started the present trend of conducting PRAs for nuclear power plants when it was published in 1975. Now, nine years later, those in the industry currently using PRA techniques are frequently asked the same question: Why should the nuclear utility industry, with so many accepted analytical tools already available, invest the time and manpower to develop a new technique with so many uncertainties?     

APPLICATIONS OF A RISK AVERSION CONCEPT

In decision theory the concept denoted variously as “risk aversion increment” or “risk premium” has not been fully exploited, although it is neither new nor complex. In this paper we will show how the concept of the risk aversion increment can be used for developing an alternative to the explicit use of the utility function. For most people the use of a risk aversion increment provides a better conceptual reference than does the use of a utility function. To illustrate the usefulness of the concept as a basis for gaining insight into problem statements and their analysis, the following applications are developed: 1) general results for the exponential utility function. 2) estimation of utility functions. 3) general results for various combinations of utility functions and probability distributions. 4) use in sequential decisions. 5) application in the theory of incentives.     

Multicriteria Decision Framework for Cybersecurity Risk Assessment and Management

Risk assessors and managers face many difficult challenges related to novel cyber systems. Among these challenges are the constantly changing nature of cyber systems caused by technical advances, their distribution across the physical, information, and sociocognitive domains, and the complex network structures often including thousands of nodes. Here, we review probabilistic and risk-based decision-making techniques applied to cyber systems and conclude that existing approaches typically do not address all components of the risk assessment triplet (threat, vulnerability, consequence) and lack the ability to integrate across multiple domains of cyber systems to provide guidance for enhancing cybersecurity. We present a decision-analysis-based approach that quantifies threat, vulnerability, and consequences through a set of criteria designed to assess the overall utility of cybersecurity management alternatives. The proposed framework bridges the gap between risk assessment and risk management, allowing an analyst to ensure a structured and transparent process of selecting risk management alternatives. The use of this technique is illustrated for a hypothetical, but realistic, case study exemplifying the process of evaluating and ranking five cybersecurity enhancement strategies. The approach presented does not necessarily eliminate biases and subjectivity necessary for selecting countermeasures, but provides justifiable methods for selecting risk management actions consistent with stakeholder and decisionmaker values and technical data.     

Integrating Software into PRA: A Test-Based Approach

Probabilistic risk assessment (PRA) is a methodology to assess the probability of failure or success of a system's operation. PRA has been proved to be a systematic, logical, and comprehensive technique for risk assessment. Software plays an increasing role in modern safety critical systems. A significant number of failures can be attributed to software failures. Unfortunately, current probabilistic risk assessment concentrates on representing the behavior of hardware systems, humans, and their contributions (to a limited extent) to risk but neglects the contributions of software due to a lack of understanding of software failure phenomena. It is thus imperative to consider and model the impact of software to reflect the risk in current and future systems. The objective of our research is to develop a methodology to account for the impact of software on system failure that can be used in the classical PRA analysis process. A test-based approach for integrating software into PRA is discussed in this article. This approach includes identification of software functions to be modeled in the PRA, modeling of the software contributions in the ESD, and fault tree. The approach also introduces the concepts of input tree and output tree and proposes a quantification strategy that uses a software safety testing technique. The method is applied to an example system, PACS.     

Evaluating Methods for Estimating Existential Risks

Researchers and commissions contend that the risk of human extinction is high, but none of these estimates have been based upon a rigorous methodology suitable for estimating existential risks. This article evaluates several methods that could be used to estimate the probability of human extinction. Traditional methods evaluated include: simple elicitation; whole evidence Bayesian; evidential reasoning using imprecise probabilities; and Bayesian networks. Three innovative methods are also considered: influence modeling based on environmental scans; simple elicitation using extinction scenarios as anchors; and computationally intensive possible‐worlds modeling. Evaluation criteria include: level of effort required by the probability assessors; level of effort needed to implement the method; ability of each method to model the human extinction event; ability to incorporate scientific estimates of contributory events; transparency of the inputs and outputs; acceptability to the academic community (e.g., with respect to intellectual soundness, familiarity, verisimilitude); credibility and utility of the outputs of the method to the policy community; difficulty of communicating the method's processes and outputs to nonexperts; and accuracy in other contexts. The article concludes by recommending that researchers assess the risks of human extinction by combining these methods.     

考虑风险因素的客户评价与选择研究

应用效用理论考虑客户终生价值的不确定性,根据客户的期望效用对客户进行评价,权衡价值与风险来优化选择客户组合.通过蒙特卡罗仿真方法得到客户终生价值分布样本,并以此计算期望效用;通过实际案例,揭示期望价值相近的客户可能表现出风险的巨大差异.对比期望价值和期望效用方法所得到的前30名客户组合,发现采用风险规避型期望效用方法得到的客户资产期望价值虽然有所减少,但风险大大降低.通过这种方法,管理者能体现自己对客户终生价值分布风险的偏好,根据当前的市场战略选择重点客户进行投资.     

Probabilistic Risk Assessment of Wastes Buried in the Ground

The differences between probabilistic risk assessment (PRA) and safety analysis (SA) are discussed, and it is shown that PRA is more suitable than SA for determining the acceptability of a technology. Since a PRA by the fault tree-event tree analysis method used for reactor safety studies does not seem to be practical for buried waste, an alternative approach is suggested using geochemical analogs. This method is illustrated for the cases of high-level and low-level radioactive waste and for chemical carcinogens released in coal burning.     

Quantifying Water Pathogen Risk in an Epidemiological Framework

Traditionally, microbial risk assessors have used point estimates to evaluate the probability that an individual will become infected. We developed a quantitative approach that shifts the risk characterization perspective from point estimate to distributional estimate, and from individual to population. To this end, we first designed and implemented a dynamic model that tracks traditional epidemiological variables such as the number of susceptible, infected, diseased, and immune, and environmental variables such as pathogen density. Second, we used a simulation methodology that explicitly acknowledges the uncertainty and variability associated with the data. Specifically, the approach consists of assigning probability distributions to each parameter, sampling from these distributions for Monte Carlo simulations, and using a binary classification to assess the output of each simulation. A case study is presented that explores the uncertainties in assessing the risk of giardiasis when swimming in a recreational impoundment using reclaimed water. Using literature-based information to assign parameters ranges, our analysis demonstrated that the parameter describing the shedding of pathogens by infected swimmers was the factor that contributed most to the uncertainty in risk. The importance of other parameters was dependent on reducing the a priori range of this shedding parameter. By constraining the shedding parameter to its lower subrange, treatment efficiency was the parameter most important in predicting whether a simulation resulted in prevalences above or below non outbreak levels. Whereas parameters associated with human exposure were important when the shedding parameter was constrained to a higher subrange. This Monte Carlo simulation technique identified conditions in which outbreaks and/or nonoutbreaks are likely and identified the parameters that most contributed to the uncertainty associated with a risk prediction.     

Harnessing the Theoretical Foundations of the Exponential and Beta‐Poisson Dose‐Response Models to Quantify Parameter Uncertainty Using Markov Chain Monte Carlo

Dose‐response models are the essential link between exposure assessment and computed risk values in quantitative microbial risk assessment, yet the uncertainty that is inherent to computed risks because the dose‐response model parameters are estimated using limited epidemiological data is rarely quantified. Second‐order risk characterization approaches incorporating uncertainty in dose‐response model parameters can provide more complete information to decisionmakers by separating variability and uncertainty to quantify the uncertainty in computed risks. Therefore, the objective of this work is to develop procedures to sample from posterior distributions describing uncertainty in the parameters of exponential and beta‐Poisson dose‐response models using Bayes's theorem and Markov Chain Monte Carlo (in OpenBUGS). The theoretical origins of the beta‐Poisson dose‐response model are used to identify a decomposed version of the model that enables Bayesian analysis without the need to evaluate Kummer confluent hypergeometric functions. Herein, it is also established that the beta distribution in the beta‐Poisson dose‐response model cannot address variation among individual pathogens, criteria to validate use of the conventional approximation to the beta‐Poisson model are proposed, and simple algorithms to evaluate actual beta‐Poisson probabilities of infection are investigated. The developed MCMC procedures are applied to analysis of a case study data set, and it is demonstrated that an important region of the posterior distribution of the beta‐Poisson dose‐response model parameters is attributable to the absence of low‐dose data. This region includes beta‐Poisson models for which the conventional approximation is especially invalid and in which many beta distributions have an extreme shape with questionable plausibility.     

Establishment Risks for Invasive Species

This article presents a quantitative methodology for evaluating the probability of invasive pest species establishing persistent populations. The estimation of pest establishment relies on data and information describing the biology and ecology of the pest and its interactions with potential host species and the regional environment. This information is developed using a model construct borrowed from theoretical population ecology. The methodology for estimating the probability of pest establishment is part of an overall framework that explores the implications of reductions in pest invasions on subsequent establishment. The risk reduction framework integrates the engineering aspects of different technologies for reducing pest entry, the biology and ecology of pest species, the suitability of potentially susceptible hosts, and the quality of available habitats. The methodology for estimating the risk of establishment is presented using an example pest, the Asian longhorned beetle ( Anoplophora glabripennis ), which has been introduced into the United States via solid wood packing materials (SWPM) used in international commerce. Uncertainties inherent to the estimation of model parameters that determine the risk of establishment are defined, quantified, and propagated through the population model. Advantages and limitations of the proposed methodology are discussed along with recommendations to make the approach more useful in the management of risks posed by the establishment of pest populations.     

Recommendations on the Testing and Use of Pseudo-Random Number Generators Used in Monte Carlo Analysis for Risk Assessment

Monte Carlo simulation requires a pseudo-random number generator with good statistical properties. Linear congruential generators (LCGs) are the most popular and well-studied computer method for generating pseudo-random numbers used in Monte Carlo studies. High quality LCGs are available with sufficient statistical quality to satisfy all but the most demanding needs of risk assessors. However, because of the discrete, deterministic nature of LCGs, it is important to evaluate the randomness and uniformity of the specific pseudo-random number subsequences used in important risk assessments. Recommended statistical tests for uniformity and randomness include the Kolmogorov-Smirnov test, extreme values test, and the runs test, including runs above and runs below the mean tests. Risk assessors should evaluate the stability of their risk model's output statistics, paying particular attention to instabilities in the mean and variance. When instabilities in the mean and variance are observed, more stable statistics, e.g., percentiles, should be reported. Analyses should be repeated using several non-overlapping pseudo-random number subsequences. More simulations than those traditionally used are also recommended for each analysis.     

Some Limitations of Qualitative Risk Rating Systems

Qualitative systems for rating animal antimicrobial risks using ordered categorical labels such as “high,”“medium,” and “low” can potentially simplify risk assessment input requirements used to inform risk management decisions. But do they improve decisions? This article compares the results of qualitative and quantitative risk assessment systems and establishes some theoretical limitations on the extent to which they are compatible. In general, qualitative risk rating systems satisfying conditions found in real‐world rating systems and guidance documents and proposed as reasonable make two types of errors: (1) Reversed rankings, i.e., assigning higher qualitative risk ratings to situations that have lower quantitative risks; and (2) Uninformative ratings, e.g., frequently assigning the most severe qualitative risk label (such as “high”) to situations with arbitrarily small quantitative risks and assigning the same ratings to risks that differ by many orders of magnitude. Therefore, despite their appealing consensus‐building properties, flexibility, and appearance of thoughtful process in input requirements, qualitative rating systems as currently proposed often do not provide sufficient information to discriminate accurately between quantitatively small and quantitatively large risks. The value of information (VOI) that they provide for improving risk management decisions can be zero if most risks are small but a few are large, since qualitative ratings may then be unable to confidently distinguish the large risks from the small. These limitations suggest that it is important to continue to develop and apply practical quantitative risk assessment methods, since qualitative ones are often unreliable.