Eliciting and Combining Decision Criteria Using a Limited Palette of Utility Functions and Uncertainty Distributions: Illustrated by Application to Pest Risk Analysis

Utility functions in the form of tables or matrices have often been used to combine discretely rated decision‐making criteria. Matrix elements are usually specified individually, so no one rule or principle can be easily stated for the utility function as a whole. A series of five matrices are presented that aggregate criteria two at a time using simple rules that express a varying degree of constraint of the lower rating over the higher. A further nine possible matrices were obtained by using a different rule either side of the main axis of the matrix to describe situations where the criteria have a differential influence on the outcome. Uncertainties in the criteria are represented by three alternative frequency distributions from which the assessors select the most appropriate. The output of the utility function is a distribution of rating frequencies that is dependent on the distributions of the input criteria. In pest risk analysis (PRA), seven of these utility functions were required to mimic the logic by which assessors for the European and Mediterranean Plant Protection Organization arrive at an overall rating of pest risk. The framework enables the development of PRAs that are consistent and easy to understand, criticize, compare, and change. When tested in workshops, PRA practitioners thought that the approach accorded with both the logic and the level of resolution that they used in the risk assessments.     

Quantitative Uncertainty Analysis for a Weed Risk Assessment System

Weed risk assessments (WRA) are used to identify plant invaders before introduction. Unfortunately, very few incorporate uncertainty ratings or evaluate the effects of uncertainty, a fundamental risk component. We developed a probabilistic model to quantitatively evaluate the effects of uncertainty on the outcomes of a question‐based WRA tool for the United States. In our tool, the uncertainty of each response is rated as Negligible, Low, Moderate, or High. We developed the model by specifying the likelihood of a response changing for each uncertainty rating. The simulations determine if responses change, select new responses, and sum the scores to determine the risk rating. The simulated scores reveal potential variation in WRA risk ratings. In testing with 204 species assessments, the ranges of simulated risk scores increased with greater uncertainty, and analyses for most species produced simulated risk ratings that differed from the baseline WRA rating. Still, the most frequent simulated rating matched the baseline rating for every High Risk species, and for 87% of all tested species. The remaining 13% primarily involved ambiguous Low Risk results. Changing final ratings based on the uncertainty analysis results was not justified here because accuracy (match between WRA tool and known risk rating) did not improve. Detailed analyses of three species assessments indicate that assessment uncertainty may be best reduced by obtaining evidence for unanswered questions, rather than obtaining additional evidence for questions with responses. This analysis represents an advance in interpreting WRA results, and has enhanced our regulation and management of potential weed species.     

What's Wrong with Hazard‐Ranking Systems? An Expository Note

Two commonly recommended principles for allocating risk management resources to remediate uncertain hazards are: (1) select a subset to maximize risk-reduction benefits (e.g., maximize the von Neumann-Morgenstern expected utility of the selected risk-reducing activities), and (2) assign priorities to risk-reducing opportunities and then select activities from the top of the priority list down until no more can be afforded. When different activities create uncertain but correlated risk reductions, as is often the case in practice, then these principles are inconsistent: priority scoring and ranking fails to maximize risk-reduction benefits. Real-world risk priority scoring systems used in homeland security and terrorism risk assessment, environmental risk management, information system vulnerability rating, business risk matrices, and many other important applications do not exploit correlations among risk-reducing opportunities or optimally diversify risk-reducing investments. As a result, they generally make suboptimal risk management recommendations. Applying portfolio optimization methods instead of risk prioritization ranking, rating, or scoring methods can achieve greater risk-reduction value for resources spent.     

Further Thoughts on the Utility of Risk Matrices

Risk matrices are commonly encountered devices for rating hazards in numerous areas of risk management. Part of their popularity is predicated on their apparent simplicity and transparency. Recent research, however, has identified serious mathematical defects and inconsistencies. This article further examines the reliability and utility of risk matrices for ranking hazards, specifically in the context of public leisure activities including travel. We find that (1) different risk assessors may assign vastly different ratings to the same hazard, (2) even following lengthy reflection and learning scatter remains high, and (3) the underlying drivers of disparate ratings relate to fundamentally different worldviews, beliefs, and a panoply of psychosocial factors that are seldom explicitly acknowledged. It appears that risk matrices when used in this context may be creating no more than an artificial and even untrustworthy picture of the relative importance of hazards, which may be of little or no benefit to those trying to manage risk effectively and rationally.     

Some Limitations of Qualitative Risk Rating Systems

Qualitative systems for rating animal antimicrobial risks using ordered categorical labels such as “high,”“medium,” and “low” can potentially simplify risk assessment input requirements used to inform risk management decisions. But do they improve decisions? This article compares the results of qualitative and quantitative risk assessment systems and establishes some theoretical limitations on the extent to which they are compatible. In general, qualitative risk rating systems satisfying conditions found in real‐world rating systems and guidance documents and proposed as reasonable make two types of errors: (1) Reversed rankings, i.e., assigning higher qualitative risk ratings to situations that have lower quantitative risks; and (2) Uninformative ratings, e.g., frequently assigning the most severe qualitative risk label (such as “high”) to situations with arbitrarily small quantitative risks and assigning the same ratings to risks that differ by many orders of magnitude. Therefore, despite their appealing consensus‐building properties, flexibility, and appearance of thoughtful process in input requirements, qualitative rating systems as currently proposed often do not provide sufficient information to discriminate accurately between quantitatively small and quantitatively large risks. The value of information (VOI) that they provide for improving risk management decisions can be zero if most risks are small but a few are large, since qualitative ratings may then be unable to confidently distinguish the large risks from the small. These limitations suggest that it is important to continue to develop and apply practical quantitative risk assessment methods, since qualitative ones are often unreliable.     

Risk Matrix Integrating Risk Attitudes Based on Utility Theory

Recent studies indicate that absence of the consideration of risk attitudes of decisionmakers in the risk matrix establishment process has become a major limitation. In order to evaluate risk in a more comprehensive manner, an approach to establish risk matrices that integrates risk attitudes based on utility theory is proposed. There are three main steps within this approach: (1) describing risk attitudes of decisionmakers by utility functions, (2) bridging the gap between utility functions and the risk matrix by utility indifference curves, and (3) discretizing utility indifference curves. A complete risk matrix establishment process based on practical investigations is introduced. This process utilizes decisionmakers’ answers to questionnaires to formulate required boundary values for risk matrix establishment and utility functions that effectively quantify their respective risk attitudes.     

A multicriteria outranking approach for modeling corporate credit ratings: An application of the Electre Tri-nC method

Corporate credit ratings are widely used in financial services for risk management, investment, and financing decisions. In this study, the use of a recently developed multicriteria outranking approach, namely the Electre Tri-nC method, is examined for constructing internal credit rating models in an expert-based judgmental framework. The models are constructed in a multicriteria classification (sorting) setting and the results are analyzed in terms of their internal properties as well as their deviations from risk rating categories defined by rating agencies (i.e. external benchmarking). A simulation/scenario analysis is conducted to examine the results and performance of the outranking models in relation to their parameters. Empirical results are provided for a sample of European firms rated by three leading rating agencies.     

What's Wrong with Risk Matrices?

Risk matrices—tables mapping "frequency" and "severity" ratings to corresponding risk priority levels—are popular in applications as diverse as terrorism risk analysis, highway construction project management, office building risk analysis, climate change risk management, and enterprise risk management (ERM). National and international standards (e.g., Military Standard 882C and AS/NZS 4360:1999) have stimulated adoption of risk matrices by many organizations and risk consultants. However, little research rigorously validates their performance in actually improving risk management decisions. This article examines some mathematical properties of risk matrices and shows that they have the following limitations. (a) Poor Resolution . Typical risk matrices can correctly and unambiguously compare only a small fraction (e.g., less than 10%) of randomly selected pairs of hazards. They can assign identical ratings to quantitatively very different risks ("range compression"). (b) Errors . Risk matrices can mistakenly assign higher qualitative ratings to quantitatively smaller risks. For risks with negatively correlated frequencies and severities, they can be "worse than useless," leading to worse-than-random decisions. (c) Suboptimal Resource Allocation . Effective allocation of resources to risk-reducing countermeasures cannot be based on the categories provided by risk matrices. (d) Ambiguous Inputs and Outputs . Categorizations of severity cannot be made objectively for uncertain consequences. Inputs to risk matrices (e.g., frequency and severity categorizations) and resulting outputs (i.e., risk ratings) require subjective interpretation, and different users may obtain opposite ratings of the same quantitative risks. These limitations suggest that risk matrices should be used with caution, and only with careful explanations of embedded judgments.     

Societal Risk Perception Among African Villagers Without Access to the Media

The impact of the media on people's risk perception was assessed by comparing risk ratings obtained from African villagers without access to the media with risk ratings obtained from African city-dwellers with access to the media and risk ratings obtained from French participants. The overall mean risk judgment observed among the Togolese villagers was lower than the mean rating observed among the Togolese city-dwellers, and lower than the mean rating observed among the French. The linear association observed between the Togolese villagers' ratings and the Togolese city-dweller ratings and the French ratings was moderate. The impact of the media on risk perception was estimated to be an increase of about 15% of the overall mean ratings, and to about 31% of the variance of the mean ratings. This impact was independent of educational level.     

Improving Risk Management: From Lame Excuses to Principled Practice

The three classic pillars of risk analysis are risk assessment (how big is the risk and how sure can we be?), risk management (what shall we do about it?), and risk communication (what shall we say about it, to whom, when, and how?). We propose two complements as important parts of these three bases: risk attribution (who or what addressable conditions actually caused an accident or loss?) and learning from experience about risk reduction (what works, and how well?). Failures in complex systems usually evoke blame, often with insufficient attention to root causes of failure, including some aspects of the situation, design decisions, or social norms and culture. Focusing on blame, however, can inhibit effective learning, instead eliciting excuses to deflect attention and perceived culpability. Productive understanding of what went wrong, and how to do better, thus requires moving past recrimination and excuses. This article identifies common blame‐shifting “lame excuses” for poor risk management. These generally contribute little to effective improvements and may leave real risks and preventable causes unaddressed. We propose principles from risk and decision sciences and organizational design to improve results. These start with organizational leadership. More specifically, they include: deliberate testing and learning—especially from near‐misses and accident precursors; careful causal analysis of accidents; risk quantification; candid expression of uncertainties about costs and benefits of risk‐reduction options; optimization of tradeoffs between gathering additional information and immediate action; promotion of safety culture; and mindful allocation of people, responsibilities, and resources to reduce risks. We propose that these principles provide sound foundations for improving successful risk management.     

A Review of Risk Matrices Used in Acute Hospitals in England

In healthcare, patient safety has received substantial attention and, in turn, a number of approaches to managing safety have been adopted from other high‐risk industries. One of these has been risk assessment, predominantly through the use of risk matrices. However, while other industries have criticized the design and use of these risk matrices, the applicability of such criticism has not been investigated formally in healthcare. This study examines risk matrices as used in acute hospitals in England and the guidance provided for their use. It investigates the applicability of criticisms of risk matrices from outside healthcare through a document analysis of the risk assessment policies, procedures, and strategies used in English hospitals. The findings reveal that there is a large variety of risk matrices used, where the design of some might increase the chance of risk misprioritization. Additionally, findings show that hospitals may provide insufficient guidance on how to use risk matrices as well as what to do in response to the existing criticisms of risk matrices. Consequently, this is likely to lead to variation in the quality of risk assessment and in the subsequent deployment of resources to manage the assessed risk. Finally, the article outlines ways in which hospitals could use risk matrices more effectively.     

Appraisal of Individual Radiation Risk in the Context of Probabilistic Exposures

There exists a growing desire to base safety criteria in different fields on the same principles. The current approach by the international Commission on Radiological Protection (ICRP) to control radiation exposure touches many aspects such as social, psychological, or economic factors that are important for such principles. This paper attempts to further explore possible ways of defining a common basis for dealing with radiation risks and other safety problems. Specifically, it introduces the following issues: (1) different types of risk are judged differently. To account for this, the concept of risk categories is introduced. (2) The dimension of time may play an important role. There is a difference between an immediate death and a death occurring 20 years after exposure to radiation. Effects such as reduced quality of life after exposure and reduction of lifetime expectancy are discussed. The paper suggests to introduce an individual risk equivalent which allows to compare risks as defined in various fields. Furthermore, it suggests the use of risk acceptance criteria which depend on the different categories of risk.     

Quality Risk Ratings in Global Supply Chains

Extended enterprises face many challenges in managing the product quality of their suppliers. Consequently characterizing the quality risk posed by value‐chain partners has become increasingly important. There have been several recent efforts to develop frameworks for rating the quality risk posed by suppliers. We develop an analytical model to examine the impact of such quality ratings on suppliers, manufacturers, and social welfare. While it might seem that quality ratings would benefit high‐quality suppliers and hurt low‐quality suppliers, we show that this is not always the case. We find that such quality ratings can hurt both types of suppliers or benefit both, depending on the market conditions. We also find that quality ratings do not always benefit the most demanding manufacturers who desire high‐quality suppliers. Finally, we find that social welfare is not always improved by risk ratings. These results suggest that public policy initiatives addressing risk ratings must be carefully considered.     

Assessing Perceptions of Synergistic Health Risk: A Comparison of Two Scales

Two approaches to measuring perceptions of synergistic risk were compared, one using the traditional Likert scale, the other using an anchored, relative scale. Perception of synergistic risk was defined as rating the combined hazard as more risky than each of its constituent single hazards. In a within-subjects design, a convenience sample from the community (N= 604) rated three hazard combinations and their constituents: Driving while Intoxicated (familiar, high synergy), Radon and Smoking (unfamiliar, high synergy), and Smoking and Driving (familiar, low synergy), on both scales. The relative scale was expected to be a more sensitive measure of synergy than the Likert scale. The effects of item order (single hazards rated first versus combined hazards rated first) were examined between subjects. Driving while Intoxicated was perceived by the large majority of participants as a synergistic risk on both scales, but neither of the other two combined hazards were rated synergistically on either scale. The relative scale demonstrated a slight advantage over the Likert scale, and presenting the single hazards first for the relative scale produced more synergistic ratings. It is recommended that anchored, relative scales be used to measure synergy and that single hazards be presented prior to the combined hazards when using relative scales.     

Disaster risk and artificial intelligence: A framework to characterize conceptual synergies and future opportunities

Artificial intelligence (AI) methods have revolutionized and redefined the landscape of data analysis in business, healthcare, and technology. These methods have innovated the applied mathematics, computer science, and engineering fields and are showing considerable potential for risk science, especially in the disaster risk domain. The disaster risk field has yet to define itself as a necessary application domain for AI implementation by defining how to responsibly balance AI and disaster risk. (1) How is AI being used for disaster risk applications; and how are these applications addressing the principles and assumptions of risk science, (2) What are the benefits of AI being used for risk applications; and what are the benefits of applying risk principles and assumptions for AI-based applications, (3) What are the synergies between AI and risk science applications, and (4) What are the characteristics of effective use of fundamental risk principles and assumptions for AI-based applications? This study develops and disseminates an online survey questionnaire that leverages expertise from risk and AI professionals to identify the most important characteristics related to AI and risk, then presents a framework for gauging how AI and disaster risk can be balanced. This study is the first to develop a classification system for applying risk principles for AI-based applications. This classification contributes to understanding of AI and risk by exploring how AI can be used to manage risk, how AI methods introduce new or additional risk, and whether fundamental risk principles and assumptions are sufficient for AI-based applications.     

A feasibility approach for constructing combinatorial designs of circulant type

In this work, we propose an optimization approach for constructing various classes of circulant combinatorial designs that can be defined in terms of autocorrelation. The problem is formulated as a so-called feasibility problem having three sets, to which the Douglas–Rachford projection algorithm is applied. The approach is illustrated on three different classes of circulant combinatorial designs: circulant weighing matrices, D-optimal matrices of circulant type, and Hadamard matrices with two circulant cores. Furthermore, we explicitly construct two new circulant weighing matrices, a CW(126, 64) and a CW(198, 100), whose existence was previously marked as unresolved in the most recent version of Strassler’s table.     

A Risk Analysis Model in Concurrent Engineering Product Development

Concurrent engineering has been widely accepted as a viable strategy for companies to reduce time to market and achieve overall cost savings. This article analyzes various risks and challenges in product development under the concurrent engineering environment. A three‐dimensional early warning approach for product development risk management is proposed by integrating graphical evaluation and review technique (GERT) and failure modes and effects analysis (FMEA). Simulation models are created to solve our proposed concurrent engineering product development risk management model. Solutions lead to identification of key risk controlling points. This article demonstrates the value of our approach to risk analysis as a means to monitor various risks typical in the manufacturing sector. This article has three main contributions. First, we establish a conceptual framework to classify various risks in concurrent engineering (CE) product development (PD). Second, we propose use of existing quantitative approaches for PD risk analysis purposes: GERT, FMEA, and product database management (PDM). Based on quantitative tools, we create our approach for risk management of CE PD and discuss solutions of the models. Third, we demonstrate the value of applying our approach using data from a typical Chinese motor company.     

Perception of Risk and Credibility at Toxic Sites

This study integrates previous research methodologies to compare the risk perceptions and responses to risk messages of agency personnel and neighbors of Superfund sites in Michigan. The integration attempted and the focus on risk messages are shaped by a critical review of the social amplification conceptual framework. The study involved all four agency groups and three groups of site neighbors actively involved in Superfund planning across the state. The first part of the study utilized the psychometric techniques of hazard rating and hazard profiles that had not previously been used in studies involving stakeholders. While agency personnel responded similarly to experts in previous studies, the responses of individuals in the neighbor groups reflected experience with toxic sites and were dissimilar to previous ratings by the general public. The second part of the study consisted of a hypothetical toxic site scenario that focused on specific risk messages at different times in the site history. Results indicate that the difference in perception of risk occurs after the first testing at a site, and that dramatic differences arise between agency and resident groups regarding the credibility of information sources and the need for independent testing. A general lack of trust in the Superfund program was demonstrated by all groups. The results indicate that problems of institutional credibility and program adequacy cannot be addressed by better risk communication.     

An Information‐Processing Approach for Evaluating In‐Store Retail Operational Design Strategies

We develop a model to evaluate retail store operational design strategies using an information‐processing perspective of organizational design. We propose that three model constructs pertaining to in‐store shopper task uncertainty—the product mix complexity, the service production complexity, and the product mix changeover—create shopper encounter information requirements (IR). These requirements can be met using specific retail service operational design choices for managing shopper encounters, namely, designing layouts for self‐service (SS) and providing employees with task empowerment (TE). The model is then operationalized using a two‐stage approach to develop new multi‐item, measurement scales. The psychometric properties and predictive validity of the scales and model are then confirmed by using structural equation modeling with survey data from 175 merchandise retail store managers. We find that our model can be generically applied across the retail industry to understand how shopper encounter IR motivate retailer store design choices and can be used to determine whether to design stores for SS or to provide store employees with TE. We then evaluate the efficacy of the studied store design choices on customer delivery satisfaction, and offer some suggestions for future research.     

Market‐Implied Spread for Earthquake CAT Bonds: Financial Implications of Engineering Decisions

In the event of natural and man‐made disasters, owners of large‐scale infrastructure facilities (assets) need contingency plans to effectively restore the operations within the acceptable timescales. Traditionally, the insurance sector provides the coverage against potential losses. However, there are many problems associated with this traditional approach to risk transfer including counterparty risk and litigation. Recently, a number of innovative risk mitigation methods, termed alternative risk transfer (ART) methods, have been introduced to address these problems. One of the most important ART methods is catastrophe (CAT) bonds. The objective of this article is to develop an integrative model that links engineering design parameters with financial indicators including spread and bond rating. The developed framework is based on a four‐step structural loss model and transformed survival model to determine expected excess returns. We illustrate the framework for a seismically designed bridge using two unique CAT bond contracts. The results show a nonlinear relationship between engineering design parameters and market‐implied spread.