Use of Multicriteria Decision Analysis to Support Weight of Evidence Evaluation

Weight of evidence (WOE) methods are key components of ecological and human health risk assessments. Most WOE applications rely on the qualitative integration of diverse lines of evidence (LOE) representing impact on ecological receptors and humans. Recent calls for transparency in assessments and justifiability of management decisions are pushing the community to consider quantitative methods for integrated risk assessment and management. This article compares and contrasts the type of information required for application of individual WOE techniques and the outcomes that they provide in ecological risk assessment and proposes a multicriteria decision analysis (MCDA) framework for integrating individual LOE in support of management decisions. The use of quantitative WOE techniques is illustrated for a hypothetical but realistic case study of selecting remedial alternatives at a contaminated aquatic site. Use of formal MCDA does not necessarily eliminate biases and judgment calls necessary for selecting remedial alternatives, but allows for transparent evaluation and fusion of individual LOE. It also provides justifiable methods for selecting remedial alternatives consistent with stakeholder and decision‐maker values.     

Improving Health Risk Assessment as a Basis for Public Health Decisions in the 21st Century

One-fifth of the way through the 21st century, a commonality of factors with those of the last 50 years may offer the opportunity to address unfinished business and current challenges. The recommendations include: (1) Resisting the tendency to oversimplify scientific assessments by reliance on single disciplines in lieu of clear weight-of-evidence expressions, and on single quantitative point estimates of health protective values for policy decisions; (2) Improving the separation of science and judgment in risk assessment through the use of clear expressions of the range of judgments that bracket protective quantitative levels for public health protection; (3) Use of comparative risk to achieve the greatest gains in health and the environment; and (4) Where applicable, reversal of the risk assessment and risk management steps to facilitate timely and substantive improvements in public health and the environment. Lessons learned and improvements in the risk assessment process are applied to the unprecedented challenges of the 21st century such as, pandemics and climate change. The beneficial application of the risk assessment and risk management paradigm to ensure timely research with consistency and transparency of assessments is presented. Institutions with mandated stability and leadership roles at the national and international levels are essential to ensure timely interdisciplinary scientific assessment at the interface with public policy as a basis for organized policy decisions, to meet time sensitive goals, and to inform the public.     

Multicriteria Decision Framework for Cybersecurity Risk Assessment and Management

Risk assessors and managers face many difficult challenges related to novel cyber systems. Among these challenges are the constantly changing nature of cyber systems caused by technical advances, their distribution across the physical, information, and sociocognitive domains, and the complex network structures often including thousands of nodes. Here, we review probabilistic and risk-based decision-making techniques applied to cyber systems and conclude that existing approaches typically do not address all components of the risk assessment triplet (threat, vulnerability, consequence) and lack the ability to integrate across multiple domains of cyber systems to provide guidance for enhancing cybersecurity. We present a decision-analysis-based approach that quantifies threat, vulnerability, and consequences through a set of criteria designed to assess the overall utility of cybersecurity management alternatives. The proposed framework bridges the gap between risk assessment and risk management, allowing an analyst to ensure a structured and transparent process of selecting risk management alternatives. The use of this technique is illustrated for a hypothetical, but realistic, case study exemplifying the process of evaluating and ranking five cybersecurity enhancement strategies. The approach presented does not necessarily eliminate biases and subjectivity necessary for selecting countermeasures, but provides justifiable methods for selecting risk management actions consistent with stakeholder and decisionmaker values and technical data.     

A Discussion of Findings and Their Possible Implications from a Workshop on Bioterrorism Threat Assessment and Risk Management

In November 2001, the Monterey Institute of International Studies convened a workshop on bioterrorism threat assessment and risk management. Risk assessment practitioners from various disciplines, but without specialized knowledge of terrorism, were brought together with security and intelligence threat analysts to stimulate an exchange that could be useful to both communities. This article, prepared by a subset of the participants, comments on the workshop's findings and their implications and makes three recommendations, two short term (use of threat assessment methodologies and vulnerability analysis) and one long term (application of quantitative risk assessment and modeling), regarding the practical application of risk assessment methods to bioterrorism issues.     

Risk Modeling, Assessment, and Management of Lahar Flow Threat

The 1991 eruption of Mount Pinatubo in the Philippines is considered one of the most violent and destructive volcanic activities in the 20th century. Lahar is the Indonesian term for volcanic ash, and lahar flows resulting from the massive amount of volcanic materials deposited on the mountain's slope posed continued post-eruption threats to the surrounding areas, destroying lives, homes, agricultural products, and infrastructures. Risks of lahar flows were identified immediately after the eruption, with scientific data provided by the Philippine Institute of Volcanology, the U.S. Geological Survey, and other research institutions. However, competing political, economic, and social agendas subordinated the importance of scientific information to policy making. Using systemic risk analysis and management, this article addresses the issues of multiple objectives and the effective integration of scientific techniques into the decision-making process. It provides a modeling framework for identifying, prioritizing, and evaluating policies for managing risk. The major considerations are: (1) applying a holistic approach to risk analysis through hierarchical holographic modeling, (2) applying statistical methods to gain insight into the problem of uncertainty in risk assessment, (3) using multiobjective trade-off analysis to address the issue of multiple decisionmakers and stakeholders in the decision-making process, (4) using the conditional expected value of extreme events to complement and supplement the expected value in quantifying risk, and (5) assessing the impacts of multistage decisions. Numerical examples based on ex post data are formulated to illustrate applications to various problems. The resulting framework from this study can serve as a general baseline model for assessing and managing risks of natural disasters, which the Philippines' lead agency-the National Disaster Coordinating Council (NDCC)-and other related organizations can use for their decision-making processes.     

Use of Advances in Technology for Maritime Risk Assessment

The maritime industry is moving toward a "goal-setting" risk-based regime. This opens the way to safety engineers to explore and exploit flexible and advanced risk modeling and decision-making approaches in the design and operation processes. In this article, following a brief review of the current status of maritime risk assessment, a design/operation selection framework and a design/operation optimization framework are outlined. A general discussion of control engineering techniques and their application to risk modeling and decision making is given. Four novel risk modeling and decision-making approaches are then outlined with illustrative examples to demonstrate their use. Such approaches may be used as alternatives to facilitate risk modeling and decision making in situations where conventional techniques cannot be appropriately applied. Finally, recommendations on further exploitation of advances in general engineering and technology are suggested with respect to risk modeling and decision making.     

Small Group Studies of Regulatory Decision-Making for Power-Frequency Electric and Magnetic Fields

Three groups of lay opinion leaders were used in a group role-playing decision exercise designed to explore problems in public risk management decision-making. The application domain was possible risks from the 60 Hz electric and magnetic fields associated with high-voltage power transmission lines. While there were differences in the make-up and dynamics of the three groups, the structure and substantive content of the tasks undertaken dominated intergroup variation in terms of the factors that were most important to group members' decisions. The groups displayed sophistication in their identification of decision attributes and in many of the arguments they advanced, but experienced difficulties in structuring and making trade-offs and decisions. The groups were not good at normalizing or otherwise manipulating quantitative information, and used it largely in the form it was received. Upper-bound risk estimates were treated operationally in most group discussions as expected values. Several kinds of strong framing effects were observed in the use of cost and risk information. Specific quantitative results obtained must be treated with care but may provide a starting place for further work on the acceptable level of transmission line risk.     

Risk and Systems Theory

The last few decades have seen increasingly widespread use of risk assessment and management techniques as aids in making complex decisions. However, despite the progress that has been made in risk science, there still remain numerous examples of risk-based decisions and conclusions that have caused great controversy. In particular, there is a great deal of debate surrounding risk assessment: the role of values and ethics and other extra-scientific factors, the efficacy of quantitative versus qualitative analysis, and the role of uncertainty and incomplete information. Many of the epistemological and methodological issues confronting risk assessment have been explored in general systems theory, where techniques exist to manage such issues. However, the use of systems theory and systems analysis tools is still not widespread in risk management. This article builds on the Alachlor risk assessment case study of Brunk, Haworth, and Lee to present a systems-based view of the risk assessment process. The details of the case study are reviewed and the authors' original conclusions regarding the effects of extra-scientific factors on risk assessment are discussed. Concepts from systems theory are introduced to provide a mechanism with which to illustrate these extra-scientific effects The role of a systems study within a risk assessment is explained, resulting in an improved view of the problem formulation process The consequences regarding the definition of risk and its role in decision making are then explored.     

Plurality of Worlds, Plurality of Risks

Risk management decisions are not made only on the basis of expert risk assessment. In numerous instances, public controversy erupts, questioning the results of previous risk assessment procedures and shaping the development of risk management episodes. This article presents a case study of risk management in the context of a 1980s controversy over aerial spraying against a spruce budworm epidemic in Quebec and draws some general conclusions concerning the relationship between risk analysis and public controversies. Actors in public controversies define risks more broadly than risk assessment experts. Moreover, public controversies only partly concern issues of risk. They are first and foremost debates about social choices in which actors carry with them a multidimensional social experience of technology, trust, credibility and decision-making institutions. This experience contributes to the construction of a plurality of emergent representations of what is at stake in a controversy, referred to in this paper as "worlds of relevance." Analysis shows that in any given public controversy, there are not just two parties arguing against each other. Rather, several "worlds of relevance" can be found that link, in a variety of ways, a variety of entities not necessarily shared by all these worlds. Each "world of relevance" presents a different definition of what the issues and the stakes of the controversy are. Risks are only part of the picture, and they are embedded in "worlds of relevance" from which they take their significance. The successful management of a controversy entails the association of entities from different worlds.     

Using Structured Decision Making to Help Implement a Precautionary Approach to Endangered Species Management

Endangered species protection is a significant risk management concern throughout North America. An extensive conceptual literature emphasizes the role to be played by precautionary approaches. Risk managers, typically working in concert with concerned stakeholders, frequently cite the concept as key to their efforts to prevent extinctions. Little has been done, however, to evaluate the multidimensional impacts of precautionary frameworks or to assist in the examination of competing precautionary risk management options as part of an applied risk management decision framework. In this article we describe how decision-aiding techniques can assist in the creation and analysis of alternative precautionary strategies, using the example of a multistakeholder committee charged with protection of endangered Cultus Lake salmon on the Canadian west coast. Although managers were required to adopt a precautionary approach, little attention had been given to how quantitative analyses could be used to help define the concept or to how a precautionary approach might be implemented in the face of difficult economic, social, and biological tradeoffs. We briefly review key steps in a structured decision-making (SDM) process and discuss how this approach was implemented to help bound the management problem, define objectives and performance measures, develop management alternatives, and evaluate their consequences. We highlight the role of strategy tables, employed to help participants identify, alternative management options. We close by noting areas of agreement and disagreement among participants and discuss the implications of decision-focused processes for other precautionary resource management efforts.     

Efficient or Fair? Operationalizing Ethical Principles in Flood Risk Management: A Case Study on the Dutch-German Rhine

Flood risk management decisions in many countries are based on decision-support frameworks which rely on cost-benefit analyses. Such frameworks are seldom informative about the geographical distribution of risk, raising questions on the fairness of the proposed policies. In the present work, we propose a new decision criterion that accounts for the distribution of risk reduction and apply it to support flood risk management decisions on a transboundary stretch of the Rhine River. Three types of interventions are considered: embankment heightening, making Room for the River, and changing the discharge distribution of the river branches. The analysis involves solving a flood risk management problem according to four alternative formulations, based on different ethical principles. Formulations based on cost optimization lead to very poor performances in some areas for the sake of reducing the overall aggregated costs. Formulations that also include equity criteria have different results depending on how these are defined. When risk reduction is distributed equally, very poor economic performance is achieved. When risk is distributed equally, results are in line with formulations based on cost optimization, while a fairer risk distribution is achieved. Risk reduction measures also differ, with the cost optimization approach strongly favoring the leverage of changing the discharge distribution and the alternative formulations spending more on embankment heightening and Room for the River, to rebalance inequalities in risk levels. The proposed method advances risk-based decision-making by allowing to consider risk distribution aspects and their impacts on the choice of risk reduction measures.     

Does Concern‐Driven Risk Management Provide a Viable Alternative to QRA?

This article discusses a concept of concern-driven risk management, in which qualitative expert judgments about whether concerns warrant specified risk management interventions are used in preference to quantitative risk assessment (QRA) to guide risk management decisions. Where QRA emphasizes formal quantitative assessment of the probable consequences caused by the recommended actions, and comparison to the probable consequences of alternatives, including the status quo, concern-driven risk management instead emphasizes perceived urgency or severity of the situation motivating recommended interventions. In many instances, especially those involving applications of the precautionary principle, no formal quantification or comparison of probable consequences for alternative decisions is seen as being necessary (or, perhaps, possible or desirable) prior to implementation of risk management measures. Such concern-driven risk management has been recommended by critics of QRA in several areas of applied risk management. Based on case studies and psychological literature on the empirical performance of judgment-based approaches to decision making under risk and uncertainty, we conclude that, although concern-driven risk management has several important potential political and psychological advantages over QRA, it is not clear that it performs better than (or as well as) QRA in identifying risk management interventions that successfully protect human health or achieve other desired consequences. Therefore, those who advocate replacing QRA with concern-driven alternatives, such as expert judgment and consensus decision processes, should assess whether their recommended alternatives truly outperform QRA, by the criterion of producing preferred consequences, before rejecting the QRA paradigm for practical applications.     

An Integrated Methodology for Assessment and Selection of the Project Risk Response Actions

In a systematic process of project risk management, after risk assessment is implemented, the risk analysts encounter the phase of assessment and selection of the project risk response actions (RA). As indicated by many researchers, there are less systematic and well-developed solutions in the area of risk response assessment and selection. The present article introduces a methodology including a modeling approach with the objective of selecting a set of RA that minimizes the undesirable deviation from achieving the project scope. The developed objective function comprises the three key success criteria of a project, namely, time, quality, and cost. Our model integrates overall project management into the project risk response planning (P2RP). Furthermore, the proposed model stresses on an equivalent importance for both "risk" and "response." We believe that applying the proposed model helps the project risk analyst in most effective and efficient manner dealing with his or her complicated RA selection problems. The application of the proposed model was implemented in projects in the construction industry in which it showed tremendous time, cost, and quality improvements.     

Stepping Out of Your Own Shadow: A Didactic Example of How Facing Uncertainty Can Improve Decision-Making

Advocates of quantitative uncertainty analysis (QUA) have invested substantial effort in explaining why uncertainty is a crucial aspect of risk and yet have devoted much less effort to explaining how QUA can improve the risk manager's performance. This paper develops a teaching example, using a personal decision problem with subtle parallels to societal risk management, to show how choices made with increasing appreciation of uncertainty are superior ones. In the hypothetical, five analysts explain the same uncertain prospect (whether to invest in a volatile stock issue), with increasing attention to the nuances of uncertainty. The path through these different perspectives on the decision demonstrates four general points applicable to environmental risk management: (1) Various point estimates with equal claim to being "best estimates" can differ markedly from each other and lead to diametrically different choices; (2) "conservatism" has both relative and absolute meanings, with different implications for decision-making; (3) both inattention to and fixation on "outliers" in the uncertainty distribution can lead the manager astray; and (4) the best QUA is one that helps discriminate among real options, that points to optimum pathways toward new information, and that spurs on the iterative search for new decision options that may outperform any of the initial ones offered.     

Ecological Risk Assessment Conceptual Model Formulation for Nonindigenous Species

This article addresses the application of ecological risk assessment at the regional scale to the prediction of impacts due to invasive or nonindigenous species (NIS). The first section describes risk assessment, the decision-making process, and introduces regional risk assessment. A general conceptual model for the risk assessment of NIS is then presented based upon the regional risk assessment approach. Two diverse examples of the application of this approach are presented. The first example is based upon the dynamics of introduced plasmids into bacteria populations. The second example is the application risk assessment approach to the invasion of a coastal marine site of Cherry Point, Washington, USA by the European green crab. The lessons learned from the two examples demonstrate that assessment of the risks of invasion of NIS will have to incorporate not only the characteristics of the invasive species, but also the other stresses and impacts affecting the region of interest.     

Times Are Tough—Brother,Can You Paradigm?

Ten years ago, the National Academy of Science released its risk assessment/risk management (RA/RM) “paradigm” that served to crystallize much of the early thinking about these concepts. By defining RA as a four-step process, operationally independent from RM, the paradigm has presented society with a scheme, or a conceptually common framework, for addressing many risky situations (e.g., carcinogens, noncarcinogens, and chemical mixtures). The procedure has facilitated decision-making in a wide variety of situations and has identified the most important research needs. The past decade, however, has revealed that additional progress is needed. These areas include addressing the appropriate interaction (not isolation) between RA and RM, improving the methods for assessing risks from mixtures, dealing with “adversity of effect,” deciding whether “hazard” should imply an exposure to environmental conditions or to laboratory conditions, and evolving the concept to include both health and ecological risk. Interest in and expectations of risk assessment are increasing rapidly. The emerging concept of “comparative risk” (i.e., distinguishing between large risks and smaller risks that may be qualitatively different) is at a level comparable to that held by the concept of “risk” just 10 years ago. Comparative risk stands in need of a paradigm of its own, especially given the current economic limitations. “Times are tough; Brother, can you paradigm?”     

Risk Management for Hazardous Materials Transportation: A Local Study in Lyons

A probabilistic risk assessment study has been undertaken in the French city of Lyons. The issue was to know whether it was justified to forbid hazardous material lorries in the city center and to divert them through the suburbs. Therefore, two routes, the City Center route and the Suburban route were compared. This paper describes the analysis and shows how the results were used in the decision-making process. It also lists the difficulties that are encountered when trying to incorporate formal risk analysis into actual decision-making processes. The risk analysis showed that rerouting is an effective option with respect to all criteria. The mathematical expectation of the number of deaths is divided by three, the reduction on the annual frequency of catastrophic accidents is even more important (about one order of magnitude for accidents involving more than 50 deaths). The spatial analysis proved that the risk was more evenly distributed along the Suburban route. However, the annual expected number of death is low: 0.5 in the worst case. So traditional decision-making approaches do not indicate the necessity of rerouting. Such a situation is believed to be typical for risk management of major hazards. In Lyons, the use of a very small risk aversion factor is sufficient to justify the rerouting option on a formal decision-aiding basis. This is rather unusual, but it is thought that the recognition of the importance of risk aversion by the decision-makers themselves is a very positive outcome from this study.     

Use of risk assessment and life cycle assessment in decision making: a common policy research agenda.

Quantitative risk assessment (RA) and life cycle assessment (LCA) are both analytical tools used to support decision making in environmental management. They have been developed and used by largely separate groups of specialists, and it is worth considering whether there is a common research agenda that may increase the relevance of these tools in decision-making processes. The validity of drawing comparisons between use of the tools is established through examining key aspects of the two approaches for their similarities and differences, including the nature of each approach and contextual and methodological aspects. Six case studies involving use of each approach in public decision making are described and used to draw out concerns about using RA and LCA in this context. The following categories of concern can be distinguished: philosophical approach of the tools; quantitative versus qualitative assessment; stakeholder participation; the nature of the results; and the usefulness of the results in relation to time and financial resource requirements. These can be distilled into a common policy research agenda focusing on: the legitimacy of using tools built on a particular perspective in decision making; recognition and role of value judgments in RA and LCA; treatment of uncertainty and variability; the influence of analytical tools in focusing attention on particular aspects of a decision-making situation; and understandability of the results for nonspecialists. It is concluded that it is time to bring together the experiences of RA and LCA specialists and benefit from cross-fertilization of ideas.     

Rethinking Risk Assessment for Public Utility Safety Regulation

To aid in their safety oversight of large‐scale, potentially dangerous energy and water infrastructure and transportation systems, public utility regulatory agencies increasingly seek to use formal risk assessment models. Yet some of the approaches to risk assessment used by utilities and their regulators may be less useful for this purpose than is supposed. These approaches often do not reflect the current state of the art in risk assessment strategy and methodology. This essay explores why utilities and regulatory agencies might embrace risk assessment techniques that do not sufficiently assess organizational and managerial factors as drivers of risk, nor that adequately represent important uncertainties surrounding risk calculations. Further, it describes why, in the special legal, political, and administrative world of the typical public utility regulator, strategies to identify and mitigate formally specified risks might actually diverge from the regulatory promotion of “safety.” Some improvements are suggested that can be made in risk assessment approaches to support more fully the safety oversight objectives of public regulatory agencies, with examples from “high‐reliability organizations” (HROs) that have successfully merged the management of safety with the management of risk. Finally, given the limitations of their current risk assessments and the lessons from HROs, four specific assurances are suggested that regulatory agencies should seek for themselves and the public as objectives in their safety oversight of public utilities.