A Procedural Evaluation of an Analytic-Deliberative Process: The Columbia River Comprehensive Impact Assessment

The U.S. Department of Energy's Columbia River Comprehensive Impact Assessment (CRCIA) was an ambitious attempt to direct its cleanup of the Hanford Nuclear Reservation toward the most significant risks to the Columbia River resulting from past plutonium production. DOE's approach was uncommonly open, including tribal, regulatory agency, and other Hanford interest group representatives on the board that was to develop the assessment approach. The CRCIA process had attributes of the "analytic-deliberative" process for risk assessment recommended by the National Research Council. Nevertheless, differences between the DOE and other participants over what was meant by the term "comprehensive" in the group's charge, coupled with differing perceptions of the likely effectiveness of remediation efforts in reducing risks, were never resolved. The CRCIA effort became increasingly fragmented and the role its products were to play in influencing future clean-up decisions increasingly ambiguous. A procedural evaluation of the CRCIA process, based on Thomas Webler's procedural normative model of public participation, reveals numerous instances in which theoretical-normative discourse disconnects occurred. These had negative implications for both the basic procedural dimensions of Webler's model-fairness and competence. Tribal and other interest group representatives lacked the technical resources necessary to make or challenge what philosopher Jurgens Habermas terms cognitive validity claims, while DOE and its contractors did not challenge normative claims made by tribal representatives. The results are cautionary for implementation of the analytic-deliberative process. They highlight the importance of bringing rigor to the evaluation of the quality of the deliberation component of risk characterization via the analytic-deliberative process, as well as to the analytic component.     

Risk-Based Approaches to Deal with Uncertainty in a Data-Poor System: Stakeholder Involvement in Hazard Identification for Marine National Parks and Marine Sanctuaries in Victoria, Australia

Management responsibilities for the system of marine national parks and sanctuaries declared in Victoria, Australia in 2002 have created imperatives for robust, scientifically defendable approaches to identifying threats to valued ecological attributes of the parks, setting management priorities, and developing monitoring systems. We are developing a protocol for ecological risk assessment in the parks that has due regard for the perception of risks by individuals, and ensures that stakeholder values are an intrinsic part of decision making. The inclusive and transparent protocol provides an opportunity for stakeholder involvement in the identification of valued attributes, as well as in the assessment of associated risks. Our approach brings together ideas about how science enters the community engagement domain in ways that promote collaboration and transparency in decision making. A series of stakeholder workshops across the state drew on the expertise of agency staff, community groups, fishers, industry representatives, academics, and knowledgeable park neighbors to identify hazards of major concern in the parks. Many hazards involved predictable, tangible threats like pollution and exotic species, but the approach also identified a number of less obvious threats related to governance issues and the knowledge-base for the parks. Importantly, the workshops with their broad range of stakeholders identified threats not previously considered by the management agency in its internal assessments, and several of these "new" threats are already the subject of action by the agency. The deliberate incorporation of local knowledge and local networks in the risk assessment process also provided opportunities for greater engagement of stakeholders with the management agency.     

How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy

Decision making is a complex task that involves a multitude of perspectives, constraints, and variables. Multiple Criteria Decision Analysis (MCDA) is a process that has been used for several decades to support decision making. It includes a series of steps that systematically help Decision Maker(s) (DM(s)) and stakeholders in structuring a decision making problem, identifying their preferences, and building a decision recommendation consistent with those preferences. Over the last decades, many studies have demonstrated the conduct of the MCDA process and how to select an MCDA method. Until now, there has not been a review of these studies, nor a proposal of a unified and comprehensive high-level representation of the MCDA process characteristics (i.e., features), which is the goal of this paper. We introduce a review of the research that defines how to conduct the MCDA process, compares MCDA methods, and presents Decision Support Systems (DSSs) to recommend a relevant MCDA method or a subset of methods. We then synthesize this research into a taxonomy of characteristics of the MCDA process, grouped into three main phases, (i) problem formulation, (ii) construction of the decision recommendation, and (iii) qualitative features and technical support. Each of these phases includes a subset of the 10 characteristics that helps the analyst implementing the MCDA process, while also being aware of the implication of these choices at each step. By showing how decision making can be split into manageable and justifiable steps, we reduce the risk of overwhelming the analyst, as well as the DMs/stakeholders during the MCDA process. A questioning strategy is also proposed to demonstrate how to apply the taxonomy to map MCDA methods and select the most relevant one(s) using real case studies. Additionally, we show how the DSSs for MCDA method recommendation can be grouped into three main clusters. This proposal can enhance a traceable and categorizable development of such systems.     

Conditional Influence Diagrams in Risk Management

This paper introduces conditional influence diagrams into risk management. A contaminated-site cleanup involving two stakeholders is used as a hypothetical case study. The treatment choices must satisfy several conflicting objectives. Any decision made by one stakeholder will affect the choices of the other stakeholder. In building the influence diagrams for each of the stakeholders, the logical relationship of all relevant factors is determined and the values of these factors are analyzed. The influence diagram for each stakeholder is conditional on the options available to the other stakeholder. The influence diagrams are, then, used to evaluate the possible choices of each stakeholder based on decision options of the other stakeholder. These results are analyzed using game theory methods to gain insights useful to risk management and to demonstrate how mutual trust and cooperation can lead to decisions benefiting both stakeholders.     

Consortium for Atlantic Regional Assessment: Information Tools for Community Adaptation to Changes in Climate or Land Use

To inform local and regional decisions about protecting short-term and long-term quality of life, the Consortium for Atlantic Regional Assessment (CARA) provides data and tools (for the northeastern United States) that can help decision makers understand how outcomes of their decisions could be affected by potential changes in both climate and land use. On an interactive, user-friendly website, CARA has amassed data on climate (historical records and future projections for seven global climate models), land cover, and socioeconomic and environmental variables, along with tools to help decision makers tailor the data for their own decision types and locations. CARA Advisory Council stakeholders help identify what information and tools stakeholders would find most useful and how to present these; they also provide in-depth feedback for subregion case studies. General lessons include: (1) decision makers want detailed local projections for periods short enough to account for extreme events, in contrast to the broader spatial and temporal observations and projections that are available or consistent at a regional level; (2) stakeholders will not use such a website unless it is visually appealing and easy to find the information they want; (3) some stakeholders need background while others want to go immediately to data, and some want maps while others want text or tables. This article also compares what has been learned across case studies of Cape May County, New Jersey, Cape Cod, Massachusetts, and Hampton Roads, Virginia, relating specifically to sea-level rise. Lessons include: (1) groups can be affected differently by physical dangers compared with economic dangers; (2) decisions will differ according to decision makers' preferences about waiting and risk tolerance; (3) future scenarios and maps can help assess the impacts of dangers to emergency evacuation routes, homes, and infrastructure, and the natural environment; (4) residents' and decision makers' perceptions are affected by information about potential local impacts from global climate change.     

A decision support system for banks

Just recently, information systems with rather unique characteristics have begun to emerge. These systems usually referred to as Decision Support Systems, feature decision models, a data base and the decision maker as subsystems and are specifically oriented to supporting decision making. This article examines Decision Support Systems and their application to banks.     

Exploring Risk Judgments in a Trade Dispute Using Bayesian Networks

Bayesian networks (BNs) are graphical modeling tools that are generally recommended for exploring what‐if scenarios, visualizing systems and problems, and for communication between stakeholders during decision making. In this article, we investigate their potential for exploring different perspectives in trade disputes. To do so, we draw on a specific case study that was arbitrated by the World Trade Organization (WTO): the Australia‐New Zealand apples dispute. The dispute centered on disagreement about judgments contained within Australia's 2006 import risk analysis (IRA). We built a range of BNs of increasing complexity that modeled various approaches to undertaking IRAs, from the basic qualitative and semi‐quantitative risk analyses routinely performed in government agencies, to the more complex quantitative simulation undertaken by Australia in the apples dispute. We found the BNs useful for exploring disagreements under uncertainty because they are probabilistic and transparently represent steps in the analysis. Different scenarios and evidence can easily be entered. Specifically, we explore the sensitivity of the risk output to different judgments (particularly volume of trade). Thus, we explore how BNs could usefully aid WTO dispute settlement. We conclude that BNs are preferable to basic qualitative and semi‐quantitative risk analyses because they offer an accessible interface and are mathematically sound. However, most current BN modeling tools are limited compared with complex simulations, as was used in the 2006 apples IRA. Although complex simulations may be more accurate, they are a black box for stakeholders. BNs have the potential to be a transparent aid to complex decision making, but they are currently computationally limited. Recent technological software developments are promising.     

The Quality of Stakeholder-Based Decisions

The increased use of stakeholder processes in environmental decision making has raised concerns about the quality of decisions these processes produce. Some claim that stakeholders make inadequate use of scientific information and analysis and are all too ready to sacrifice technical quality for political expediency. This article looks to the case study record to examine the quality of the decisions from stakeholder-based processes. The data for the analysis come from a "case survey," in which researchers coded information from 239 published case studies of stakeholder involvement in environmental decision making. These cases reflect a diversity of planning, management, and implementation activities carried out by environmental and natural resource agencies at many levels of government. Overall, the case-study record suggests that there should be little concern that stakeholder processes are resulting in low-quality decisions. The majority of cases contain evidence of stakeholders improving decisions over the status quo; adding new information, ideas, and analysis; and having adequate access to technical and scientific resources. Indeed, data suggest that it is the more intensive stakeholder processes--precisely those that have aroused recent concern--that are more likely to result in higher-quality decisions.     

Multi-criteria decision support and evaluation of strategies for nuclear remediation management

Environmental emergency situations can differ in many ways, for instance according to their causes and the dimension of their impacts. Yet, they share the characteristic of sudden onset and the necessity for a coherent and effective emergency management. In this paper we consider decision support in the event of a nuclear or radiological accident in Europe. RODOS, an acronym for real-time on-line decision support system, is a decision support system designed to provide support from the early phases through to the medium and long-term phases. This work highlights the role of multi-criteria decision analysis (MCDA) within RODOS in ensuring the transparency of decision processes within emergency and remediation management. Special emphasis is placed on the evaluation of alternative remediation or countermeasure strategies using the multi-criteria decision support tool Web-HIPRE in scenario focused decision making workshops involving different stakeholder and expert groups. Decision support is enhanced by a module that generates natural language explanations to facilitate the understanding of the evaluation process, therefore contributing to the direct involvement of the decision makers, with the aim of increasing their confidence in the results of the analyses carried out, forming an audit trail for the decision making process and improving the acceptability of the system as a whole.     

Deliberation: Integrating Analytical Results into Environmental Decisions Involving Multiple Stakeholders

The National Research Council has recommended the use of an analytic/deliberative decision making process in environmental restoration decisions that involve multiple stakeholders. This work investigates the use of the results of risk assessment and multiattribute utility analysis (the "analysis") in guiding the deliberation. These results include the ranking of proposed remedial action alternatives according to each stakeholder's preferences, as well as the identification of the major reasons for these rankings. The stakeholder preferences are over a number of performance measures that include the traditional risk assessment metrics, e.g., individual worker risk, as well as programmatic, cultural, and cost-related impacts. Based on these results, a number of proposals are prepared for consideration by the stakeholders during the deliberation. These proposals are the starting point for the formulation of actual recommendations by the group. In our case study, these recommendations included new remedial action alternatives that were created by the stakeholders after an extensive discussion of the detailed analytical results.     

Belief in Public Efficacy, Trust, and Attitudes Toward Modern Genetic Science

Government and policymakers want to engage the public in a dialogue about the conduct and consequences of science and increasingly seek to actively involve citizens in decision-making processes. Implicit in this thinking is that greater transparency and public inclusion will help dispel fears associated with new scientific advancements, foster greater public trust in those accountable, and ultimately increase the acceptability of new technologies. Less understood, however, are public perceptions about such high-level involvement in science and how these map onto public trust and attitudes within a diverse population. This article uses the concept of public efficacy -- the extent to which people believe that the public might be able to affect the course of decision making -- to explore differences in trust, attentiveness, and attitudes toward modern genetic science. Using nationally representative data from the 2003 British Social Attitudes Survey, we begin by examining the characteristics of those who have a positive belief about public involvement in this area of scientific inquiry. We then focus on how this belief maps on to indicators of public trust in key stakeholder groups, including the government and genetic scientists. Finally, we consider the relationship between public efficacy and trust and attitudes toward different applications of genetic technology. Our findings run contrary to assumptions that public involvement in science will foster greater trust and lead to a climate of greater acceptance for genetic technology. A belief in public efficacy does not uniformly equate with more trusting attitudes toward stakeholders but is associated with less trust in government rules. Whereas trust is positively correlated with more permissive attitudes about technologies such as cloning and gene therapy, people who believe in high-level public involvement are less likely to think that these technologies should be allowed than those who do not.     

Who Holds the Stakes? A Case Study of Stakeholder Identification at Two Nuclear Weapons Production Sites

Traditional risk assessments, including those involving the United States Department of Energy (USDOE), are often criticized for producing useless or noncredible management responses because they did not meaningfully involve the public. The first step to involve the public is to identify appropriate active participants (stakeholders). This study was done to understand the processes used to identify stakeholders to serve on advisory boards established at the two largest remediation sites in the United States: the Hanford Nuclear Reservation in Washington state and the Savannah River Site in South Carolina. The Hanford stakeholder identification process produced an interest-based board whereas the Savannah River Site strategy produced population-based representation. The basic goals of the stakeholder advisory groups were similar. However, different processes were used to identify the participants for the groups in part because of distinctly different social and cultural conditions in the areas affected by the operations of the two facilities, and in part because of the different level of trust of the USDOE and their contractors at Hanford compared with Savannah River. The discussion analyzes their different needs and potential for successful citizen participation.     

Role for Risk Communication in Closing Military Waste Sites

Lessons learned from environmental and occupational hazard risk management practices over the past 30 years have led the Department of Defense to explore alternative risk management approaches. Policies for cleanup of environmentally hazardous waste sites are undergoing examination and are being reframed. A Demonstration Risk Communication Program is described which incorporates principles that integrate risk-based scientific information as well as community values, perceptions, and needs in a democratic process that includes the public as an active participant from the earliest stages. A strong scientific foundation for assessment and characterization of risk is viewed as necessary but not sufficient; the public's values must be actively integrated into the negotiated criteria. The Demonstration Program uses a model to prepare the participants and to guide them through the process. A five-step process is presented: (1) create risk communications process action team including at least one member of the specific site audience; (2) professionally train participants on team dynamics including interpersonal communication skills; (3) train risk communicators to deliver a cogent presentation of the message to secure a decision acceptable to both the government and the public; (4) identify existing biases, perceptions, and values held by all participants; and (5) develop risk message incorporating science and values. The process action team approach assumes the participants enter into the effort with the goal of improved environment and safeguarded public health. The team approach avoids confrontational or adversarial interactions and focuses on a dialogue from which a negotiated team response develops. Central to the program is the recognition that communication is only effective when the dialogue is two-way.     

Use of Multicriteria Decision Analysis to Support Weight of Evidence Evaluation

Weight of evidence (WOE) methods are key components of ecological and human health risk assessments. Most WOE applications rely on the qualitative integration of diverse lines of evidence (LOE) representing impact on ecological receptors and humans. Recent calls for transparency in assessments and justifiability of management decisions are pushing the community to consider quantitative methods for integrated risk assessment and management. This article compares and contrasts the type of information required for application of individual WOE techniques and the outcomes that they provide in ecological risk assessment and proposes a multicriteria decision analysis (MCDA) framework for integrating individual LOE in support of management decisions. The use of quantitative WOE techniques is illustrated for a hypothetical but realistic case study of selecting remedial alternatives at a contaminated aquatic site. Use of formal MCDA does not necessarily eliminate biases and judgment calls necessary for selecting remedial alternatives, but allows for transparent evaluation and fusion of individual LOE. It also provides justifiable methods for selecting remedial alternatives consistent with stakeholder and decision‐maker values.     

Risk Communication in Environmental Restoration Programs

The author advocates adoption of a convergence model in place of the traditional source-receiver model of communication for communicating with members of the public who have a stake in remediation of a nearby site. The source-receiver model conceives of communication as the transmission of a message from a risk management agency (sender) to a target audience of the public (receivers). The underlying theme is that the sender intends to change the perception of the receiver of either the issue or the sender of information. The author draws on her experience at a Department of Energy (DoE) site undergoing remediation to illustrate why the convergence model is more appropriate in the context of cleanup. This alternative model focuses on the Latin derivation of communication as sharing or making common to many (i.e., as involving a relationship between participants who engage in a process of communication). The focus appears to be consistent with recently issued DoE policy that calls for involving the public in identifying issues and problems and in formulating and evaluating decision alternatives in cleanup. By emphasizing context, process, and participants, as opposed to senders and receivers, the model identifies key issues to address in facilitating consensus concerning the risks of cleanup. Similarities between the institutional context of DoE and Department of Defense (DoD) suggest that a convergence model may also prove to be an appropriate conceptual foundation for risk communication at contaminated DoD sites.     

Evidence maps: communicating risk assessments in societal controversies: the case of engineered nanoparticles

The transparent and fair characterization of scientific evidence for reporting the results of a hazard assessment is a demanding task. In this article, we present an approach for characterizing evidence--the evidence map approach. The theoretical starting point is to view evidence characterization as a form of argumentation. Thus, evidence maps are designed to depict the evidence base, the pro and con arguments, and the remaining uncertainties, which together lead experts to their conclusions when summarizing and evaluating the scientific evidence about a potential hazard. To illustrate its use, the evidence maps approach is applied to characterizing the health-relevant effects of engineered nanoparticles. Empirical data from an online survey suggests that the use of evidence maps improves the reporting of hazard assessments. Nonexperts prefer to receive the information included in an evidence map in order to come to an informed judgment. Furthermore, the benefits and limitations of evidence maps are discussed in the light of recent literature on risk communication. Finally, the article underlines the need for further research in order to increase quality of evidence reporting.     

The European Commission''s White Paper "Strategy for a Future Chemicals Policy": A Review

The European Commission has proposed a radical new policy for the regulation of chemicals in the EU in the form of a White Paper. The current system has separate regulatory provisions for "new" chemicals (introduced to the market since September 18, 1981) and "existing" chemicals (on the market before September 18,1981). The proposed future policy will have a single unified regulatory system for all chemicals, which should result in better regulation of chemicals in the EU single market. It will be better because risk assessments will be targeted at the chemicals of greatest concern. Furthermore, the system will be streamlined, making regulatory decisions faster, and thus reducing the so-called burden of the past (the large number of chemicals that have never been assessed for their risks to human health or the environment). The new system incorporates the precautionary principle, which will be applied where there is an early indication of unacceptable risk or where there is undue delay in the regulatory process. Moreover, the new strategy is intended to promote greater transparency for all stakeholders.     

Risk Analysis,Decision Analysis,Causal Analysis,and Economics: A Personal Perspective from More Than 40 years Experience

I entered the field of risk analysis forty years ago from a background in physics followed by doctoral training and experience in decision analysis. I came into the Society for Risk Analysis (SRA) after participating as a committee member in the 1983 National Academies report, Risk Assessment in the Federal Government: Managing the Process. The insights and recommendations from this report, and successor reports from 1996 and 2008, merit revisiting on this 40th anniversary. Risk analysis includes risk assessment, a process of summarizing applicable science to inform decisions; and risk management, a process of making informed choices, usually involving multiple stakeholders. Inherent in both is the need to deal with complexity, uncertainty, and differing perspectives and goals. The lessons I have learned include the need for a conceptual separation of risk management from risk assessment, the benefit of an iterative dialogue between these activities, and the wisdom of articulating and assessing what we know, what we want, and what we can do as we seek to understand and manage risks affecting ourselves and those we advise.     

企业经理风险决策行为的实证研究

本文报告的两项研究系统深入地探讨了中国企业经理的风险决策行为。在研究一中,214名企业经理填答了效用测量表;在研究二中,82名企业经理填答风险决策情景模拟表。结果发现,个人特征(如性别、职位、所在企业的性质与规模等)、决策任务特征(如结果值的符号与大小)以及决策环境特征(如企业文化、企业经营状况、上司的风险态度等)均是影响企业经理风险决策行为的关键变量。本文为建立风险决策行为的理论或模型提供了基本的参考资料。