Ecological Risk Ranking: Development and Evaluation of a Method for Improving Public Participation in Environmental Decision Making

This article reports an extension of the Carnegie Mellon risk-ranking method to incorporate ecological risks and their attributes. On the basis of earlier risk-perception studies, we identified a set of 20 relevant attributes for describing health, safety, and environmental hazards in standardized risk summary sheets. In a series of three ranking sessions, 23 laypeople ranked 10 such hazards in a fictional Midwestern U.S. county using both holistic and multiattribute ranking procedures. Results were consistent with those from previous studies involving only health and safety hazards, providing additional evidence for the validity of the method and the replicability of the resulting rankings. Holistic and multiattribute risk rankings were reasonably consistent both for individuals and for groups. Participants reported that they were satisfied with the procedures and results, and indicated their support for using the method to advise real-world risk-management decisions. Agreement among participants increased over the course of the exercise, perhaps because the materials and deliberations helped participants to correct their misconceptions and clarify their values. Overall, health and safety attributes were judged more important than environmental attributes. However, the overlap between the importance rankings of these two sets of attributes suggests that some information about environmental impacts is important to participants' judgments in comparative risk-assessment tasks.     

A Deliberative Method for Ranking Risks (II): Evaluation of Validity and Agreement among Risk Managers

A deliberative method for ranking risks was evaluated in a study involving 218 risk managers. Both holistic and multiattribute procedures were used to assess individual and group rankings of health and safety risks facing students at a fictitious middle school. Consistency between the rankings that emerged from these two procedures was reasonably high for individuals and for groups, suggesting that these procedures capture an underlying construct of riskiness. Participants reported high levels of satisfaction with their groups' decision-making processes and the resulting rankings, and these reports were corroborated by regression analyses. Risk rankings were similar across individuals and groups, even though individuals and groups did not always agree on the relative importance of risk attributes. Lower consistency between the risk rankings from the holistic and multiattribute procedures and lower agreement among individuals and groups regarding these rankings were observed for a set of high-variance risks. Nonetheless, the generally high levels of consistency, satisfaction, and agreement suggest that this deliberative method is capable of producing risk rankings that can serve as informative inputs to public risk-management decision making.     

A New Multicriteria Risk Mapping Approach Based on a Multiattribute Frontier Concept

Invasive species risk maps provide broad guidance on where to allocate resources for pest monitoring and regulation, but they often present individual risk components (such as climatic suitability, host abundance, or introduction potential) as independent entities. These independent risk components are integrated using various multicriteria analysis techniques that typically require prior knowledge of the risk components’ importance. Such information is often nonexistent for many invasive pests. This study proposes a new approach for building integrated risk maps using the principle of a multiattribute efficient frontier and analyzing the partial order of elements of a risk map as distributed in multidimensional criteria space. The integrated risks are estimated as subsequent multiattribute frontiers in dimensions of individual risk criteria. We demonstrate the approach with the example of Agrilus biguttatus Fabricius, a high‐risk pest that may threaten North American oak forests in the near future. Drawing on U.S. and Canadian data, we compare the performance of the multiattribute ranking against a multicriteria linear weighted averaging technique in the presence of uncertainties, using the concept of robustness from info‐gap decision theory. The results show major geographic hotspots where the consideration of tradeoffs between multiple risk components changes integrated risk rankings. Both methods delineate similar geographical regions of high and low risks. Overall, aggregation based on a delineation of multiattribute efficient frontiers can be a useful tool to prioritize risks for anticipated invasive pests, which usually have an extremely poor prior knowledge base.     

Prioritizing Environmental Health Risks in the UAE

This article presents the results of a comparative environmental risk‐ranking exercise that was conducted in the United Arab Emirates (UAE) to inform a strategic planning process led by the Environment Agency‐Abu Dhabi (EAD). It represents the first national‐level application of a deliberative method for comparative risk ranking first published in this journal. The deliberative method involves a five‐stage process that includes quantitative risk assessment by experts and deliberations by groups of stakeholders. The project reported in this article considered 14 categories of environmental risks to health identified through discussions with EAD staff: ambient and indoor air pollution; drinking water contamination; coastal water pollution; soil and groundwater contamination; contamination of fruits, vegetables, and seafood; ambient noise; stratospheric ozone depletion; electromagnetic fields from power lines; health impacts from climate change; and exposure to hazardous substances in industrial, construction, and agricultural work environments. Results from workshops involving 73 stakeholders who met in five separate groups to rank these risks individually and collaboratively indicated strong consensus that outdoor and indoor air pollution are the highest priorities in the UAE. Each of the five groups rated these as being among the highest risks. All groups rated soil and groundwater contamination as being among the lowest risks. In surveys administered after the ranking exercises, participants indicated that the results of the process represented their concerns and approved of using the ranking results to inform policy decisions. The results ultimately shaped a strategic plan that is now being implemented.     

Regional Risk Assessment for Point Source Pollution Based on a Water Quality Model of the Taipu River,China

Point source pollution is one of the main threats to regional environmental health. Based on a water quality model, a methodology to assess the regional risk of point source pollution is proposed. The assessment procedure includes five parts: (1) identifying risk source units and estimating source emissions using Monte Carlo algorithms; (2) observing hydrological and water quality data of the assessed area, and evaluating the selected water quality model; (3) screening out the assessment endpoints and analyzing receptor vulnerability with the Choquet fuzzy integral algorithm; (4) using the water quality model introduced in the second step to predict pollutant concentrations for various source emission scenarios and analyzing hazards of risk sources; and finally, (5) using the source hazard values and receptor vulnerability scores to estimate overall regional risk. The proposed method, based on the Water Quality Analysis Simulation Program (WASP), was applied in the region of the Taipu River, which is in the Taihu Basin, China. Results of source hazard and receptor vulnerability analysis allowed us to describe aquatic ecological, human health, and socioeconomic risks individually, and also integrated risks in the Taipu region, from a series of risk curves. Risk contributions of sources to receptors were ranked, and the spatial distribution of risk levels was presented. By changing the input conditions, we were able to estimate risks for a range of scenarios. Thus, the proposed procedure may also be used by decisionmakers for long‐term dynamic risk prediction.     

Approximate weighting method for multiattribute decision problems with imprecise parameters

A multiattribute decision problem with imprecise parameters refers to one in which at least one of the parameters such as attribute weights and value scores is not represented by precise numerical values. Some well-known types of incomplete attribute weights are chosen and analyzed to find their extreme points. In doing so, we show that their coefficients matrix, by itself or by the change of variables, belongs to a class of M-matrix which enables us to find its extreme points readily due to the inverse-positive property.The knowledge of extreme points not only helps us to prioritize alternatives but also supports iterative exploration of decision-maker’s preference by investigating modified extreme points caused by additional preference information. A wide range of eligible attribute weights, however, often fail to result in the best alternative or a complete ranking of alternatives. To address this situation, we consider an approximate weighting method, so called the minimizing squared deviations from extreme points (MSD) which locates the attribute weights at the barycenter of a weight set. Accordingly, the MSD approach extends the rank order centroid (ROC) weighting method which is known to outperform other approximate weighting methods in case of ranked attribute weights. The evidence of the MSD’s superiority over a linear program-based weighting method is verified via simulation analysis under different forms of incomplete attribute weights.     

Probability Elicitation Under Severe Time Pressure: A Rank‐Based Method

Probability elicitation protocols are used to assess and incorporate subjective probabilities in risk and decision analysis. While most of these protocols use methods that have focused on the precision of the elicited probabilities, the speed of the elicitation process has often been neglected. However, speed is also important, particularly when experts need to examine a large number of events on a recurrent basis. Furthermore, most existing elicitation methods are numerical in nature, but there are various reasons why an expert would refuse to give such precise ratio‐scale estimates, even if highly numerate. This may occur, for instance, when there is lack of sufficient hard evidence, when assessing very uncertain events (such as emergent threats), or when dealing with politicized topics (such as terrorism or disease outbreaks). In this article, we adopt an ordinal ranking approach from multicriteria decision analysis to provide a fast and nonnumerical probability elicitation process. Probabilities are subsequently approximated from the ranking by an algorithm based on the principle of maximum entropy, a rule compatible with the ordinal information provided by the expert. The method can elicit probabilities for a wide range of different event types, including new ways of eliciting probabilities for stochastically independent events and low‐probability events. We use a Monte Carlo simulation to test the accuracy of the approximated probabilities and try the method in practice, applying it to a real‐world risk analysis recently conducted for DEFRA (the U.K. Department for the Environment, Farming and Rural Affairs): the prioritization of animal health threats.     

Ranking the Risks from Multiple Hazards in a Small Community

Natural hazards, human-induced accidents, and malicious acts have caused great losses and disruptions to society. After September 11, 2001, critical infrastructure protection has become a national focus in the United States and is likely to remain one for the foreseeable future. Damage to the infrastructures and assets could be mitigated through predisaster planning and actions. A systematic methodology was developed to assess and rank the risks from these multiple hazards in a community of 20,000 people. It is an interdisciplinary study that includes probabilistic risk assessment (PRA), decision analysis, and expert judgment. Scenarios are constructed to show how the initiating events evolve into undesirable consequences. A value tree, based on multi-attribute utility theory (MAUT), is used to capture the decisionmaker's preferences about the impacts on the infrastructures and other assets. The risks from random failures are ranked according to their expected performance index (PI), which is the product of frequency, probabilities, and consequences of a scenario. Risks from malicious acts are ranked according to their PI as the frequency of attack is not available. A deliberative process is used to capture the factors that could not be addressed in the analysis and to scrutinize the results. This methodology provides a framework for the development of a risk-informed decision strategy. Although this study uses the Massachusetts Institute of Technology campus as a case study of a real project, it is a general methodology that could be used by other similar communities and municipalities.     

Probabilistic Risk Analysis and Terrorism Risk

Since the terrorist attacks of September 11, 2001, and the subsequent establishment of the U.S. Department of Homeland Security (DHS), considerable efforts have been made to estimate the risks of terrorism and the cost effectiveness of security policies to reduce these risks. DHS, industry, and the academic risk analysis communities have all invested heavily in the development of tools and approaches that can assist decisionmakers in effectively allocating limited resources across the vast array of potential investments that could mitigate risks from terrorism and other threats to the homeland. Decisionmakers demand models, analyses, and decision support that are useful for this task and based on the state of the art. Since terrorism risk analysis is new, no single method is likely to meet this challenge. In this article we explore a number of existing and potential approaches for terrorism risk analysis, focusing particularly on recent discussions regarding the applicability of probabilistic and decision analytic approaches to bioterrorism risks and the Bioterrorism Risk Assessment methodology used by the DHS and criticized by the National Academies and others.     

Drivers and Outcomes of Long‐term Orientation in Cooperative Relationships

We argue in this paper that a buyer may deliberately develop long‐term orientation as a governance mechanism to deal with risks arising from exchange hazards, and to reduce the opportunistic behaviour of a supplier. While the exchange hazards of asset specificity pose a safeguarding problem, those of market uncertainty pose an adaptation problem. We test our model on a sample of 221 procurement partnerships. Our results show that satisfactory prior history of a supplier, asset specificity of the buyer and market uncertainty are all positively related to a buyer's long‐term orientation towards a supplier. Consistent with the idea that asset specificity and market uncertainty pose different governance problems, we find that satisfactory prior history reduces the positive relationship between asset specificity and a buyer's long‐term orientation, but enhances the positive relationship between market uncertainty and a buyer's long‐term orientation. We also find that a buyer's long‐term orientation fully mediates the relationship between satisfactory prior history and a supplier's opportunistic behaviour. Finally, implications on the theory and the practices of relationship governance are discussed.     

Deliberation: Integrating Analytical Results into Environmental Decisions Involving Multiple Stakeholders

The National Research Council has recommended the use of an analytic/deliberative decision making process in environmental restoration decisions that involve multiple stakeholders. This work investigates the use of the results of risk assessment and multiattribute utility analysis (the "analysis") in guiding the deliberation. These results include the ranking of proposed remedial action alternatives according to each stakeholder's preferences, as well as the identification of the major reasons for these rankings. The stakeholder preferences are over a number of performance measures that include the traditional risk assessment metrics, e.g., individual worker risk, as well as programmatic, cultural, and cost-related impacts. Based on these results, a number of proposals are prepared for consideration by the stakeholders during the deliberation. These proposals are the starting point for the formulation of actual recommendations by the group. In our case study, these recommendations included new remedial action alternatives that were created by the stakeholders after an extensive discussion of the detailed analytical results.     

On Academic Rankings,Unacceptable Methods,and the Social Obligations of Business Schools

Inspired by recent discussions of the systematic costs that external rankings impose on academic institutions, and the undeniable shifts in the landscape of institutional data, a concerted and pragmatic re‐evaluation of ranking efforts has begun. In this study, multiple administrators and researchers representing both public and private institutions across the United States weigh in on these issues. While reaffirming the social contract we hold with society, we argue that the fundamental methodological shortcomings of existing rankings, and ultimately any ordinal ranking system, limit the value of current rankings. These shortcomings emerge from the conceptualization and the architecture of comparisons, and are evident in survey designs, data collection methods, and data aggregation procedures. Our discussion continues by outlining the minimal requirements that a socially responsible, transparent, flexible, and highly representative rating (vs. ranking) approach should employ. Ultimately, we call on academic institutions and organizing bodies to take a collective stand against existing rankings and to embrace the strategic use of multidimensional alternatives that faithfully serve prospective students, parents, and other key stakeholders. We conclude with a number of suggestions and opportunities for practice‐oriented research in the decision sciences aimed to support this fundamental shift in evaluative framing.     

A Deliberative Method for Ranking Risks (I): Overview and Test Bed Development

Risk ranking offers a potentially powerful means for gathering public input to help set risk-management priorities. In most rankings conducted to date, the categories and attributes used to describe the risks have varied widely, the materials and procedures have not been designed to facilitate comparisons among risks on all important attributes, and the validity and reproducibility of the resulting rankings have not been assessed. To address these needs, a risk-ranking method was developed in which risk experts define and categorize the risks to be ranked, identify the relevant risk attributes, and characterize the risks in a set of standardized risk summary sheets, which are then used by lay or other groups in structured ranking exercises. To evaluate this method, a test bed involving 22 health and safety risks in a fictitious middle school was created. This article provides an overview of the risk-ranking method and describes the challenges faced in designing the middle school test bed. A companion article in this issue reports on the validity of the ranking procedures and the level of agreement among risk managers regarding ranking of risks and attributes.     

Engaging with Comparative Risk Appraisals: Public Views on Policy Priorities for Environmental Risk Governance

Communicating the rationale for allocating resources to manage policy priorities and their risks is challenging. Here, we demonstrate that environmental risks have diverse attributes and locales in their effects that may drive disproportionate responses among citizens. When 2,065 survey participants deployed summary information and their own understanding to assess 12 policy‐level environmental risks singularly, their assessment differed from a prior expert assessment. However, participants provided rankings similar to those of experts when these same 12 risks were considered as a group, allowing comparison between the different risks. Following this, when individuals were shown the prior expert assessment of this portfolio, they expressed a moderate level of confidence with the combined expert analysis. These are important findings for the comprehension of policy risks that may be subject to augmentation by climate change, their representation alongside other threats within national risk assessments, and interpretations of agency for public risk management by citizens and others.     

When Can Scientific Studies Promote Consensus Among Conflicting Stakeholders?

While scientific studies may help conflicting stakeholders come to agreement on a best management option or policy, often they do not. We review the factors affecting trust in the efficacy and objectivity of scientific studies in an analytical‐deliberative process where conflict is present, and show how they may be incorporated in an extension to the traditional Bayesian decision model. The extended framework considers stakeholders who differ in their prior beliefs regarding the probability of possible outcomes (in particular, whether a proposed technology is hazardous), differ in their valuations of these outcomes, and differ in their assessment of the ability of a proposed study to resolve the uncertainty in the outcomes and their hazards—as measured by their perceived false positive and false negative rates for the study. The Bayesian model predicts stakeholder‐specific preposterior probabilities of consensus, as well as pathways for increasing these probabilities, providing important insights into the value of scientific information in an analytic‐deliberative decision process where agreement is sought. It also helps to identify the interactions among perceived risk and benefit allocations, scientific beliefs, and trust in proposed scientific studies when determining whether a consensus can be achieved. The article provides examples to illustrate the method, including an adaptation of a recent decision analysis for managing the health risks of electromagnetic fields from high voltage transmission lines.     

Leveraging the Strengths of Choice Models and Neural Networks: A Multiproduct Comparative Analysis*

Choice models and neural networks are two approaches used in modeling selection decisions. Defining model performance as the out‐of‐sample prediction power of a model, we test two hypotheses: (i) choice models and neural network models are equal in performance, and (ii) hybrid models consisting of a combination of choice and neural network models perform better than each stand‐alone model. We perform statistical tests for two classes of linear and nonlinear hybrid models and compute the empirical integrated rank (EIR) indices to compare the overall performances of the models. We test the above hypotheses by using data for various brand and store choices for three consumer products. Extensive jackknifing and out‐of‐sample tests for four different model specifications are applied for increasing the external validity of the results. Our results show that using neural networks has a higher probability of resulting in a better performance. Our findings also indicate that hybrid models outperform stand‐alone models, in that using hybrid models guarantee overall results equal or better than the two stand‐alone models. The improvement is particularly significant in cases where neither of the two stand‐alone models is very accurate in prediction, indicating that the proposed hybrid models may capture aspects of predictive accuracy that neither stand‐alone model is capable of on their own. Our results are particularly important in brand management and customer relationship management, indicating that multiple technologies and mixture of technologies may yield more accurate and reliable outcomes than individual ones.     

Risk Ranking for Foodborne Microbial Hazards in New Zealand: Burden of Disease Estimates

Priority setting for food safety management at a national level requires risks to be ranked according to defined criteria. In this study, two approaches (disability‐adjusted life years (DALYs) and cost of illness (COI)) were used to generate estimates of the burden of disease for certain potentially foodborne diseases (campylobacteriosis, salmonellosis, listeriosis (invasive, perinatal, and nonperinatal), infection with Shiga toxin‐producing Escherichia coli (STEC), yersiniosis, and norovirus infection) and their sequelae in New Zealand. A modified Delphi approach was used to estimate the food‐attributable proportion for these diseases. The two approaches gave a similar ranking for the selected diseases, with campylobacteriosis and its sequelae accounting for the greatest proportion of the overall burden of disease by far.     

Disaster Risk Management Policies and the Measurement of Resilience for Philippine Regions

How can a government prioritize disaster risk management policies across regions and types of interventions? Using an economic model to assess welfare risk and resilience to disasters, this article systematically tackles the questions: (1) How much asset and welfare risks does each region in the Philippines face from riverine flood disasters? (2) How resilient is each region to riverine flood disasters? (3) What are, per region, the possible interventions to strengthen resilience to riverine flood disasters and what will be their measured benefit? We study the regions of the Philippines to demonstrate the channels through which macroeconomic asset and output losses from disasters translate to consumption and welfare losses at the micro-economic level. Apart from the regional prioritizations, we identify a menu of policy options ranked according to their level of effectiveness in increasing resilience and reducing welfare risk from riverine floods. The ranking of priorities varies for different regions when their level of expected value at risk is different. This suggests that there are region-specific conditions and drivers that need to be integrated into considerations and policy decisions, so that these are effectively addressed.     

Categorizing Risks for Risk Ranking

Any practical process of risk ranking must group hazards into a manageable number of categories. Defining such categories requires value choices that can have important implications for the rankings that result. Most risk-management organizations will find it useful to begin defining categories in terms of environmental loadings or initiating events. However, the resulting categories typically need to be modified in light of other considerations. Risk-ranking projects can benefit from considering several alternative categorization strategies and drawing upon elements of each in developing their final categorization of risks. In principle, conducting multiple ranking exercises by using different categorizations could be interesting and useful. In practice, agencies are unlikely to have either the resources or patience to do this, but other groups in society might. Done well, such additional independent rankings could add valuable inputs to democratic risk-management decision making.