Ecological Risk Ranking: Development and Evaluation of a Method for Improving Public Participation in Environmental Decision Making

This article reports an extension of the Carnegie Mellon risk-ranking method to incorporate ecological risks and their attributes. On the basis of earlier risk-perception studies, we identified a set of 20 relevant attributes for describing health, safety, and environmental hazards in standardized risk summary sheets. In a series of three ranking sessions, 23 laypeople ranked 10 such hazards in a fictional Midwestern U.S. county using both holistic and multiattribute ranking procedures. Results were consistent with those from previous studies involving only health and safety hazards, providing additional evidence for the validity of the method and the replicability of the resulting rankings. Holistic and multiattribute risk rankings were reasonably consistent both for individuals and for groups. Participants reported that they were satisfied with the procedures and results, and indicated their support for using the method to advise real-world risk-management decisions. Agreement among participants increased over the course of the exercise, perhaps because the materials and deliberations helped participants to correct their misconceptions and clarify their values. Overall, health and safety attributes were judged more important than environmental attributes. However, the overlap between the importance rankings of these two sets of attributes suggests that some information about environmental impacts is important to participants' judgments in comparative risk-assessment tasks.     

A Deliberative Method for Ranking Risks (II): Evaluation of Validity and Agreement among Risk Managers

A deliberative method for ranking risks was evaluated in a study involving 218 risk managers. Both holistic and multiattribute procedures were used to assess individual and group rankings of health and safety risks facing students at a fictitious middle school. Consistency between the rankings that emerged from these two procedures was reasonably high for individuals and for groups, suggesting that these procedures capture an underlying construct of riskiness. Participants reported high levels of satisfaction with their groups' decision-making processes and the resulting rankings, and these reports were corroborated by regression analyses. Risk rankings were similar across individuals and groups, even though individuals and groups did not always agree on the relative importance of risk attributes. Lower consistency between the risk rankings from the holistic and multiattribute procedures and lower agreement among individuals and groups regarding these rankings were observed for a set of high-variance risks. Nonetheless, the generally high levels of consistency, satisfaction, and agreement suggest that this deliberative method is capable of producing risk rankings that can serve as informative inputs to public risk-management decision making.     

Personal Values, Beliefs, and Ecological Risk Perception

A mail survey on ecological risk perception was administered in the summer of 2002 to a randomized sample of the lay public and to selected risk professionals at the U.S. Environmental Protection Agency (US EPA). The ranking of 24 ecological risk items, from global climate change to commercial fishing, reveals that the lay public is more concerned about low-probability, high-consequence risks whereas the risk professionals are more concerned about risks that pose long-term, ecosystem-level impacts. To test the explanatory power of the value-belief-norm (VBN) theory for risk perception, respondents were questioned about their personal values, spiritual beliefs, and worldviews. The most consistent predictors of the risk rankings are belief in the new ecological paradigm (NEP) and Schwartz's altruism. The NEP and Schwartz's altruism explain from 19% to 46% of the variance in the risk rankings. Religious beliefs account for less than 6% of the variance and do not show a consistent pattern in predicting risk perception although religious fundamentalists are generally less concerned about the risk items. While not exerting as strong an impact, social-structural variables do have some influence on risk perception. Ethnicities show no effect on the risk scales but the more educated and financially well-off are less concerned about the risk items. Political leanings have no direct influence on risk rankings, but indirectly affect rankings through the NEP. These results reveal that the VBN theory is a plausible explanation for the differences measured in the respondents' perception of ecological risk.     

Setting Risk Priorities: A Formal Model

This article presents a model designed to capture the major aspects of setting priorities among risks, a common task in government and industry. The model has both design features, under the control of the rankers (e.g., how success is evaluated), and context features, properties of the situations that they are trying to understand (e.g., how quickly uncertainty can be reduced). The model is demonstrated in terms of two extreme ranking strategies. The first, sequential risk ranking , devotes all its resources, in a given period, to learning more about a single risk, and its place in the overall ranking. This strategy characterizes the process for a society (or organization or individual) that throws itself completely into dealing with one risk after another. The other extreme strategy, simultaneous risk ranking , spreads available resources equally across all risks. It characterizes the most methodical of ranking exercises. Given ample ranking resources, simultaneous risk ranking will eventually provide an accurate set of priorities, whereas sequential ranking might never get to some risks. Resource constraints, however, may prevent simultaneous rankers from examining any risk very thoroughly. The model is intended to clarify the nature of ranking tasks, predict the efficacy of alternative strategies, and improve their design.     

Categorizing Risks for Risk Ranking

Any practical process of risk ranking must group hazards into a manageable number of categories. Defining such categories requires value choices that can have important implications for the rankings that result. Most risk-management organizations will find it useful to begin defining categories in terms of environmental loadings or initiating events. However, the resulting categories typically need to be modified in light of other considerations. Risk-ranking projects can benefit from considering several alternative categorization strategies and drawing upon elements of each in developing their final categorization of risks. In principle, conducting multiple ranking exercises by using different categorizations could be interesting and useful. In practice, agencies are unlikely to have either the resources or patience to do this, but other groups in society might. Done well, such additional independent rankings could add valuable inputs to democratic risk-management decision making.     

A Scale of Risk

This article proposes a conceptual framework for ranking the relative gravity of diverse risks. This framework identifies the moral considerations that should inform the evaluation and comparison of diverse risks. A common definition of risk includes two dimensions: the probability of occurrence and the associated consequences of a set of hazardous scenarios. This article first expands this definition to include a third dimension: the source of a risk. The source of a risk refers to the agents involved in the creation or maintenance of a risk and captures a central moral concern about risks. Then, a scale of risk is proposed to categorize risks along a multidimensional ranking, based on a comparative evaluation of the consequences, probability, and source of a given risk. A risk is ranked higher on the scale the larger the consequences, the greater the probability, and the more morally culpable the source. The information from the proposed comparative evaluation of risks can inform the selection of priorities for risk mitigation.     

Prioritizing Environmental Health Risks in the UAE

This article presents the results of a comparative environmental risk‐ranking exercise that was conducted in the United Arab Emirates (UAE) to inform a strategic planning process led by the Environment Agency‐Abu Dhabi (EAD). It represents the first national‐level application of a deliberative method for comparative risk ranking first published in this journal. The deliberative method involves a five‐stage process that includes quantitative risk assessment by experts and deliberations by groups of stakeholders. The project reported in this article considered 14 categories of environmental risks to health identified through discussions with EAD staff: ambient and indoor air pollution; drinking water contamination; coastal water pollution; soil and groundwater contamination; contamination of fruits, vegetables, and seafood; ambient noise; stratospheric ozone depletion; electromagnetic fields from power lines; health impacts from climate change; and exposure to hazardous substances in industrial, construction, and agricultural work environments. Results from workshops involving 73 stakeholders who met in five separate groups to rank these risks individually and collaboratively indicated strong consensus that outdoor and indoor air pollution are the highest priorities in the UAE. Each of the five groups rated these as being among the highest risks. All groups rated soil and groundwater contamination as being among the lowest risks. In surveys administered after the ranking exercises, participants indicated that the results of the process represented their concerns and approved of using the ranking results to inform policy decisions. The results ultimately shaped a strategic plan that is now being implemented.     

Augmenting the Deliberative Method for Ranking Risks

The Department of Homeland Security (DHS) characterized and prioritized the physical cross‐border threats and hazards to the nation stemming from terrorism, market‐driven illicit flows of people and goods (illegal immigration, narcotics, funds, counterfeits, and weaponry), and other nonmarket concerns (movement of diseases, pests, and invasive species). These threats and hazards pose a wide diversity of consequences with very different combinations of magnitudes and likelihoods, making it very challenging to prioritize them. This article presents the approach that was used at DHS to arrive at a consensus regarding the threats and hazards that stand out from the rest based on the overall risk they pose. Due to time constraints for the decision analysis, it was not feasible to apply multiattribute methodologies like multiattribute utility theory or the analytic hierarchy process. Using a holistic approach was considered, such as the deliberative method for ranking risks first published in this journal. However, an ordinal ranking alone does not indicate relative or absolute magnitude differences among the risks. Therefore, the use of the deliberative method for ranking risks is not sufficient for deciding whether there is a material difference between the top‐ranked and bottom‐ranked risks, let alone deciding what the stand‐out risks are. To address this limitation of ordinal rankings, the deliberative method for ranking risks was augmented by adding an additional step to transform the ordinal ranking into a ratio scale ranking. This additional step enabled the selection of stand‐out risks to help prioritize further analysis.     

Engaging with Comparative Risk Appraisals: Public Views on Policy Priorities for Environmental Risk Governance

Communicating the rationale for allocating resources to manage policy priorities and their risks is challenging. Here, we demonstrate that environmental risks have diverse attributes and locales in their effects that may drive disproportionate responses among citizens. When 2,065 survey participants deployed summary information and their own understanding to assess 12 policy‐level environmental risks singularly, their assessment differed from a prior expert assessment. However, participants provided rankings similar to those of experts when these same 12 risks were considered as a group, allowing comparison between the different risks. Following this, when individuals were shown the prior expert assessment of this portfolio, they expressed a moderate level of confidence with the combined expert analysis. These are important findings for the comprehension of policy risks that may be subject to augmentation by climate change, their representation alongside other threats within national risk assessments, and interpretations of agency for public risk management by citizens and others.     

Are Some Risk Comparisons More Effective Under Conflict?: A Replication and Extension of Roth et al.

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Only one study, published by Roth et al. in this journal in 1990, has tested the 1988 predictions by Covello et al. as to the public's relative preferences for 14 kinds of risk comparisons as they might be used by a factory manager to explain risks of his ethylene oxide plant. That study found no correlations between the Covello predictions and seven different measures of "acceptability" of Covello's examples of each type of comparison. However, two critics of the Roth study, as well as its own authors, suggested that a scenario involving local risks, a conflict-ridden situation, and a plant manager unknown to the townspeople might better evoke Covello-like preferences than the distant, calm, friends-involving scenario used by Roth. The research reported here replicated the Roth study using the same scenario, risk comparison examples, and evaluation measures, and added a second scenario intended to replicate the conditions suggested by critics. Over 200 New Jersey residents answered the study questionnaire. The replication scenario reproduced Roth's results, and the conflict scenario also evoked no rankings correlated with Covello's predictions. Furthermore, neither agreement nor disagreement with five statements representing "conflict"--respondents' reports that the industrial-plant scenario made them angry, they lived near industry, they were concerned about industrial risks, people in their home town were angry about industrial pollution, and they worried "frequently" about long-term effects of pollution--correlated with Covello's predictions. Over half of all ratings ascribed to the comparisons in aggregate were positive, and most detailed comments offered by respondents also were positive, despite many criticisms and suggestions for their improvement. The wide variability in individuals' rankings also undermines the notion of any single ranking of preferred comparisons. These findings have implications for use of risk comparisons, but also reveal the inaccuracy of the field's assumptions about public reaction to industrial risk information, including risk comparison.     

A Risk-Based Approach to Setting Priorities in Protecting Bridges Against Terrorist Attacks

This article presents an approach to the problem of terrorism risk assessment and management by adapting the framework of the risk filtering, ranking, and management method. The assessment is conducted at two levels: (1) the system level, and (2) the asset-specific level. The system-level risk assessment attempts to identify and prioritize critical infrastructures from an inventory of system assets. The definition of critical infrastructures offered by Presidential Decision Directive 63 was used to determine the set of attributes to identify critical assets--categorized according to national, regional, and local impact. An example application is demonstrated using information from the Federal Highway Administration National Bridge Inventory for the State of Virginia. Conversely, the asset-specific risk assessment performs an in-depth analysis of the threats and vulnerabilities of a specific critical infrastructure. An illustration is presented to offer some insights in risk scenario identification and prioritization, multiobjective evaluation of management options, and extreme-event analysis for critical infrastructure protection.     

Context‐Specific,Scenario‐Based Risk Scales

Reacting to an emergency requires quick decisions under stressful and dynamic conditions. To react effectively, responders need to know the right actions to take given the risks posed by the emergency. While existing research on risk scales focuses primarily on decision making in static environments with known risks, these scales may be inappropriate for conditions where the decision maker's time and mental resources are limited and may be infeasible if the actual risk probabilities are unknown. In this article, we propose a method to develop context‐specific, scenario‐based risk scales designed for emergency response training. Emergency scenarios are used as scale points, reducing our dependence on known probabilities; these are drawn from the targeted emergency context, reducing the mental resources required to interpret the scale. The scale is developed by asking trainers/trainees to rank order a range of risk scenarios and then aggregating these orderings using a Kemeny ranking. We propose measures to assess this aggregated scale's internal consistency, reliability, and validity, and we discuss how to use the scale effectively. We demonstrate our process by developing a risk scale for subsurface coal mine emergencies and test the reliability of the scale by repeating the process, with some methodological variations, several months later.     

A Score Comparison Method as an Aid to Integrating Separate Comparative Risk Rankings into a Single, Comparative Risk Ranking

A Score Comparison Method (SCM), for use in comparative risk projects, is described. It provides a degree of analytical guidance for those undertaking to integrate environmental issues which have been placed into separate, qualitative rankings according to different types of risk into a single, qualitative, integrated risk ranking. Its use in an actual case is shown.     

Investigating Heterogeneity in the Characterization of Risks Using Best Worst Scaling

This research proposes and implements a new approach to the elicitation and analysis of perceptions of risk. We use best worst scaling (BWS) to elicit the levels of control respondents believe they have over risks and the level of concern those risks prompt. The approach seeks perceptions of control and concern over a large risk set and the elicitation method is structured so as to reduce the cognitive burden typically associated with ranking over large sets. The BWS approach is designed to yield strong discrimination over items. Further, the approach permits derivation of individual‐level values, in this case of perceptions of control and worry, and analysis of how these vary over observable characteristics, through estimation of random parameter logit models. The approach is implemented for a set of 20 food and nonfood risks. The results show considerable heterogeneity in perceptions of control and worry, that the degree of heterogeneity varies across the risks, and that women systematically consider themselves to have less control over the risks than men.     

Risk Comparisons, Conflict, and Risk Acceptability Claims

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Even fewer have examined the public's relative preferences among different kinds of risk comparisons. Two studies, published in this journal in 1990 and 2003, used seven measures of "acceptability" to examine public reaction to 14 examples of risk comparisons, as used by a hypothetical factory manager to explain risks of his ethylene oxide plant. This study examined the effect on preferences of scenarios involving low or high conflict between the factory manager and residents of the hypothetical town (as had the 2003 study), and inclusion of a claim that the comparison demonstrated the risks' acceptability. It also tested the Finucane et al. (2000) affect hypothesis that information emphasizing low risks-as in these risk comparisons-would raise benefits estimates without changing risk estimates. Using similar but revised scenarios, risk comparison examples (10 instead of 14), and evaluation measures, an opportunity sample of 303 New Jersey residents rated the comparisons, and the risks and benefits of the factory. On average, all comparisons received positive ratings on all evaluation measures in all conditions. Direct and indirect measures showed that the conflict manipulation worked; overall, No-Conflict and Conflict scenarios evoked scores that were not significantly different. The attachment to each risk comparison of a risk acceptability claim ("So our factory's risks should be acceptable to you.") did not worsen ratings relative to conditions lacking this claim. Readers who did or did not see this claim were equally likely to infer an attempt to persuade them to accept the risk from the comparison. As in the 2003 article, there was great individual variability in inferred rankings of the risk comparisons. However, exposure to the risk comparisons did not reduce risk estimates significantly (while raising benefit estimates), and Conflict-Claim respondents found the risk of the hypothetical factory less acceptable than No-Conflict respondents. Results suggest that neither risk comparisons nor risk acceptability claims are automatically anathema to audiences, but they may have tiny or unintended effects on audience judgments about risky situations.     

Evaluation of Multicriteria Decision Analysis Algorithms in Food Safety: A Case Study on Emerging Zoonoses Prioritization

Decision making in food safety is a complex process that involves several criteria of different nature like the expected reduction in the number of illnesses, the potential economic or health-related cost, or even the environmental impact of a given policy or intervention. Several multicriteria decision analysis (MCDA) algorithms are currently used, mostly individually, in food safety to rank different options in a multifactorial environment. However, the selection of the MCDA algorithm is a decision problem on its own because different methods calculate different rankings. The aim of this study was to compare the impact of different uncertainty sources on the rankings of MCDA problems in the context of food safety. For that purpose, a previously published data set on emerging zoonoses in the Netherlands was used to compare different MCDA algorithms: MMOORA, TOPSIS, VIKOR, WASPAS, and ELECTRE III. The rankings were calculated with and without considering uncertainty (using fuzzy sets), to assess the importance of this factor. The rankings obtained differed between algorithms, emphasizing that the selection of the MCDA method had a relevant impact in the rankings. Furthermore, considering uncertainty in the ranking had a high influence on the results. Both factors were more relevant than the weights associated with each criterion in this case study. A hierarchical clustering method was suggested to aggregate results obtained by the different algorithms. This complementary step seems to be a promising way to decrease extreme difference among algorithms and could provide a strong added value in the decision-making process.     

Aggregate, Disaggregate, and Hybrid Analyses of Ecological Risk Perceptions

Laypeople's perceptions of health and safety risks have been widely studied, but only a few studies have addressed perceptions of ecological hazards. We assembled a list of 39 attributes of ecological hazards from the literatures on comparative risk assessment, ecological health, environmental conservation and management, environmental psychology, and risk perception. In Study 1, 125 laypeople evaluated 83 hazards on subsets of this attribute set. Factor analysis of attribute ratings (averaged over participants) revealed six oblique factors: ecological impacts, human impacts, human benefits, aesthetic impacts, scientific understanding, and controllability. These factors predicted mean judgments of overall riskiness, ecological riskiness, acceptability, and regulatory strictness. In Study 2, 30 laypeople each evaluated 34 hazards on 17 attributes and 3 dependent variables. Aggregate-level factor analysis of these data replicated the appropriate portion of the factor solution and yielded similar regression results. Parallel analyses at the individual-participant level yielded factors that explained less variance in judgments of overall riskiness, ecological riskiness, and acceptability. However, the decrease in explanatory power was much less than is often reported for disaggregate-level analyses of psychometric data. This discrepancy illustrates the importance of distinguishing between the level of analysis (aggregate versus disaggregate) and the focus of analysis (distinctions among hazards versus distinctions among participants). In a hybrid analysis, aggregate-level factor scores predicted individual participants' riskiness judgments reasonably well. Psychometric studies such as these provide a sound empirical basis for selecting attributes of ecological hazards for use in comparative risk assessment.     

A New Multicriteria Risk Mapping Approach Based on a Multiattribute Frontier Concept

Invasive species risk maps provide broad guidance on where to allocate resources for pest monitoring and regulation, but they often present individual risk components (such as climatic suitability, host abundance, or introduction potential) as independent entities. These independent risk components are integrated using various multicriteria analysis techniques that typically require prior knowledge of the risk components’ importance. Such information is often nonexistent for many invasive pests. This study proposes a new approach for building integrated risk maps using the principle of a multiattribute efficient frontier and analyzing the partial order of elements of a risk map as distributed in multidimensional criteria space. The integrated risks are estimated as subsequent multiattribute frontiers in dimensions of individual risk criteria. We demonstrate the approach with the example of Agrilus biguttatus Fabricius, a high‐risk pest that may threaten North American oak forests in the near future. Drawing on U.S. and Canadian data, we compare the performance of the multiattribute ranking against a multicriteria linear weighted averaging technique in the presence of uncertainties, using the concept of robustness from info‐gap decision theory. The results show major geographic hotspots where the consideration of tradeoffs between multiple risk components changes integrated risk rankings. Both methods delineate similar geographical regions of high and low risks. Overall, aggregation based on a delineation of multiattribute efficient frontiers can be a useful tool to prioritize risks for anticipated invasive pests, which usually have an extremely poor prior knowledge base.     

Nano Risk Analysis: Advancing the Science for Nanomaterials Risk Management

Scientists, activists, industry, and governments have raised concerns about health and environmental risks of nanoscale materials. The Society for Risk Analysis convened experts in September 2008 in Washington, DC to deliberate on issues relating to the unique attributes of nanoscale materials that raise novel concerns about health risks. This article reports on the overall themes and findings of the workshop, uncovering the underlying issues for each of these topics that become recurring themes. The attributes of nanoscale particles and other nanomaterials that present novel issues for risk analysis are evaluated in a risk analysis framework, identifying challenges and opportunities for risk analysts and others seeking to assess and manage the risks from emerging nanoscale materials and nanotechnologies. Workshop deliberations and recommendations for advancing the risk analysis and management of nanotechnologies are presented.