What's Wrong with Risk Matrices?

Risk matrices—tables mapping "frequency" and "severity" ratings to corresponding risk priority levels—are popular in applications as diverse as terrorism risk analysis, highway construction project management, office building risk analysis, climate change risk management, and enterprise risk management (ERM). National and international standards (e.g., Military Standard 882C and AS/NZS 4360:1999) have stimulated adoption of risk matrices by many organizations and risk consultants. However, little research rigorously validates their performance in actually improving risk management decisions. This article examines some mathematical properties of risk matrices and shows that they have the following limitations. (a) Poor Resolution . Typical risk matrices can correctly and unambiguously compare only a small fraction (e.g., less than 10%) of randomly selected pairs of hazards. They can assign identical ratings to quantitatively very different risks ("range compression"). (b) Errors . Risk matrices can mistakenly assign higher qualitative ratings to quantitatively smaller risks. For risks with negatively correlated frequencies and severities, they can be "worse than useless," leading to worse-than-random decisions. (c) Suboptimal Resource Allocation . Effective allocation of resources to risk-reducing countermeasures cannot be based on the categories provided by risk matrices. (d) Ambiguous Inputs and Outputs . Categorizations of severity cannot be made objectively for uncertain consequences. Inputs to risk matrices (e.g., frequency and severity categorizations) and resulting outputs (i.e., risk ratings) require subjective interpretation, and different users may obtain opposite ratings of the same quantitative risks. These limitations suggest that risk matrices should be used with caution, and only with careful explanations of embedded judgments.     

How to Design Rating Schemes of Risk Matrices: A Sequential Updating Approach

Risk matrices have been widely used as a risk evaluation tool in many fields due to their simplicity and intuitive nature. Designing a rating scheme, i.e., determining the number of ratings used in a risk matrix and assigning different ratings to different cells, is an essential part of risk matrix construction. However, most of the related literature has focused on applying a risk matrix to various fields, instead of researching how to design risk matrices. Based on the analysis of several current rules, we propose a new approach, namely, the sequential updating approach (SUA), to design the rating scheme of a risk matrix in a reliable way. In this article, we propose three principles and a rating algorithm based on these principles. The three principles, namely, adjusted weak consistency, consistent internality, and continuous screening, characterize a good rating scheme. The resulting rating scheme has been proven to be unique. A global rating algorithm is then proposed to create the design that satisfies the three principles. We then explore the performance of the SUA. An illustrative application is first given to explain the feasibility of our approach. The sensitivity analysis shows that our method captures a resolution‐reliability tradeoff for decisionmakers in choosing an appropriate rating scheme for a risk matrix. Finally, we compare the designs based on the SUA and Cox's axioms, highlighting the advantages of the SUA.     

What's Wrong with Hazard‐Ranking Systems? An Expository Note

Two commonly recommended principles for allocating risk management resources to remediate uncertain hazards are: (1) select a subset to maximize risk-reduction benefits (e.g., maximize the von Neumann-Morgenstern expected utility of the selected risk-reducing activities), and (2) assign priorities to risk-reducing opportunities and then select activities from the top of the priority list down until no more can be afforded. When different activities create uncertain but correlated risk reductions, as is often the case in practice, then these principles are inconsistent: priority scoring and ranking fails to maximize risk-reduction benefits. Real-world risk priority scoring systems used in homeland security and terrorism risk assessment, environmental risk management, information system vulnerability rating, business risk matrices, and many other important applications do not exploit correlations among risk-reducing opportunities or optimally diversify risk-reducing investments. As a result, they generally make suboptimal risk management recommendations. Applying portfolio optimization methods instead of risk prioritization ranking, rating, or scoring methods can achieve greater risk-reduction value for resources spent.     

Lay Understanding of Synergistic Risk: The Case of Radon and Cigarette Smoking

The combination of radon and smoking produces a synergistic risk of lung cancer. Lay understanding of this risk was examined from the perspectives of mental models theory, the psychometric approach to risk perception, and optimistic bias. As assessed by interview, participants ( N = 50) had more extensive mental models for the risks of smoking than for the risks of radon or the combination of radon and smoking; 32% knew little or nothing about radon. Despite reading an informational brochure, their risk-perception ratings of the three hazards showed no perception of the synergy between smoking and radon risk, although the combined hazard was rated as less familiar but more controllable than the average of the single hazards ( p < .01). No evidence of optimistic bias for the health consequences of radon, or the combination of radon and smoking was observed. Participants appeared to be combining the single-hazard risks subadditively to arrive at their combined-hazard risk perceptions. Further research on the integration of perceived risks would be beneficial for designing optimal communications about synergistic risk.     

Assessing Perceptions of Synergistic Health Risk: A Comparison of Two Scales

Two approaches to measuring perceptions of synergistic risk were compared, one using the traditional Likert scale, the other using an anchored, relative scale. Perception of synergistic risk was defined as rating the combined hazard as more risky than each of its constituent single hazards. In a within-subjects design, a convenience sample from the community (N= 604) rated three hazard combinations and their constituents: Driving while Intoxicated (familiar, high synergy), Radon and Smoking (unfamiliar, high synergy), and Smoking and Driving (familiar, low synergy), on both scales. The relative scale was expected to be a more sensitive measure of synergy than the Likert scale. The effects of item order (single hazards rated first versus combined hazards rated first) were examined between subjects. Driving while Intoxicated was perceived by the large majority of participants as a synergistic risk on both scales, but neither of the other two combined hazards were rated synergistically on either scale. The relative scale demonstrated a slight advantage over the Likert scale, and presenting the single hazards first for the relative scale produced more synergistic ratings. It is recommended that anchored, relative scales be used to measure synergy and that single hazards be presented prior to the combined hazards when using relative scales.     

Cancer Anxiety and the Perception of Risk in Alarmed Communities

Cancer is a significant public health topic and is frequently a factor in public reactions to environmental hazards. It may be reasonable to suggest that a unique form of health anxiety exists - one specific to cancer. In this article, we explore a measure of cancer anxiety that has applicability to risk perception in the specific context of communities that are alarmed over suspect cancer rates thought to be associated with environmental hazards. A literature search was used to identify survey questions previously used to measure cancer worry, fear, anxiety, etc. A list of 24 items was employed in a mail survey sent to 30 communities in which cancer rates were under investigation (part of a broader study). An analysis of the dataset of 1,111 responses yields two versions of a cancer anxiety scale: one an additive combination of 21 items (alpha= 0.77) and the other a two-factor model consisting of nine- and four-item subscales (alpha= 0.74 and 0.69). The resulting scales are evaluated for their ability to predict perception of risk from the environmental hazards in these cases. Controlling for age, sex, and cancer status, the scales explain between 2% and 10% of risk perception (full R(2) values ranging from 0.17 to 0.24). Given the range of concepts required to model risk perception, we conclude that this measure of cancer anxiety is sufficiently reliable and robust to be recommended for use in circumstances involving hazards associated with cancer. Other uses and further development of the measure are discussed.     

The Significance of Socioeconomic and Ethnic Diversity for the Risk Communication Process

Risk communication is being characterized as one way of facilitating more effective, democratic and participatory risk management strategies. An emphasis on formal communication approaches as a means to improve decisions and decrease conflict will highlight the challenge of managing hazards within a culturally heterogeneous society. Communication and participatory strategies will be considered successful only if diverse communities can be engaged as partners in the policy process. Because responses to risks are embedded and evolve within broader social environments, achieving the promise of risk communication across a diverse society may not be possible absent an understanding of how sociocultural variables and past experiences shape the exchange of ideas or information in any particular situation. This paper considers the implications of ethnic and socioeconomic variability for the risk communication process, summarizing theoretical perspectives and empirical evidence on the link between sociocultural features and risk responses. Specifically, the factors that define the context of communication may influence: the initial framing of a risk issue, particularly, the adoption of an environmental justice vs. scientific/economic perspective; the perceived importance of various aspects of the decision problem; and prior beliefs about environmental hazards and agencies involved in risk management. Two examples of situations requiring communications about risk are presented and illustrate how these principles could operate in minority or lower-income communities. A significant challenge for health and regulatory officials will be to engage in an interactive process of information and opinion exchanges that is reasonable and effective within vastly different socioeconomic and cultural contexts.     

Some Limitations of Qualitative Risk Rating Systems

Qualitative systems for rating animal antimicrobial risks using ordered categorical labels such as “high,”“medium,” and “low” can potentially simplify risk assessment input requirements used to inform risk management decisions. But do they improve decisions? This article compares the results of qualitative and quantitative risk assessment systems and establishes some theoretical limitations on the extent to which they are compatible. In general, qualitative risk rating systems satisfying conditions found in real‐world rating systems and guidance documents and proposed as reasonable make two types of errors: (1) Reversed rankings, i.e., assigning higher qualitative risk ratings to situations that have lower quantitative risks; and (2) Uninformative ratings, e.g., frequently assigning the most severe qualitative risk label (such as “high”) to situations with arbitrarily small quantitative risks and assigning the same ratings to risks that differ by many orders of magnitude. Therefore, despite their appealing consensus‐building properties, flexibility, and appearance of thoughtful process in input requirements, qualitative rating systems as currently proposed often do not provide sufficient information to discriminate accurately between quantitatively small and quantitatively large risks. The value of information (VOI) that they provide for improving risk management decisions can be zero if most risks are small but a few are large, since qualitative ratings may then be unable to confidently distinguish the large risks from the small. These limitations suggest that it is important to continue to develop and apply practical quantitative risk assessment methods, since qualitative ones are often unreliable.     

Taxonomic Analysis of Perceived Risk: Modeling Individual and Group Perceptions Within Homogeneous Hazard Domains

Previous studies of risk perception have typically focused on the mean judgments of a group of people regarding the riskiness (or safety) of a diverse set of hazardous activities, substances, and technologies. This paper reports the results of two studies that take a different path. Study 1 investigated whether models within a single technological domain were similar to previous models based on group means and diverse hazards. Study 2 created a group taxonomy of perceived risk for only one technological domain, railroads, and examined whether the structure of that taxonomy corresponded with taxonomies derived from prior studies of diverse hazards. Results from Study 1 indicated that the importance of various risk characteristics in determining perceived risk differed across individuals and across hazards, but not so much as to invalidate the results of earlier studies based on group means and diverse hazards. In Study 2, the detailed analysis of railroad hazards produced a structure that had both important similarities to, and dissimilarities from, the structure obtained in prior research with diverse hazard domains. The data also indicated that railroad hazards are really quite diverse, with some approaching nuclear reactors in their perceived seriousness. These results suggest that information about the diversity of perceptions within a single domain of hazards could provide valuable input to risk-management decisions.     

Public Perceptions of the Potential Hazards Associated with Food Production and Food Consumption: An Empirical Study

Although public perceptions of food-related hazards receive much media comment and debate, the research literature on such perceptions is sparse and piecemeal. In the reported study, 216 people completed a questionnaire relating to their perceptions of the "risk characteristics" of potential hazards associated with various aspects of food production and food consumption. Responses were examined via principal-components analysis to obtain a structural representation of risk perception of the kind provided by Fischhoff, Slovic, and their colleagues in their seminal psychometric work.^(1,2) A three-component solution accounting for 87% of the variance was obtained, with the dimensions labeled as "severity,""unknown," and "number of people exposed." The findings also yielded information pointing to evidence of the phenomenon of unrealistic optimism. We conclude that our findings offer a useful base upon which further in-depth research integrating different perspectives on risk perception with respect to food-related hazards may be developed.     

Risk Comparisons, Conflict, and Risk Acceptability Claims

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Even fewer have examined the public's relative preferences among different kinds of risk comparisons. Two studies, published in this journal in 1990 and 2003, used seven measures of "acceptability" to examine public reaction to 14 examples of risk comparisons, as used by a hypothetical factory manager to explain risks of his ethylene oxide plant. This study examined the effect on preferences of scenarios involving low or high conflict between the factory manager and residents of the hypothetical town (as had the 2003 study), and inclusion of a claim that the comparison demonstrated the risks' acceptability. It also tested the Finucane et al. (2000) affect hypothesis that information emphasizing low risks-as in these risk comparisons-would raise benefits estimates without changing risk estimates. Using similar but revised scenarios, risk comparison examples (10 instead of 14), and evaluation measures, an opportunity sample of 303 New Jersey residents rated the comparisons, and the risks and benefits of the factory. On average, all comparisons received positive ratings on all evaluation measures in all conditions. Direct and indirect measures showed that the conflict manipulation worked; overall, No-Conflict and Conflict scenarios evoked scores that were not significantly different. The attachment to each risk comparison of a risk acceptability claim ("So our factory's risks should be acceptable to you.") did not worsen ratings relative to conditions lacking this claim. Readers who did or did not see this claim were equally likely to infer an attempt to persuade them to accept the risk from the comparison. As in the 2003 article, there was great individual variability in inferred rankings of the risk comparisons. However, exposure to the risk comparisons did not reduce risk estimates significantly (while raising benefit estimates), and Conflict-Claim respondents found the risk of the hypothetical factory less acceptable than No-Conflict respondents. Results suggest that neither risk comparisons nor risk acceptability claims are automatically anathema to audiences, but they may have tiny or unintended effects on audience judgments about risky situations.     

Aggregate, Disaggregate, and Hybrid Analyses of Ecological Risk Perceptions

Laypeople's perceptions of health and safety risks have been widely studied, but only a few studies have addressed perceptions of ecological hazards. We assembled a list of 39 attributes of ecological hazards from the literatures on comparative risk assessment, ecological health, environmental conservation and management, environmental psychology, and risk perception. In Study 1, 125 laypeople evaluated 83 hazards on subsets of this attribute set. Factor analysis of attribute ratings (averaged over participants) revealed six oblique factors: ecological impacts, human impacts, human benefits, aesthetic impacts, scientific understanding, and controllability. These factors predicted mean judgments of overall riskiness, ecological riskiness, acceptability, and regulatory strictness. In Study 2, 30 laypeople each evaluated 34 hazards on 17 attributes and 3 dependent variables. Aggregate-level factor analysis of these data replicated the appropriate portion of the factor solution and yielded similar regression results. Parallel analyses at the individual-participant level yielded factors that explained less variance in judgments of overall riskiness, ecological riskiness, and acceptability. However, the decrease in explanatory power was much less than is often reported for disaggregate-level analyses of psychometric data. This discrepancy illustrates the importance of distinguishing between the level of analysis (aggregate versus disaggregate) and the focus of analysis (distinctions among hazards versus distinctions among participants). In a hybrid analysis, aggregate-level factor scores predicted individual participants' riskiness judgments reasonably well. Psychometric studies such as these provide a sound empirical basis for selecting attributes of ecological hazards for use in comparative risk assessment.     

Out of Sight,Out of Mind? Geographic and Social Predictors of Flood Risk Awareness

The persistent gap in flood risk awareness in Canada, and elsewhere in North America, is a continual source of worry for researchers and emergency managers; many people living in at‐risk places are simply unaware of risks and of their proximity to hazards. This study seeks to understand which residents were aware of flood risk, using unique representative survey data of Calgary residents living in the city's flood‐prone neighborhoods collected after the devastating and costly 2013 Southern Alberta Flood. The article uses logistic regression models to analyze which residents were aware of risk to their homes. Findings indicate that, in addition to various demographic predictors, many of the geographic predictors (including the elevation of one's home relative to the river) are significant predictors of awareness. Having a direct sight line to one of Calgary's two rivers is also a significant predictor in some of the models, suggesting that the visibility of hazards matters for flood risk perception, although this effect fades when many of the geographic predictors are added. Finally, the models indicate that several variables related to local, neighborhood‐based social networks are significant as well. These findings reveal that both physical surroundings and social context are important for understanding risk awareness. The article concludes by discussing the relevance for social science research on disasters and hazards, as well as for planners and emergency managers.     

Interactive Hazards Education Program for Youth in a Low SES Community: A Quasi‐Experimental Pilot Study

A pilot study of an interactive hazards education program was carried out in Canberra (Australia), with direct input from youth participants. Effects were evaluated in relation to youths’ interest in disasters, motivation to prepare, risk awareness, knowledge indicators, perceived preparedness levels, planning and practice for emergencies, and fear and anxiety indicators. Parents also provided ratings, including of actual home‐based preparedness activities. Using a single group pretest‐posttest with benchmarking design, a sample of 20 youths and their parents from a low SES community participated. Findings indicated beneficial changes on a number of indicators. Preparedness indicators increased significantly from pre‐ to posttest on both youth (p < 0.01) and parent ratings (p < 0.01). Parent ratings reflected an increase of just under six home‐based preparedness activities. Youth knowledge about disaster mitigation also was seen to increase significantly (p < 0.001), increasing 39% from pretest levels. While personalized risk perceptions significantly increased (p < 0.01), anxiety and worry levels were seen either not to change (generalized anxiety, p > 0.05) or to reduce between pre‐ and posttest (hazards‐specific fears, worry, and distress, ps ranged from p < 0.05 to < 0.001). In terms of predictors of preparedness, a number of variables were found to predict posttest preparedness levels, including information searching done by participants between education sessions. These pilot findings are the first to reflect quasi‐experimental outcomes for a youth hazards education program carried out in a setting other than a school that focused on a sample of youth from a low SES community.     

Agency Communication, Community Outrage, and Perception of Risk: Three Simulation Experiments

Three experimental studies were conducted employing hypothetical news stories to compare the effects on reader risk perceptions of two situations: when agency communication behavior was reported to be responsive to citizens' risk concerns, vs. when the agency was reported to be unresponsive. In the first two experiments, news stories of public meetings filled with distrust and controversy led to ratings indicating greater perceived risk than news stories reporting no distrust or controversy, even though the risk information was held constant. This effect appeared clearly when the differences in meeting tone were extreme and subjects made their ratings from their recall of the stories, but it was much weaker when the differences were moderate and subjects were allowed to go back over the news stories to help separate risk information from conflict information. In the third experiment, news stories about a spill cleanup systematically varied the seriousness of the spill, the amount of technical information provided in the story, and the agency behavior and resulting community outrage. The outrage manipulation significantly affected affective and cognitive components of perceived risk, but not hypothetical behavioral intentions. Seriousness and technical detail had very little effect on perceived risk.     

Risk Perception Among Smokers: A Qualitative Study

The aim of this qualitative study is to identify the dimensions people used to assess the risk of smoking and then compare them with those used by health professionals in primary healthcare. Five discussion groups were conducted. The field work was carried out in Barcelona (Spain), from February 2005 to January 2006. Data were analyzed following a semantic‐thematic categorical content analysis (ACC‐ts). Results showed that people tend to employ stereotypical discourses when evaluating the risk of smoking. Similarly, they reassess the risk in the context of their life experience and incorporate new nuances to the arguments sustaining their behavior. Once this reassessment takes place, the decision to continue smoking emerges, and smokers come up with additional arguments justifying their habit (i.e., age, benefits related to costs). Professionals are aware of this process and its multidimensional nature. Nevertheless, their discourse loses this multidimensional feature when discussing the strategies they use at smoking cessation interventions. This qualitative study increases the understanding of various meanings that people attribute to their life experience. These assumptions may be useful for developing health practices that are closer to people. As a practical utility of these results, it would be interesting to apply a preliminary assessment of the different meanings that people attribute to smoking from their life context in risk communication.     

A Review of Risk Matrices Used in Acute Hospitals in England

In healthcare, patient safety has received substantial attention and, in turn, a number of approaches to managing safety have been adopted from other high‐risk industries. One of these has been risk assessment, predominantly through the use of risk matrices. However, while other industries have criticized the design and use of these risk matrices, the applicability of such criticism has not been investigated formally in healthcare. This study examines risk matrices as used in acute hospitals in England and the guidance provided for their use. It investigates the applicability of criticisms of risk matrices from outside healthcare through a document analysis of the risk assessment policies, procedures, and strategies used in English hospitals. The findings reveal that there is a large variety of risk matrices used, where the design of some might increase the chance of risk misprioritization. Additionally, findings show that hospitals may provide insufficient guidance on how to use risk matrices as well as what to do in response to the existing criticisms of risk matrices. Consequently, this is likely to lead to variation in the quality of risk assessment and in the subsequent deployment of resources to manage the assessed risk. Finally, the article outlines ways in which hospitals could use risk matrices more effectively.     

How Do Individuals Apply Risk Information When Choosing Among Health Care Interventions?

A sample of 3,201 Danes was subjected to personal interviews in which they were asked to state their preferences for risk-reducing health care interventions based on information on absolute risk reduction (ARR) and relative risk reduction (RRR). The aim of the study was to measure the relative weighting of different types of risk information under various circumstances. The effect of presenting questions, and of explicitly formulating RRR, was analyzed. A preference for increases in RRR was demonstrated. There was a stronger inclination to choose the intervention that offered the highest RRR if RRR was explicitly stated. Individuals with more than 10 years of schooling also demonstrated a preference for increased ARR, but only when facing individually framed choices. In a social choice context, preferences for RRR remained intact, but the magnitude of ARR had no impact on choices. Results imply that social framing may induce a propensity to prefer interventions that target high-risk populations. Those respondents who had received < or = 10 years of schooling demonstrated preferences for RRR but not ARR, and no impact of social framing was observed.     

A Methodology for Modeling Regional Terrorism Risk

Over the past decade, terrorism risk has become a prominent consideration in protecting the well‐being of individuals and organizations. More recently, there has been interest in not only quantifying terrorism risk, but also placing it in the context of an all‐hazards environment in which consideration is given to accidents and natural hazards, as well as intentional acts. This article discusses the development of a regional terrorism risk assessment model designed for this purpose. The approach taken is to model terrorism risk as a dependent variable, expressed in expected annual monetary terms, as a function of attributes of population concentration and critical infrastructure. This allows for an assessment of regional terrorism risk in and of itself, as well as in relation to man‐made accident and natural hazard risks, so that mitigation resources can be allocated in an effective manner. The adopted methodology incorporates elements of two terrorism risk modeling approaches (event‐based models and risk indicators), producing results that can be utilized at various jurisdictional levels. The validity, strengths, and limitations of the model are discussed in the context of a case study application within the United States.