Stochastic Judgments in the AHP: The Measurement of Rank Reversal Probabilities

This paper presents a methodology for analyzing Analytic Hierarchy Process (AHP) rankings if the pairwise preference judgments are uncertain (stochastic). If the relative preference statements are represented by judgment intervals, rather than single values, then the rankings resulting from a traditional (deterministic) AHP analysis based on single judgment values may be reversed, and therefore incorrect. In the presence of stochastic judgments, the traditional AHP rankings may be stable or unstable, depending on the nature of the uncertainty. We develop multivariate statistical techniques to obtain both point estimates and confidence intervals of the rank reversal probabilities, and show how simulation experiments can be used as an effective and accurate tool for analyzing the stability of the preference rankings under uncertainty. If the rank reversal probability is low, then the rankings are stable and the decision maker can be confident that the AHP ranking is correct. However, if the likelihood of rank reversal is high, then the decision maker should interpret the AHP rankings cautiously, as there is a subtantial probability that these rankings are incorrect. High rank reversal probabilities indicate a need for exploring alternative problem formulations and methods of analysis. The information about the extent to which the ranking of the alternatives is sensitive to the stochastic nature of the pairwise judgments should be valuable information into the decision-making process, much like variability and confidence intervals are crucial tools for statistical inference. We provide simulation experiments and numerical examples to evaluate our method. Our analysis of rank reversal due to stochastic judgments is not related to previous research on rank reversal that focuses on mathematical properties inherent to the AHP methodology, for instance, the occurrence of rank reversal if a new alternative is added or an existing one is deleted.     

A Deliberative Method for Ranking Risks (II): Evaluation of Validity and Agreement among Risk Managers

A deliberative method for ranking risks was evaluated in a study involving 218 risk managers. Both holistic and multiattribute procedures were used to assess individual and group rankings of health and safety risks facing students at a fictitious middle school. Consistency between the rankings that emerged from these two procedures was reasonably high for individuals and for groups, suggesting that these procedures capture an underlying construct of riskiness. Participants reported high levels of satisfaction with their groups' decision-making processes and the resulting rankings, and these reports were corroborated by regression analyses. Risk rankings were similar across individuals and groups, even though individuals and groups did not always agree on the relative importance of risk attributes. Lower consistency between the risk rankings from the holistic and multiattribute procedures and lower agreement among individuals and groups regarding these rankings were observed for a set of high-variance risks. Nonetheless, the generally high levels of consistency, satisfaction, and agreement suggest that this deliberative method is capable of producing risk rankings that can serve as informative inputs to public risk-management decision making.     

Hybrid Processing of Stochastic and Subjective Uncertainty Data

Uncertainty analyses typically recognize separate stochastic and subjective sources of uncertainty, but do not systematically combine the two, although a large amount of data used in analyses is partly stochastic and partly subjective. We have developed methodology for mathematically combining stochastic and subjective sources of data uncertainty, based on new "hybrid number" approaches. The methodology can be utilized in conjunction with various traditional techniques, such as PRA (probabilistic risk assessment) and risk analysis decision support. Hybrid numbers have been previously examined as a potential method to represent combinations of stochastic and subjective information, but mathematical processing has been impeded by the requirements inherent in the structure of the numbers, e.g., there was no known way to multiply hybrids. In this paper, we will demonstrate methods for calculating with hybrid numbers that avoid the difficulties. By formulating a hybrid number as a probability distribution that is only fuzzily known, or alternatively as a random distribution of fuzzy numbers, methods are demonstrated for the full suite of arithmetic operations, permitting complex mathematical calculations. It will be shown how information about relative subjectivity (the ratio of subjective to stochastic knowledge about a particular datum) can be incorporated. Techniques are also developed for conveying uncertainty information visually, so that the stochastic and subjective components of the uncertainty, as well as the ratio of knowledge about the two, are readily apparent. The techniques demonstrated have the capability to process uncertainty information for independent, uncorrelated data, and for some types of dependent and correlated data. Example applications are suggested, illustrative problems are shown, and graphical results are given.     

Managing Demand Risk in Tactical Supply Chain Planning for a Global Consumer Electronics Company

We consider the problem of managing demand risk in tactical supply chain planning for a particular global consumer electronics company. The company follows a deterministic replenishment‐and‐planning process despite considerable demand uncertainty. As a possible way to formally address uncertainty, we provide two risk measures, “demand‐at‐risk” (DaR) and “inventory‐at‐risk” (IaR) and two linear programming models to help manage demand uncertainty. The first model is deterministic and can be used to allocate the replenishment schedule from the plants among the customers as per the existing process. The other model is stochastic and can be used to determine the “ideal” replenishment request from the plants under demand uncertainty. The gap between the output of the two models as regards requested replenishment and the values of the risk measures can be used by the company to reallocate capacity among different products and to thus manage demand/inventory risk.     

Further Notes on Public Response to Uncertainty in Risks and Science

Given the prevalence of uncertainty and variability in estimates of environmental health risks, it is important to know how citizens interpret information representing uncertainty in risk estimates. Ranges of risk estimates from a hypothetical industry source elicited divergent evaluations of risk assessors' honesty and competence among New Jersey residents within one mile of one or more factories. A plurality saw ranges of risk estimates as both honest and competent, but with most judging such ranges as deficient on one or both dimensions. They wanted definitive conclusions about safety, tended to believe the high end of the range was more likely to be an accurate estimate of the risk, and believed that institutions only discuss risks when they are "high." Acknowledgment of scientific, as opposed to self-interested, reasons for uncertainty and disputes among experts was low. Attitude toward local industry seemed associated with, if not a cause of, attitudes about ranges of risk estimates. These reactions by industry neighbors appear to replicate the findings of Johnson and Slovic (1995, 1998), despite the hypothetical producer of risk estimates being industry instead of government. Respondents were older and less educated on average than were the earlier samples, but more diverse. Regression analyses suggested attitude toward industry was a major factor in these reactions, although other explanations (e.g., level of scientific understanding independent of general education) were not tested in this study.     

Are Some Risk Comparisons More Effective Under Conflict?: A Replication and Extension of Roth et al.

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Only one study, published by Roth et al. in this journal in 1990, has tested the 1988 predictions by Covello et al. as to the public's relative preferences for 14 kinds of risk comparisons as they might be used by a factory manager to explain risks of his ethylene oxide plant. That study found no correlations between the Covello predictions and seven different measures of "acceptability" of Covello's examples of each type of comparison. However, two critics of the Roth study, as well as its own authors, suggested that a scenario involving local risks, a conflict-ridden situation, and a plant manager unknown to the townspeople might better evoke Covello-like preferences than the distant, calm, friends-involving scenario used by Roth. The research reported here replicated the Roth study using the same scenario, risk comparison examples, and evaluation measures, and added a second scenario intended to replicate the conditions suggested by critics. Over 200 New Jersey residents answered the study questionnaire. The replication scenario reproduced Roth's results, and the conflict scenario also evoked no rankings correlated with Covello's predictions. Furthermore, neither agreement nor disagreement with five statements representing "conflict"--respondents' reports that the industrial-plant scenario made them angry, they lived near industry, they were concerned about industrial risks, people in their home town were angry about industrial pollution, and they worried "frequently" about long-term effects of pollution--correlated with Covello's predictions. Over half of all ratings ascribed to the comparisons in aggregate were positive, and most detailed comments offered by respondents also were positive, despite many criticisms and suggestions for their improvement. The wide variability in individuals' rankings also undermines the notion of any single ranking of preferred comparisons. These findings have implications for use of risk comparisons, but also reveal the inaccuracy of the field's assumptions about public reaction to industrial risk information, including risk comparison.     

风险条件下研发项目改进DEA选择模型

研发项目是企业提高竞争力的关键,研发项目选择关系到企业的战略和未来发展趋势。研发项目在生命周期内面临技术风险、突发风险和市场风险,本文考虑到研发项目的风险特征,结合企业战略,构建了基于效率排序的研发项目选择模型。首先,本文对研发项目的有效性和相对于其它备选项目的相对优势进行分析;然后针对数据包络分析模型排序的不足,结合平衡计分卡理念,规范研发项目投入的比例,引入实物期权理论,改进数据包络分析模型,实现了对研发项目的效率排序;最后通过实际算例对模型进行验证、横向对比和敏感性分析,结果表明：相对于标准数据包络分析模型,本文提出的模型具有更高的效率区分度,并且能够敏感地捕捉到研发项目投入和产出的变化对效率影响的演变过程。     

Augmenting the Deliberative Method for Ranking Risks

The Department of Homeland Security (DHS) characterized and prioritized the physical cross‐border threats and hazards to the nation stemming from terrorism, market‐driven illicit flows of people and goods (illegal immigration, narcotics, funds, counterfeits, and weaponry), and other nonmarket concerns (movement of diseases, pests, and invasive species). These threats and hazards pose a wide diversity of consequences with very different combinations of magnitudes and likelihoods, making it very challenging to prioritize them. This article presents the approach that was used at DHS to arrive at a consensus regarding the threats and hazards that stand out from the rest based on the overall risk they pose. Due to time constraints for the decision analysis, it was not feasible to apply multiattribute methodologies like multiattribute utility theory or the analytic hierarchy process. Using a holistic approach was considered, such as the deliberative method for ranking risks first published in this journal. However, an ordinal ranking alone does not indicate relative or absolute magnitude differences among the risks. Therefore, the use of the deliberative method for ranking risks is not sufficient for deciding whether there is a material difference between the top‐ranked and bottom‐ranked risks, let alone deciding what the stand‐out risks are. To address this limitation of ordinal rankings, the deliberative method for ranking risks was augmented by adding an additional step to transform the ordinal ranking into a ratio scale ranking. This additional step enabled the selection of stand‐out risks to help prioritize further analysis.     

不确定环境下研发投资决策的期权博弈模型

假定产出价格(随机需求)服从带跳的几何布朗运动来模拟研发项目中突发事件和市场的不确定性特点,拓展了用几何布朗运动模拟市场不确定性的双寡头期权博弈模型,同时也是在带跳的几何布朗运动的实物期权方法中融入了竞争策略互动的影响.敏感性分析结果表明随着这两类不确定性的增大,参与双方进入门槛值都变大.突发事件带来的不确定提高会使得参与双方的期权价值降低,但是市场的不确定性变大对于追随者来说等待是有价值的,对领先者的期权价值的影响却是不定的.     

Categorizing Risks for Risk Ranking

Any practical process of risk ranking must group hazards into a manageable number of categories. Defining such categories requires value choices that can have important implications for the rankings that result. Most risk-management organizations will find it useful to begin defining categories in terms of environmental loadings or initiating events. However, the resulting categories typically need to be modified in light of other considerations. Risk-ranking projects can benefit from considering several alternative categorization strategies and drawing upon elements of each in developing their final categorization of risks. In principle, conducting multiple ranking exercises by using different categorizations could be interesting and useful. In practice, agencies are unlikely to have either the resources or patience to do this, but other groups in society might. Done well, such additional independent rankings could add valuable inputs to democratic risk-management decision making.     

On the Need for Restricting the Probabilistic Analysis in Risk Assessments to Variability

It is common perspective in risk analysis that there are two kinds of uncertainties: i) variability as resulting from heterogeneity and stochasticity (aleatory uncertainty) and ii) partial ignorance or epistemic uncertainties resulting from systematic measurement error and lack of knowledge. Probability theory is recognized as the proper tool for treating the aleatory uncertainties, but there are different views on what is the best approach for describing partial ignorance and epistemic uncertainties. Subjective probabilities are often used for representing this type of ignorance and uncertainties, but several alternative approaches have been suggested, including interval analysis, probability bound analysis, and bounds based on evidence theory. It is argued that probability theory generates too precise results when the background knowledge of the probabilities is poor. In this article, we look more closely into this issue. We argue that this critique of probability theory is based on a conception of risk assessment being a tool to objectively report on the true risk and variabilities. If risk assessment is seen instead as a method for describing the analysts’ (and possibly other stakeholders’) uncertainties about unknown quantities, the alternative approaches (such as the interval analysis) often fail in providing the necessary decision support.     

基于两阶段资源分配协调机制的分布式多项目随机调度

企业趋向于多项目共享全局资源的分布式协同管理。但在多项目实际执行时，全局资源可用量往往由于外部环境的动态变化而存在不确定性，活动中断、资源浪费等现象频发，项目管理变得愈加复杂。本文将不确定的全局资源可用量建模为随机变量，设计两阶段资源分配协调机制，在预分配阶段，考虑项目允许的最大活动中断次数约束，建立各项目调度的马尔可夫动态决策过程模型；预分配结束后，基于活动重要度依次对剩余全局资源进行协调再分配，以提高资源利用率并减少平均项目延期。设计基于全局资源协调分配的Rollout近似动态规划算法进行求解。开展问题库算例实验研究与案例分析，验证协调机制与求解算法的性能；同时，探讨并分析不确定参数对目标结果的影响。     

Fishing in a Polluted Estuary: Fishing Behavior, Fish Consumption, and Potential Risk

People make subjective judgments about hazards relying on what they know and feel. These risk perceptions may be based on accurate or inaccurate information and are often optimistically biased. The existence of uncertainties in the evaluation of many environmental hazards effects how risks are perceived. This paper examines fish consumption and risk perception of urban fishermen in the New York/New Jersey estuary, in areas where there were consumption advisories. We interviewed 318 fishermen and crabbers in the Arthur Kill, Raritan Bay, and New Jersey shore. Fish were eaten an average of at least four times per month in all regions, but fishermen in the Arthur Kill fished most frequently, averaging over eight times per month. Although 60% of fishermen and crabbers in the Arthur Kill reported hearing warnings about consuming fish caught in these waters, 70% of fishermen and 76% of crabbers said they ate their catch. Significantly fewer fishermen in the Bay and Shore regions had heard warnings (28% and 30%, respectively), and more reported consuming their catch (88% and 82%, respectively). In all regions, most people thought that the fish were safe to eat, many believing they were "fresher" than store bought fish. Thus, most people ignored the consumption advisories in effect for these waters. Some of these people are consuming high quantities of fish and crabs, and thus are exposed to potentially deleterious levels of contaminants. In general, people failed to consider the possibility of chronic effects and did not perceive that this enjoyable, familiar pastime could be hazardous. Further, fishermen generally had great confidence in their own knowledge, which proved to be inaccurate in many cases, and often expressed distrust in the information source (government). Clearly, simply issuing consumption advisories is insufficient to promote risk-reducing behavior.     

Importance of Uncertainty and Variability to Predicted Risks from Trophic Transfer of PCBs in Dredged Sediments

Biomagnification of organochlorine and other persistent organic contaminants by higher trophic level organisms represents one of the most significant sources of uncertainty and variability in evaluating potential risks associated with disposal of dredged materials. While it is important to distinguish between population variability (e.g., true population heterogeneity in fish weight, and lipid content) and uncertainty (e.g., measurement error), they can be operationally difficult to define separately in probabilistic estimates of human health and ecological risk. We propose a disaggregation of uncertain and variable parameters based on: (1) availability of supporting data; (2) the specific management and regulatory context (in this case, of the U.S. Army Corps of Engineers/U.S. Environmental Protection Agency tiered approach to dredged material management); and (3) professional judgment and experience in conducting probabilistic risk assessments. We describe and quantitatively evaluate several sources of uncertainty and variability in estimating risk to human health from trophic transfer of polychlorinated biphenyls (PCBs) using a case study of sediments obtained from the New York-New Jersey Harbor and being evaluated for disposal at an open water off-shore disposal site within the northeast region. The estimates of PCB concentrations in fish and dietary doses of PCBs to humans ingesting fish are expressed as distributions of values, of which the arithmetic mean or mode represents a particular fractile. The distribution of risk values is obtained using a food chain biomagnification model developed by Gobas by specifying distributions for input parameters disaggregated to represent either uncertainty or variability. Only those sources of uncertainty that could be quantified were included in the analysis. Results for several different two-dimensional Latin Hypercube analyses are provided to evaluate the influence of the uncertain versus variable disaggregation of model parameters. The analysis suggests that variability in human exposure parameters is greater than the uncertainty bounds on any particular fractile, given the described assumptions.     

Probabilistic dominance criteria for comparing uncertain alternatives: A tutorial

This paper is a tutorial which demonstrates the current state-of-the-art methods for incorporating risk into project selection decision making. The projects under consideration might be R&D, IT, or other capital expenditure programs. We will show six decision making methods: 1. mean-variance (MV), 2. mean-semivariance, 3. mean-critical probability, 4. stochastic dominance, 5. almost stochastic dominance (ASD), and 6. mean-Gini. We will also describe the assumptions about the risk attitudes of the decision maker which are associated with each of the techniques. While all these methods have been previously applied elsewhere, this is the first paper which shows all of their applications in the project selection context, together with their interrelationships, strengths and weaknesses. We have applied all six techniques to the same group of five hypothetical projects and evaluated the resulting nondominated sets. Among the methods reviewed here, stochastic dominance is recommended because it requires the least restrictive assumptions. ASD and mean-Gini are recommended when stochastic dominance is not practical or when it does not yield definitive choices. MV, mean-semivariance, and mean-critical probability are shown to be flawed.     

Risk Comparisons, Conflict, and Risk Acceptability Claims

Despite many claims for and against the use of risk comparisons in risk communication, few empirical studies have explored their effect. Even fewer have examined the public's relative preferences among different kinds of risk comparisons. Two studies, published in this journal in 1990 and 2003, used seven measures of "acceptability" to examine public reaction to 14 examples of risk comparisons, as used by a hypothetical factory manager to explain risks of his ethylene oxide plant. This study examined the effect on preferences of scenarios involving low or high conflict between the factory manager and residents of the hypothetical town (as had the 2003 study), and inclusion of a claim that the comparison demonstrated the risks' acceptability. It also tested the Finucane et al. (2000) affect hypothesis that information emphasizing low risks-as in these risk comparisons-would raise benefits estimates without changing risk estimates. Using similar but revised scenarios, risk comparison examples (10 instead of 14), and evaluation measures, an opportunity sample of 303 New Jersey residents rated the comparisons, and the risks and benefits of the factory. On average, all comparisons received positive ratings on all evaluation measures in all conditions. Direct and indirect measures showed that the conflict manipulation worked; overall, No-Conflict and Conflict scenarios evoked scores that were not significantly different. The attachment to each risk comparison of a risk acceptability claim ("So our factory's risks should be acceptable to you.") did not worsen ratings relative to conditions lacking this claim. Readers who did or did not see this claim were equally likely to infer an attempt to persuade them to accept the risk from the comparison. As in the 2003 article, there was great individual variability in inferred rankings of the risk comparisons. However, exposure to the risk comparisons did not reduce risk estimates significantly (while raising benefit estimates), and Conflict-Claim respondents found the risk of the hypothetical factory less acceptable than No-Conflict respondents. Results suggest that neither risk comparisons nor risk acceptability claims are automatically anathema to audiences, but they may have tiny or unintended effects on audience judgments about risky situations.     

Farmers’ Risk‐Based Decision Making Under Pervasive Uncertainty: Cognitive Thresholds and Hazy Hedging

Researchers in judgment and decision making have long debunked the idea that we are economically rational optimizers. However, problematic assumptions of rationality remain common in studies of agricultural economics and climate change adaptation, especially those that involve quantitative models. Recent movement toward more complex agent‐based modeling provides an opportunity to reconsider the empirical basis for farmer decision making. Here, we reconceptualize farmer decision making from the ground up, using an in situ mental models approach to analyze weather and climate risk management. We assess how large‐scale commercial grain farmers in South Africa (n = 90) coordinate decisions about weather, climate variability, and climate change with those around other environmental, agronomic, economic, political, and personal risks that they manage every day. Contrary to common simplifying assumptions, we show that these farmers tend to satisfice rather than optimize as they face intractable and multifaceted uncertainty; they make imperfect use of limited information; they are differently averse to different risks; they make decisions on multiple time horizons; they are cautious in responding to changing conditions; and their diverse risk perceptions contribute to important differences in individual behaviors. We find that they use two important nonoptimizing strategies, which we call cognitive thresholds and hazy hedging, to make practical decisions under pervasive uncertainty. These strategies, evident in farmers' simultaneous use of conservation agriculture and livestock to manage weather risks, are the messy in situ performance of naturalistic decision‐making techniques. These results may inform continued research on such behavioral tendencies in narrower lab‐ and modeling‐based studies.     

Government Risk Management Priorities: A Comparison of the Preferences of Asian Indian Americans and Other Americans

A survey was conducted of approximately 200 Asian Indian Americans and 200 other residents of New Jersey in order to understand the risk management priorities that they want government to have. We found that Asian Indian Americans, especially younger women, focused on personal/family risks, such as alcohol and drug abuse, sexual abuse, and domestic violence. The New Jersey comparison group, in contrast, placed war/terrorism and loss of health care services and insurance at the top of their priorities for government. These results suggest stressful acculturation-related issues within the Asian Indian community. Both populations want more risk management from government than they believe government is currently providing. Respondents who wanted more from government tended to dread the risk, be fearful of the consequences, trust government, and have a feeling of personal efficacy. Within the Asian Indian American sample, wide variations were observed by language spoken at home and religious affiliation. Notably, Muslims and Hindi language speakers tended not to trust government and hence wanted less government involvement. This study supports our call for studies of recent migrant populations and Johnson's for testing ethnic identity and acculturation as factors in risk judgments.