A Comparative Analysis of PRA and Intelligent Adversary Methods for Counterterrorism Risk Management

In counterterrorism risk management decisions, the analyst can choose to represent terrorist decisions as defender uncertainties or as attacker decisions. We perform a comparative analysis of probabilistic risk analysis (PRA) methods including event trees, influence diagrams, Bayesian networks, decision trees, game theory, and combined methods on the same illustrative examples (container screening for radiological materials) to get insights into the significant differences in assumptions and results. A key tenent of PRA and decision analysis is the use of subjective probability to assess the likelihood of possible outcomes. For each technique, we compare the assumptions, probability assessment requirements, risk levels, and potential insights for risk managers. We find that assessing the distribution of potential attacker decisions is a complex judgment task, particularly considering the adaptation of the attacker to defender decisions. Intelligent adversary risk analysis and adversarial risk analysis are extensions of decision analysis and sequential game theory that help to decompose such judgments. These techniques explicitly show the adaptation of the attacker and the resulting shift in risk based on defender decisions.     

A CASE IN ON-LINE DECISION ANALYSIS FOR PRODUCT PLANNING1

This report presents an application of quantitative decision-analytic techniques to consider an opportunity for capital investment in a new product. Decision analysis is a well-established technique for evaluating major decisions in which substantial resources and time are available for the analysis and the problem is definitely formulated at the outset. This case study, however, illustrates how decision-analytic techniques can be used on-line to improve decision making in a situation in which time and resources are limited, and the manager's perception of the problem changes in response to feedback from the analysis. The basic decision problem was whether or not the AIL division of Cutler-Hammer, Incorporated should purchase a six-month option on a flight-safety system patent. The president of AIL had only a few weeks in which to make a decision. The analytic approach focused on a probabilistic discounted earnings model. This model was refined over a two-week period through a sequence of iterations which incorporated the results of both direct and indirect assessments of probabilities and values. The analysis affected the company's decision process by providing a vehicle for structuring the ongoing communication between AIL's president and his advisors, and by focusing their attention on the most important issues. In the end, the analysis facilitated a unanimous decision from the decision-making group not to purchase the option, but to consider participating in the license as a subcontractor at a later date. This strategy had not been considered at the outset.     

Farmers’ Risk‐Based Decision Making Under Pervasive Uncertainty: Cognitive Thresholds and Hazy Hedging

Researchers in judgment and decision making have long debunked the idea that we are economically rational optimizers. However, problematic assumptions of rationality remain common in studies of agricultural economics and climate change adaptation, especially those that involve quantitative models. Recent movement toward more complex agent‐based modeling provides an opportunity to reconsider the empirical basis for farmer decision making. Here, we reconceptualize farmer decision making from the ground up, using an in situ mental models approach to analyze weather and climate risk management. We assess how large‐scale commercial grain farmers in South Africa (n = 90) coordinate decisions about weather, climate variability, and climate change with those around other environmental, agronomic, economic, political, and personal risks that they manage every day. Contrary to common simplifying assumptions, we show that these farmers tend to satisfice rather than optimize as they face intractable and multifaceted uncertainty; they make imperfect use of limited information; they are differently averse to different risks; they make decisions on multiple time horizons; they are cautious in responding to changing conditions; and their diverse risk perceptions contribute to important differences in individual behaviors. We find that they use two important nonoptimizing strategies, which we call cognitive thresholds and hazy hedging, to make practical decisions under pervasive uncertainty. These strategies, evident in farmers' simultaneous use of conservation agriculture and livestock to manage weather risks, are the messy in situ performance of naturalistic decision‐making techniques. These results may inform continued research on such behavioral tendencies in narrower lab‐ and modeling‐based studies.     

Structuring Conflict in Individual,Face-to-Face,and Computer-Mediated Group Decision Making: Carping Versus Objective Devil's Advocacy*

Researchers and practitioners have long been interested in the effects of cognitive conflict techniques on individual and group decision making. One widely used and studied technique, devil's advocacy (DA), has been found to enhance decision-making performance for both individuals and groups. Devil's advocacy begins with a recommended decision, followed by a critique of the decision that questions its assumptions. Researchers have not yet examined the effects of the objectivity of the devil's advocacy comments in computer-mediated environments. This paper reports the results of a laboratory experiment that focused on this question by comparing the effects of an objective, nonemotional DA to an emotional, “carping” DA within individuals and groups using either computer-mediated or face-to-face communication. In a manner consistent with prior research, both DA treatments were operationalized through the use of paper-based consulting reports. The results suggest that individuals and computer-mediated groups develop and consider more solution alternatives than face-to-face groups, and that subjects given the objective DA treatment produce higher quality decisions than those given the carping DA treatment. Face-to-face groups in the carping DA treatment considered the fewest alternative solutions in their decision-making process, reached the lowest solution quality, yet reached decision consensus in the fewest voting rounds. The practical implications of the results suggest that questioning statements made by a devil's advocate should be objective, regardless of group communication condition. Carping devil's advocacy appears to stifle group decision outcomes when groups are using face-to-face communication.     

The Microeconomics of Efficient Group Behavior: Identification1

Consider a group consisting of S members facing a common budget constraint p'ξ=1: any demand vector belonging to the budget set can be (privately or publicly) consumed by the members. Although the intragroup decision process is not known, it is assumed to generate Pareto‐efficient outcomes; neither individual consumptions nor intragroup transfers are observable. The paper analyzes when, to what extent, and under which conditions it is possible to recover the underlying structure—individual preferences and the decision process—from the group's aggregate behavior. We show that the general version of the model is not identified. However, a simple exclusion assumption (whereby each member does not consume at least one good) is sufficient to guarantee generic identifiability of the welfare‐relevant structural concepts.     

Probabilistic dominance criteria for comparing uncertain alternatives: A tutorial

This paper is a tutorial which demonstrates the current state-of-the-art methods for incorporating risk into project selection decision making. The projects under consideration might be R&D, IT, or other capital expenditure programs. We will show six decision making methods: 1. mean-variance (MV), 2. mean-semivariance, 3. mean-critical probability, 4. stochastic dominance, 5. almost stochastic dominance (ASD), and 6. mean-Gini. We will also describe the assumptions about the risk attitudes of the decision maker which are associated with each of the techniques. While all these methods have been previously applied elsewhere, this is the first paper which shows all of their applications in the project selection context, together with their interrelationships, strengths and weaknesses. We have applied all six techniques to the same group of five hypothetical projects and evaluated the resulting nondominated sets. Among the methods reviewed here, stochastic dominance is recommended because it requires the least restrictive assumptions. ASD and mean-Gini are recommended when stochastic dominance is not practical or when it does not yield definitive choices. MV, mean-semivariance, and mean-critical probability are shown to be flawed.     

AGGREGATING EXPERT OPINION IN DECISION-MAKING

In using nominal groups for decision making, it is necessary to use some mechanical procedure for combining the evaluations. A simulation model is used to compare procedures for the case where a nominal group of m evaluators must select the best of n alternatives and where the evaluations are subject to random errors. Criteria are the probability of making a correct selection and the relative quality of the choice.     

PREFERENCE MAPPING OF PERCEIVED INFORMATION STRUCTURE: IMPLICATIONS FOR DECISION SUPPORT SYSTEMS DESIGN

In a recent issue of this journal, Watkins [13] presented an approach for discovery of decision-maker perceptions of the complexity (dimensionality) of information items that might be supplied by a decision support system. Through use of multidimensional scaling and cluster analysis, relatively homogeneous groups of decision makers, sharing common perceptions of various information items, were formed. This prior research was referred to as a first step in suggesting that information reports could be tailored to groups of decision makers classified on the basis of common perceptions of information. The current research extends the prior study by evaluating decision maker preferences for information in a variety of decision-making scenarios in relation to the previously identified perceptions of the information. Based on the results of the study, conclusions are made which suggest that the tailoring of information to groups of decision makers should be based on both perceptions and preferences for information. Even so, it is demonstrated that the decision tasks have an impact on the preferences for information which may affect the attempt to tailor information to groups of decision makers.     

Evidence‐based Management in Practice: Opening up the Decision Process,Decision‐maker and Context

Evidence‐based management (EBM) has been subject to a number of persuasive critiques in recent years. Concerns have been raised that: EBM over‐privileges rationality as a basis for decision‐making; ‘scientific’ evidence is insufficient and incomplete as a basis for management practice; understanding of how EBM actually plays out in practice is limited; and, although ideas were originally taken from evidence‐based medicine, individual‐situated expertise has been forgotten in the transfer. To address these concerns, the authors adopted an approach of ‘opening up’ the decision process, the decision‐maker and the context (Langley et al. ( 1995 ). ‘Opening up decision making: the view from the black stool’, Organization Science, 6, pp. 260–279). The empirical investigation focuses on an EBM decision process involving an operations management problem in a hospital emergency department in Australia. Based on interview and archival research, it describes how an EBM decision process was enacted by a physician manager. It identifies the role of ‘fit’ between the decision‐maker and the organizational context in enabling an evidence‐based process and develops insights for EBM theory and practice.     

The Emergence of Artificial Intelligence as a Reference Discipline for Decision Support Systems Research

It is timely and appropriate to examine both philosophical and pragmatical issues associated with formalizing the adoption of artificial intelligence as a reference discipline for decision support systems research. This paper reflects on where we were when the first special issue of Decision Sciences on expert systems and decision support systems was published, addresses the dynamics of what has taken place subsequent to the publication of that first special issue, sets forth a proposition to stimulate ongoing dialog with respect to synergies between the decision support system research agenda and the research agenda of the artificial intelligence discipline, and demonstrates how the papers appearing in this follow-up special issue of Decision Sciences are representative of an emerging, challenging, and exciting new decision support systems era.     

AN EMPIRICAL EXAMINATION OF SOCIAL POWER IN TERMS OF DECISION THEORY1

The paper presents an extension of decision theory to the analysis of social power. The power of a person, A, over another person, B, is viewed in terms of the effect A has on B's decision. The analysis is based on the idea that B's decision regarding the performance of alternative behaviors is a function of 1) B's utility for the consequences of the behaviors and 2) B's subjective probabilities that the behaviors will lead to these consequences. In these terms, A's power over B lies in A's ability to mediate various consequences for B, contingent upon B's compliance or noncompliance. Subjects were asked to consider eight situations in which hypothetical individuals had to make a choice between two courses of action. In each situation another person (A) was attempting to induce the hypothetical individual (B) to choose one of the alternatives, while various situational factors were influencing B to choose the other alternative. The subjects were asked to consider B's utilities and subjective probabilities in each situation and to indicate whether or not B should comply with A and to make ratings of A's power. The decision theory analysis did well in predicting whether or not subjects would indicate that B should comply with A. Also, subjects generally were able to correctly specify whether A or the situational factors had more influence over B's decision. Finally, the subjects' ratings of A's power in the eight situations were highly related to the decision theoretic measure of power.     

Tailoring value elicitation to decision makers' numeracy and fluency: Expressing value judgments in numbers or words

In organizational settings, options evaluation requires managers to express value judgments on multiple criteria. This research investigates the influence of decision makers' numeracy (ability to use appropriate numerical principles) and fluency (ability to express oneself in words) on their subjective experience of value elicitation as supported by two different techniques: direct rating and MACBETH. The former asks for value judgments to be expressed numerically, the latter non-numerically. The results of our experiment indicate that the two techniques are not psychologically equivalent: decision makers with higher numeracy express values more easily when assisted by the numerical technique whereas decision makers with higher fluency find value elicitation easier with the non-numerical technique. These findings highlight the importance of tailoring value elicitation to decision makers' numeracy and fluency. Implications for decision scientists and analysts are discussed.     

What do we loose when machines take the decisions?

This paper concerns the technical issues raised when humans are replaced by artificial intelligence (AI) in organisational decision making, or decision making in general. Such automation of human tasks and decision making can of course be beneficial through saving human resources, and through (ideally) leading to better solutions and decisions. However, to guarantee better decisions, the current AI techniques still have some way to go in most areas, and many of the techniques also suffer from weaknesses such as lack of transparency and explainability. The goal of the paper is not to argue against using any kind of AI in organisational decision making. AI techniques have a lot to offer, and can for instance assess a lot more possible decisions—and much faster—than any human can. The purpose is just to point to the weaknesses that AI techniques still have, and that one should be aware of when considering to implement AI to automate human decisions. Significant current AI research goes into reducing its limitations and weaknesses, but this is likely to become a fairly long-term effort. People and organisations might be tempted to fully automate certain crucial aspects of decision making without waiting for these limitations and weaknesses to be reduced—or, even worse, not even being aware of those weaknesses and what is lost in the automatisation process.

     

Random Expected Utility

We develop and analyze a model of random choice and random expected utility. A decision problem is a finite set of lotteries that describe the feasible choices. A random choice rule associates with each decision problem a probability measure over choices. A random utility function is a probability measure over von Neumann–Morgenstern utility functions. We show that a random choice rule maximizes some random utility function if and only if it is mixture continuous, monotone (the probability that a lottery is chosen does not increase when other lotteries are added to the decision problem), extreme (lotteries that are not extreme points of the decision problem are chosen with probability 0), and linear (satisfies the independence axiom).     

Chainwise Paired Comparisons

The pairwise comparison technique is a building block of the Analytic Hierarchy Process (AHP), which has been popularly used for multicriteria decision analysis. This paper develops a shortcut technique in which only n paired comparisons forming a closed chain are needed for n decision elements. Together with the development of a simple and intuitive measure of (inconsistency, this technique derives the relative weights of decision elements via easy step-by-step calculations on a spreadsheet format. Its performance has been tested on Saaty's wealth of nations example. It is important to notice that ranking and weights yielded from this alternative technique are identical to Harker's incomplete pairwise comparison solution for the same chain orientation for the example tested.     

Multi-attribute utility methods in group decision making: Past applications and potential for inclusion in GDSS

This paper reviews a variety of studies in which multi-attribute utility theory (MAU) has been used successfully to aid group decision processes and discusses the incorporation of MAU in Group Decision Support Systems (GDSS). GDSS of various levels of automation have been proposed to aid group decision making. A number of commercial GDSSs are available which are capable of aiding generation of alternatives, their evaluation, and selection. Many authors have promoted the incorporation of quantitative decision making models in GDSS. The most appropriate quantitative tool for GDSS is MAU, but MAU has rarely been utilized. The cases reviewed illustrate that the use of MAU techniques is not prohibitively difficult or complex.     

Concepts,Theory, and Techniques PRINCIPLES OF MULTIOBJECTIVE OPTIMIZATION*

This paper attempts to isolate and analyze the principal ideas of multiobjective optimization. We do this without casting aspersions on single-objective optimization or championing any one multiobjective technique. We examine each fundamental idea for strengths and weaknesses and subject two—efficiency and utility—to extended consideration. Some general recommendations are made in light of this analysis. Besides the simple advice to retain single-objective optimization as a possible approach, we suggest that three broad classes of multiobjective techniques are very promising in terms of reliably, and believably, achieving a most preferred solution. These are: (1) partial generation of the efficient set, a rubric we use to unify a wide spectrum of both interactive and analytic methods; (2) explicit utility maximization, a much-overlooked approach combining multiattribute decision theory and mathematical programming; and (3) interactive implicit utility maximization, the popular class of methods introduced by Geoffrion, Dyer, and Feinberg [24] and extended significantly by others.     

Risk Assessment and Alternatives Assessment: Comparing Two Methodologies

The selection and use of chemicals and materials with less hazardous profiles reflects a paradigm shift from reliance on risk minimization through exposure controls to hazard avoidance. This article introduces risk assessment and alternatives assessment frameworks in order to clarify a misconception that alternatives assessment is a less effective tool to guide decision making, discusses factors promoting the use of each framework, and also identifies how and when application of each framework is most effective. As part of an assessor's decision process to select one framework over the other, it is critical to recognize that each framework is intended to perform different functions. Although the two frameworks share a number of similarities (such as identifying hazards and assessing exposure), an alternatives assessment provides a more realistic framework with which to select environmentally preferable chemicals because of its primary reliance on assessing hazards and secondary reliance on exposure assessment. Relevant to other life cycle impacts, the hazard of a chemical is inherent, and although it may be possible to minimize exposure (and subsequently reduce risk), it is challenging to assess such exposures through a chemical's life cycle. Through increased use of alternatives assessments at the initial stage of material or product design, there will be less reliance on post facto risk‐based assessment techniques because the potential for harm is significantly reduced, if not avoided, negating the need for assessing risk in the first place.