PERCEIVED INFORMATION STRUCTURE: IMPLICATIONS FOR DECISION SUPPORT SYSTEM DESIGN

Top-level decision making in business organizations is characterized by high degrees of uncertainty, incomplete information, and conflicting objectives. To support top-level decision making effectively, decision support systems (DSSs) have been proposed. Information supplied by a DSS must be selective in that not all possible information sets may be feasibly or economically represented in the data base. This study suggests that discovery of perceptual complexity (dimensionality) of information items, and the subsequent categorization of decision makers having the same perceptions of those information items, is a first step in the ultimate design of an effective DSS. Through the use of multidimensional scaling in a field setting, this study shows the feasibility of creating relatively homogeneous groups of decision makers according to the content and number of dimensions associated with various information items. Further results of the research suggest that information can be tailored to classes of users, which has cost-benefit implications as well as the potential to improve the quality of the resultant decisions.     

AN ADAPTIVE DECISION SUPPORT SYSTEM FOR ACADEMIC RESOURCE PLANNING

Institutions of higher learning are growing increasingly interested in the use of model-based approaches to their resource allocation problems. Recent modeling approaches, however, have failed to consider that resource allocation planning is not a well-structured decision process. Additionally, many decision makers are necessarily involved in the academic planning process and may assume dissimilar perspectives on the importance of achieving different goals and objectives. Furthermore, satisfactory allocation solutions can be expected to vary considerably from decision maker to decision maker as the individual's cognitive processes, perceptions, and evaluations are taken into consideration. This paper describes a decision support system (DSS) approach that attempts to adapt to a variety of academic decision makers with differing planning views in an environment of multiple conflicting objectives. This DSS, which was successfully tested on four academic decision makers in a large midwestern university, shows considerable promise for providing decision support to decision makers with varied problem-solving styles.     

Structuring Conflict in Individual,Face-to-Face,and Computer-Mediated Group Decision Making: Carping Versus Objective Devil's Advocacy*

Researchers and practitioners have long been interested in the effects of cognitive conflict techniques on individual and group decision making. One widely used and studied technique, devil's advocacy (DA), has been found to enhance decision-making performance for both individuals and groups. Devil's advocacy begins with a recommended decision, followed by a critique of the decision that questions its assumptions. Researchers have not yet examined the effects of the objectivity of the devil's advocacy comments in computer-mediated environments. This paper reports the results of a laboratory experiment that focused on this question by comparing the effects of an objective, nonemotional DA to an emotional, “carping” DA within individuals and groups using either computer-mediated or face-to-face communication. In a manner consistent with prior research, both DA treatments were operationalized through the use of paper-based consulting reports. The results suggest that individuals and computer-mediated groups develop and consider more solution alternatives than face-to-face groups, and that subjects given the objective DA treatment produce higher quality decisions than those given the carping DA treatment. Face-to-face groups in the carping DA treatment considered the fewest alternative solutions in their decision-making process, reached the lowest solution quality, yet reached decision consensus in the fewest voting rounds. The practical implications of the results suggest that questioning statements made by a devil's advocate should be objective, regardless of group communication condition. Carping devil's advocacy appears to stifle group decision outcomes when groups are using face-to-face communication.     

Feedback in DSS as a Source of Control: Experiments with the Timing of Feedback

Well-designed feedback can improve decision making but to date there has been no comprehensive study of feedback in decision support systems (DSS) that could guide developers in its design. This work examines the opportunities and means to enhance cognitive control in decision making by providing appropriate feedback with DSS. It concentrates on the timing of feedback, which has been shown to affect the use of feedback and also demonstrates the potential advantages of using information technology over and above manual decision environments. Two experiments test the effects of controlling the timing of cognitive feedback on the user's cognitive control. The first tests the effect of timing one source of cognitive feedback, and the second tests the effect of timing two sources of feedback. The findings suggest that the design of feedback should become a routine part of the development of DSS.     

The Influences of User Characteristics on Performance in a Logistics DSS Application

Decision Support Systems (DSS) are widely used in logistics decision applications, and a large number and variety of systems are commercially available. We investigate the contributions of user characteristics including experiences, data preferences, intuition, and effort to decision performance in a logistics DSS context. The study includes a laboratory experiment in which decision makers with varied experiences used a DSS to make facility network design decisions for problems of varying complexity. Two variants of the DSS are utilized in order to examine the interactions of a DSS decision aid with user characteristics. We find that intuition and effort are associated with decision-making performance. High analytic ability is not related to intuition, however. Education and previous experience are associated with performance. Yet these characteristics are also unrelated to intuition. Decision makers who highly value disaggregated data provided by the DSS tend to perform poorly. Also, the results suggest that the effects of users' experiences and preferences on performance are influenced by an analytical decision aid.     

Data and Model Management in a Generalized MCDM-DSS*

Drawing upon the choice models developed in the multiple criteria decision making (MCDM) area, this paper proposes an architecture for designing an intelligent decision support system (DSS) that is intended to aid in making choices among multiple alternatives along multiple dimensions. It argues that effective support can be provided to the decision maker when the knowledge-based DSS is capable of dynamically selecting choice models appropriate to the domain and context of a particular problem being specified by the decision maker, and of properly applying them to the problem solution. Development of a prototype intended to partially represent application of the architecture is described. The paper concludes with suggestions for research extensions.     

FINDING SYNERGY BETWEEN DECISION SUPPORT SYSTEMS AND EXPERT SYSTEMS RESEARCH

This article examines the potential for synergy between decision support systems (DSS) and expert systems research. Three predominant research traditions in DSS (applications, design, and technology) are examined and used as a basis to identify ways in which expert systems and DSS research interrelate. A range of emerging trends in DSS research that focus on these commonalities is discussed.     

Complexity Factors and Intuition-based Methods for Facility Network Design

Logistics managers frequently utilize decision support systems (DSS) to make facility network design decisions. Many DSS do not provide optimization capabilities, but instead rely on scenario evaluation as a means for developing solutions. We experimentally assessed the performances of decision makers, including experienced managers, who used four variants of a scenario evaluation-based DSS to solve realistically sized network design problems of varying complexities. Complexity factors included DSS attributes, problem size, network types, and demand dispersion patterns. Decision makers' performances were assessed relative to optimal solutions. Overall, the decision makers generated relatively high-quality solutions using the DSS variants. The type of design problem solved did not significantly impact problem-solving performance. However, performance degraded and variability in solution quality escalated as problem size was increased. The availability of incremental solution cost improvement cues in the DSS significantly improved solution quality and reduced performance variability. Iconic graphic enhancements to the DSS did not consistently affect performance. However, significant interactions existed among the effects of DSS graphics capabilities, DSS information cues, and problem attributes.     

RANK GENERATION,PRESERVATION, AND REVERSAL IN THE ANALYTIC HIERARCHY DECISION PROCESS

Decision making has the objective of finding the best alternative or set of alternatives by considering a number of goals, objectives, criteria, competitors, and other important factors. The analytic hierarchy process is a decision aid used to assist a decision maker in sorting out the complexity of a decision problem and making use of his or her judgments. A decision maker must be assured that the arithmetic operations of any such decision process are the right ones—that they surface the correct ranking and values of the alternatives and preserve or alter ranks appropriately when new alternatives are added or deleted. In this paper it will be shown that with absolute measurement, rank always is preserved, with relative measurement, rank changes with nspect to scveral criteria only because of the structural dependence (involving both numbers and measurements) of criteria on alternatives. A discussion of the effect on rank of replicas and near replicas of the alternatives also is given.     

The Effectiveness of Decisional Guidance: An Empirical Evaluation

Decisional guidance is defined as how a decision support system (DSS) influences its users as they structure and execute the decision‐making process. It is assumed that decisional guidance has profound effects on decision making, but these effects are understudied and empirically unproven. This paper describes an empirical, laboratory‐experiment‐based evaluation of the effectiveness of deliberate decisional guidance and its four types. We developed and used a comprehensive model consisting of four evaluation criteria: decision quality, user satisfaction, user learning, and decision‐making efficiency. On these criteria, we compared decisional guidance versus no guidance, informative versus suggestive decisional guidance, and predefined versus dynamic decisional guidance. We found that deliberate decisional guidance was more effective on all four criteria; suggestive guidance was more effective in improving decision quality and user satisfaction, and informative guidance was more effective in user learning about the problem domain, whereas dynamic guidance was more effective than predefined guidance in improving decision quality and user learning; and both suggestive guidance and dynamic guidance reduced the decision time.     

The Effects of Memory Structure on Using Rule-Based Expert Systems for Training: A Framework and an Empirical Test*

One justification for eliciting and representing the judgment strategy of an expert in a rule-based expert system (RBES) is to facilitate knowledge transfer to individuals with less expertise. However, prior research suggests complexities and potential problems when using RBESs for training. In this paper, a conceptual framework of user learning from RBESs is presented. It is proposed that learning may be ineffective when the problem representation of the RBES is inconsistent with the user's mental representation of the task-domain knowledge. An experiment was conducted to examine the effects of consistency (inconsistency) between the problem-solving strategy of RBESs and individuals' memory structures. Groups of subjects whose memory structure either matched or did not match two RBESs' problem-solving strategies were examined using an internal control evaluation task. The results suggest that learning was facilitated only for groups with congruence between the RBES's problem-solving strategy and a subject's memory structure.     

The Design of a Knowledge-Based Decision Support System to Support the Information Analyst in Determining Requirements

Decision support system (DSS) researchers and designers continue to look for unstructured organizational tasks where there is a critical need for intelligent computer-based support. One such decision task is information requirements determination. Requirements determination is recognized as the most crucial phase of the systems development life cycle. Unfortunately, most methodologies and CASE tools focus only on how to specify the requirements once they are determined. There is very little computer support for the process of determining requirements. This paper discusses the conceptual design and development of a knowledge-based DSS to support information analysts in the critical decision task of determining requirements for the design of effective information systems. The expert modeling support system has the expertise to assist the analyst in studying the organization as a whole and in modeling the system under study in the context of the overall organization's goals and needs. The focus of the paper is on the problems associated with building the knowledge base component of the intelligent decision support system. A prototype implementation of the system is described.     

EFFECTS OF CONFLICTING ANALYSES ON MANAGERIAL DECISION MAKING: A LABORATORY EXPERIMENT

Researchers in a variety of disciplines have recommended the use of multiple conflicting data analyses to improve managerial decision making through the challenging of assumptions. This study deals with the effects of single data analyses and conflicting analyses on managers' solutions to a case analysis task. Results showed that managers who received conflicting analyses produced solutions with higher expected profits than those who received single analyses. Implications for the use of analytic decision aids are noted.     

Strategic Assessment Model (SAM): A Multiple Criteria Decision Support System for Evaluation of Strategic Alternatives*

The evaluation of strategic alternatives is a particularly difficult task. This difficulty is due to the complexities inherent in the evaluation process and the lack of structured information. The evaluation process must consider a multitude of relevant information from both the internal and external environments of the organization. Various analytical and normative models have helped decision makers utilize large volumes of information in strategic evaluation; however, most of these models have some limitations. We present a multiple criteria decision support system, called strategic assessment model (SAM), that addresses some of the limitations inherent in the existing models. SAM captures the decision maker's beliefs through a series of sequential, rational, and analytical processes. The environmental forces—decomposed into internal, task, general opportunities, and threats—are used along with the analytic hierarchy process (AHP), subjective probabilities, the entropy concept, and utility theory to enhance the decision maker's intuition in evaluating a set of strategic alternatives.     

DECISION SUPPORT SYSTEMS FOR ILL-STRUCTURED PROBLEMS: AN EMPIRICAL STUDY

Decision support systems (DSSs) are more complex than most other traditional decision-aid systems. For what types of problems are they more effective, and what design characteristics make them more effective? The laboratory experiment reported here examined the effect of three design characteristics of these systems in the context of decision makers faced with ill-structured problems. The characteristics were presence or absence of decision-aid heuristics, degree of interaction between the user and the system, and whether or not the system was computerized. The dependent variables were (1) quality of user performance, (2) user productivity of ideas, (3) user confidence in the quality of his/her performance, (4) user satisfaction with the decision aid or support system, (5) changes in user attitude toward the problem addressed, and (6) changes in user attitude toward computers. Use of heuristics and increased interaction had positive effects on decision quality, user productivity, and attitude toward computers; they had negative effects on user confidence, satisfaction, and attitude toward the problem addressed. Whether or not the system was computerized did not have a significant effect on any dependent variable. The findings concerning negative effects, in particular, suggest the need for research on the design of heuristics for addressing ill-structured problems—heuristics that will deliver the positive but not the negative effects observed in this study. The findings also suggest the need for research on how to benefit from computers in the context of solving ill-structured problems.     

AN EMPIRICAL ASSESSMENT OF COMPUTER-ASSISTED DECISION ANALYSIS*

Decision analysis tools often are used in semistructured and ill-structured situations. While some researchers have argued that computerized decision analysis programs may improve decision quality in such situations, research support for this assertion is weak. In this laboratory study, business students used a computerized decision-analysis program with short strategic-management cases to prepare decision reports. Independent raters' perceptions of aided and unaided decision performance were contrasted, attitudes of students toward the program were assessed, individual differences were correlated with attitudes, and the program's impact on students' perceptions of their decision processes and performance was examined. Student attitudes toward the computerized aid were favorable, and individual differences in reactions generally were as predicted. However, the program did not improve the independent ratings of students' decision reports and recommendations. These findings suggest that computerized decision aids should be adopted cautiously. If such aids result in positive user affect and heightened decision confidence without corresponding improvements in decision quality, they may be dysfunctional. Rigorous research methodologies which incorporate independent evaluations of analyses and decisions are recommended for use in future research on computerized decision-analysis programs.     

An Empirical Investigation of Information Search Strategies with Implications for Decision Support System Design

Understanding how people search through and combine information before making decisions is an important concern in the study of decision making and in the design of decision support systems (DSS). The purpose of this study is to examine DSS search strategies in relation to the body of empirical research on information load. Prior research is examined in relation to basic empirically-testable hypotheses and compared to the results of this study as a way of validating the procedures used here. The transfer of information load empirical research to a DSS scenario is confirmed statistically. Then, results of an analysis of information search patterns under DSS conditions are described. Finally, implications for DSS design are discussed in terms of possible search support mechanisms for specific search strategies.     

The Determinants of DSS Success: An Integrated Model*

The results of a field study investigating the determinants of decision support systems (DSS) success are presented. A multivariate model was developed and tested using multiple regression hierarchical analysis on responses from 118 DSS users. Several specific hypotheses are also proposed and tested. Data analyses indicate that DSS success, as measured by DSS satisfaction and perceived benefits, depends on several factors: previous user experience with DSS, user involvement, user training, top management support, information sources, the level of managerial activity, and task structure, difficulty and interdependence. Based on the research results, several suggestions are made to improve the likelihood of successful DSS implementation.     

An Empirical Investigation of the Effects of Decision Aids on Problem-Solving Processes*

Despite the growing number of investigations examining decision support systems (DSS), relatively few empirical studies have evaluated the effects of DSS on problem-solving processes. This study uses protocol analysis to investigate the impact of a specific decision aid on problem-solving processes in a semi-structured problem. Results indicate that decision aids influence the problem-solving processes of decision makers. The effect of a decision aid is found to be contingent on familiarity with the decision aid, task familiarity, and the interaction of these two factors. Suggestions for incorporating interaction effects and problem-solving processes into future examinations are proposed.