The Determinants of DSS Success: An Integrated Model*

The results of a field study investigating the determinants of decision support systems (DSS) success are presented. A multivariate model was developed and tested using multiple regression hierarchical analysis on responses from 118 DSS users. Several specific hypotheses are also proposed and tested. Data analyses indicate that DSS success, as measured by DSS satisfaction and perceived benefits, depends on several factors: previous user experience with DSS, user involvement, user training, top management support, information sources, the level of managerial activity, and task structure, difficulty and interdependence. Based on the research results, several suggestions are made to improve the likelihood of successful DSS implementation.     

Data and Model Management in a Generalized MCDM-DSS*

Drawing upon the choice models developed in the multiple criteria decision making (MCDM) area, this paper proposes an architecture for designing an intelligent decision support system (DSS) that is intended to aid in making choices among multiple alternatives along multiple dimensions. It argues that effective support can be provided to the decision maker when the knowledge-based DSS is capable of dynamically selecting choice models appropriate to the domain and context of a particular problem being specified by the decision maker, and of properly applying them to the problem solution. Development of a prototype intended to partially represent application of the architecture is described. The paper concludes with suggestions for research extensions.     

The Interaction Effects Between General Thinking Skills and an Interactive Graphics-Based DSS to Support Problem Structuring*

An interactive graphics-based problem-structuring aid, GISMO, based on concepts from structural modeling and motivated by research in imagery theory and cognitive psychology was incorporated into a decision support system. A laboratory experiment was conducted to investigate the relationships between three individual general thinking skills (GTS) and the effectiveness of the problem-structuring tool. A theoretical model of the roles of GTS and GISMO in problem formulation and a methodology for testing the model are presented. The model posits that the effectiveness of visual problem-structuring aids in formulating or understanding complex problems is influenced by specific cognitive skills (i.e., visual-thinking, verbal, and logical-reasoning skills). The results indicate GISMO use is related to higher levels of problem-structure understanding, regardless of verbal and logical-reasoning skills. Also, the results indicate a strong interaction effect between GISMO use and visual-thinking skill. Subjects classified as high visual thinkers benefited more from the use of GISMO than the low visual-thinking subjects. The results of the experiment lend support for the findings of a previous study of GISMO reported in the literature, and the image theorist view that the ability to create and use visual, mental images is related to better problem-solving performance.     

PROJECT IPMA: APPLYING DECISION SUPPORT SYSTEM DESIGN PRINCIPLES TO BUILDING EXPERT-BASED SYSTEMS*

There currently is no widely accepted design framework for building expert systems. Knowledge engineers describe the process in terms of a set of heuristics or as a series of phases. Most paradigms are based on post-development reflections. This paper explores an expert-based system constructed using decision support system design techniques. An actual system in experimental use at the Department of Entomology, Oregon State University, was designed using the approach presented. The system, Integrated Pest Management Assistant (IPMA), will be used by extension agents to help orchard owners make decisions regarding pear tree pest infestations and to offer remedial strategies for eradicating potentially destructive insects.     

Automatic Generation of Symbolic Multiattribute Ordinal Knowledge-Based DSSs: Methodology and Applications*

A learning-by-example algorithm, the ordinal learning model (OLM), that automatically generates symbolic rule-bases from examples was applied to four real-world multiattribute ordinal problem domains. The model automatically generates consistent and irredundant symbolic classification rules that mimic, in many aspects, the behavior of human subjects who solved similar problems during empirical studies. The OLM's performance is compared with those of regression analysis and with C4, a well-known symbolic learning-by-example decision tree building algorithm. The OLM uses mainly comparison operations and does not attempt to optimize the rule-bases it generates. Yet, the results show that the OLM's predictions are very accurate and the resulting rule-bases are relatively compact. The time required for constructing the rule-bases via the OLM was very competitive as well.     

Combining Neural Networks and Statistical Predictions to Solve the Classification Problem in Discriminant Analysis*

A number of recent studies have compared the performance of neural networks (NNs) to a variety of statistical techniques for the classification problem in discriminant analysis. The empirical results of these comparative studies indicate that while NNs often outperform the more traditional statistical approaches to classification, this is not always the case. Thus, decision makers interested in solving classification problems are left in a quandary as to what tool to use on a particular data set. We present a new approach to solving classification problems by combining the predictions of a well-known statistical tool with those of an NN to create composite predictions that are more accurate than either of the individual techniques used in isolation.     

Automating Knowledge Acquisition and Refinement for Decision Support: A Connectionist Inductive Inference Model*

An important application of expert systems technology is to provide support for nonstructured decision making. Usually, nonstructured decision making is characterized by heavy reliance on heuristic knowledge, which is very difficult to articulate or document, and therefore traditional knowledge acquisition approaches are not very successful. The quality and effectiveness of an expert system supporting unstructured decision making is affected when traditional knowledge acquisition approaches are used. To alleviate this problem a model is proposed that combines inductive inference and neural network computing, and an example is presented that illustrates the potential of this model in unstructured decision support.     

COMPUTER GRAPHICS AS DECISION AIDS: DIRECTIONS FOR RESEARCH*

Computer-generated graphics are becoming increasingly available to decision makers. Despite claims on the part of vendors that the use of graphics will improve decision speed and quality over traditional methods of data display, the available evidence is far from supportive. Initial studies show graphics to be no more effective in communicating information than tables. Correct interpretation of graphical displays appears to require training, which most users lack. Furthermore, there is evidence that those features that make a graph visually attractive—such as color, design complexity, and realism—may actually detract from accurate comprehension. This paper summarizes the literature dealing with the human use of graphics, develops several propositions based on persistent trends in the literature, and suggests directions for future research.     

Some Empirical Findings on Short-Term Forecasting: Technique Complexity and Combinations

The purpose of this research is to determine if prior findings that favor simple forecasting techniques and technique combinations hold true in a short-term forecasting environment, where demand data can be quite volatile. Twenty-two time series of daily data from a real business setting are used to test one-period ahead forecasts, the epitome of short-term forecasting. The time series vary systematically as to data volatility and forecast difficulty. Forecast accuracy is measured in terms of both mean absolute percentage error (MAPE) and mean percentage error (MPE).     

The Application of Neural Networks and a Qualitative Response Model to the Auditor's Going Concern Uncertainty Decision*

An auditor gives a going concern uncertainty opinion when the client company is at risk of failure or exhibits other signs of distress that threaten its ability to continue as a going concern. The decision to issue a going concern opinion is an unstructured task that requires the use of the auditor's judgment. In cases where judgment is required, the auditor may benefit from the use of statistical analysis or other forms of decision models to support the final decision. This study uses the generalized reduced gradient (GRG2) optimizer for neural network learning, a backpropagation neural network, and a logit model to predict which firms would receive audit reports reflecting a going concern uncertainty modification. The GRG2 optimizer has previously been used as a more efficient optimizer for solving business problems. The neural network model formulated using GRG2 has the highest prediction accuracy of 95 percent. It performs best when tested with a small number of variables on a group of data sets, each containing 70 observations. While the logit procedure fails to converge when using our eight variable model, the GRG2 based neural network analysis provides consistent results using either eight or four variable models. The GRG2 based neural network is proposed as a robust alternative model for auditors to support their assessment of going concern uncertainty affecting the client company.     

Validating Expert Systems: A Demonstration Using Personal Choice Expert,A Flexible Employee Benefit System*

A method for validating expert systems, based on validation approaches from psychology and Turing's “imitation game,” is demonstrated using a flexible employee benefits expert system. Psychometric validation has three aspects: the extent to which the system and expert decisions agree (criterionrelated validity), the inputs and processes used by experts compared to the system (content validity), and differences between expert and novice decisions (construct validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies the Turing Test. Personal Choice Expert (PCE) was designed to help employees of a Fortune 500 firm choose benefits in their flexible benefits system. Its recommendations do not significantly differ from those given by independent experts. Hence, if the system-independent expert agreement (criterion-related validity) were the only standard, PCE could be considered valid. However, construct analysis suggests that re-engineering may be required. High intra-expert agreement exists only for some benefit recommendations (e.g., dental care and long-term disability) and not for others (e.g., short-term disability, accidental death and dismemberment, and life insurance). Insights offered by these methods are illustrated and examined.     

Complexity Factors and Intuition-based Methods for Facility Network Design

Logistics managers frequently utilize decision support systems (DSS) to make facility network design decisions. Many DSS do not provide optimization capabilities, but instead rely on scenario evaluation as a means for developing solutions. We experimentally assessed the performances of decision makers, including experienced managers, who used four variants of a scenario evaluation-based DSS to solve realistically sized network design problems of varying complexities. Complexity factors included DSS attributes, problem size, network types, and demand dispersion patterns. Decision makers' performances were assessed relative to optimal solutions. Overall, the decision makers generated relatively high-quality solutions using the DSS variants. The type of design problem solved did not significantly impact problem-solving performance. However, performance degraded and variability in solution quality escalated as problem size was increased. The availability of incremental solution cost improvement cues in the DSS significantly improved solution quality and reduced performance variability. Iconic graphic enhancements to the DSS did not consistently affect performance. However, significant interactions existed among the effects of DSS graphics capabilities, DSS information cues, and problem attributes.     

Interface Style and Training Task Difficulty as Determinants of Effective Computer-Assisted Knowledge Transfer*

The transfer of expert knowledge to novices is one means of improving decision quality. Research needs to identify (1) the knowledge to be transferred to novices, and (2) the best method for transferring that knowledge. Studies that compare the judgment behavior of experienced and novice auditors address the first issue. The present study addresses the second issue in the context of using a computer-assisted training (CAT) program. CAT was selected for study because of evidence that it can both improve the effectiveness and reduce the costs of training. An experiment was conducted in which two factors were manipulated: (1) the design of the human-computer interface of the CAT program, and (2) the content of training tasks. The judgment of interest involved causal reasoning about the relationships between various internal control procedures and possible errors. The results indicate that alternative styles of interaction with a CAT program differ in terms of learning effectiveness. In addition, there was also evidence that training task content affected learning.     

Decisional Conflict and User Acceptance of Multicriteria Decision-Making Aids*

Despite the development of increasingly sophisticated and refined multicriteria decision-making (MCDM) methods, an examination of the experimental evidence indicates that users most often prefer relatively unsophisticated methods. In this paper, we synthesize theories and empirical findings from the psychology of judgment and choice to provide a new theoretical explanation for such user preferences. Our argument centers on the assertion that the MCDM method preferred by decision makers is a function of the degree to which the method tends to introduce decisional conflict. The model we develop relates response mode, decision strategy, and the salience of decisional conflict to user preferences among decision aids. We then show that the model is consistent with empirical results in MCDM studies. Next, the role of decisional conflict in problem formulation aids is briefly discussed. Finally, we outline future research needed to thoroughly test the theoretical mechanisms we have proposed.     

Judgment Error and Expert Critics in Forecasting Tasks*

This paper presents an approach to elicit and critique intuitive forecasts. It attempts to retain the valid component of experts' judgments while correcting some biases. Decision science and expert system techniques were unified with the design of an expert critiquing system. A real world application of the resulting critic verifies the approach. A discussion addresses how to create a similar critic for other forecast settings.     

User Perceptions of Decision Support Effectiveness: Two Production Planning Experiments*

Decision support systems continue to be very popular in business, despite mixed research evidence as to their effectiveness. We hypothesize that what-if analysis, a prominent feature of most decision support systems, creates an “illusion of control” causing users to overestimate its effectiveness. Two experiments involving a production planning task are reported which examine decision makers' perceptions of the effectiveness of what-if analysis relative to the alternatives of unaided decision making, and quantitative decision rules. Experiment 1 found that almost all subjects believed what-if analysis was superior to unaided decision making, although using what-if analysis had no significant effect on performance. Experiment 2 found that decision makers were indifferent between what-if analysis and a quantitative decision rule which, if used, would have led to significant cost savings. Thus, what-if analysis did create an illusion of control: decision makers perceived performance differences where none existed, and did not detect large differences when they were present. In both experiments, decision makers exhibited difficulty realizing that their positive beliefs about what-if analysis were exaggerated. Such misjudgments could lead people to continue using what-if analysis even when it is not beneficial and to avoid potentially superior decision support technologies.     

An Evaluation of the Role and Performance of a Decision Support System in Business Education*

With the rapid growth in end-user computing, there has been an increasing demand on business schools to train managers in the use of computers for effective decision support. While computer-aided instruction (CAI) has been in vogue for over a decade, there are very few studies that evaluate the impact of computers on decision-making skills in a business education context. The study makes use of a controlled experiment to evaluate the impact of computers on the accuracy and quality of decision making in a business decision support context. It also addresses the relationships between decision performance and various student characteristics such as aptitude, attitude, domain experience, domain expertise, gender, and system experience. The study revealed that computer support positively influenced both dimensions of decision performance.