A Simulation of Partial Information Use in Decision Making: Implications for DSS Design

Information matrices are often the output produced by a decision support system. These matrices are a common method for expressing a decision situation under different decision-making scenarios. The decisions involved in designing a decision support system to generate the information matrix are important and involve several cost and benefit components. A designer needs guidance in making effective design decisions in this context. Such guidance can be provided by considering the relationships among specific design decisions, costs, and benefits. The general objective of this study is to provide a comprehensive framework for this purpose. This study is the first to develop and present a comprehensive cost-benefit framework for evaluating design decisions for a variety of scenarios. The specific objective of this research is to provide guidance regarding the number of available information dimensions to incorporate in a computer-based decision aid. Simulation experiments are conducted with a completely specified model based on the cost-benefit framework (including needed assumptions) to evaluate how many information dimensions to include for a specific information matrix size to achieve a balance between information use costs and decision quality. Based upon extensive simulation analyses for a hypothetical decision maker, the practical guideline found for designers is to include only the top half of the relevant information dimensions in any specific decision support system. Over a large number of repeating choice decisions, the savings in cognitive effort and information gathering costs clearly offset relatively minor losses in decision quality.     

Decision Making in an Automated Environment: The Effects of Anonymity and Proximity with a Group Decision Support System*

Recent advances in information systems technology have made it possible to support the work of interacting groups using networked personal computers. A laboratory experiment was conducted using a group decision support system to evaluate effects of anonymity and proximity on group process in automated group problem solving. Twenty groups of four persons each performed an idea-generating task using an interactive electronic brainstorming program. This experiment's main findings were: (1) Group members working anonymously and apart generated more comments. (2) Working in the same room increased satisfaction. (3) Highest levels of perceived system effectiveness were reported under anonymity.     

The Effects of an Interactive Graphics-Based DSS to Support Problem Structuring

Graphical, interactive, problem-structuring tools were devised and incorporated into a DSS. These tools were based on concepts of structural modeling and motivated by propositions and findings in memory theory and imagery theory. A laboratory study was then conducted to investigate the relationship between decision-maker characteristics and use of the problem-structuring aid. The experiment was designed to determine whether use of the tool to assist in problem formulation led to a better understanding of problem structure. A moderately complex management simulation and its companion DSS were used in the experiment. Eighty-four subjects were divided into two groups: a control group that did not use the Graphical Interactive Structural Modeling Option (GISMO), and an experimental group that did. The results showed a statistically significant effect between cognitive ability and use of GISMO. Field independents (i.e., high cognitive ability) effectively combined GISMO use with image processing skills to understand the structure of the problem domain, but field dependents (i.e., low cognitive ability) were ineffective. There was very little difference in problem understanding in field dependents who used the tool and those who did not.     

An Empirical Investigation of the Effects of Decision Aids on Problem-Solving Processes*

Despite the growing number of investigations examining decision support systems (DSS), relatively few empirical studies have evaluated the effects of DSS on problem-solving processes. This study uses protocol analysis to investigate the impact of a specific decision aid on problem-solving processes in a semi-structured problem. Results indicate that decision aids influence the problem-solving processes of decision makers. The effect of a decision aid is found to be contingent on familiarity with the decision aid, task familiarity, and the interaction of these two factors. Suggestions for incorporating interaction effects and problem-solving processes into future examinations are proposed.     

Automated Discovery in Managerial Problem Formulation: Formation of Causal Hypotheses for Cognitive Mapping

Development of knowledge acquisition techniques known as automated discovery systems has occurred in deep and narrow domains of knowledge. Automated discovery is the generation of new knowledge by a computer system on its own, without the help of another knowledge source. This paper describes research and validation of an automated discovery system for a wide and shallow domain—business management. The system continues recent advances in expert systems research which have enhanced cognitive mapping, a problem formulation tool. The system perceives the behavior of distal variables in the environment through probabilistic cues-to-causality, and generates previously unknown hypotheses by aggregating the probabilities into a single criterion of causal relatedness. The system is validated against the source code of a simulated managerial environment, and causal relationships posited by decision makers experienced in the play of the gaming simulator.     

The Interaction Effects Between General Thinking Skills and an Interactive Graphics-Based DSS to Support Problem Structuring*

An interactive graphics-based problem-structuring aid, GISMO, based on concepts from structural modeling and motivated by research in imagery theory and cognitive psychology was incorporated into a decision support system. A laboratory experiment was conducted to investigate the relationships between three individual general thinking skills (GTS) and the effectiveness of the problem-structuring tool. A theoretical model of the roles of GTS and GISMO in problem formulation and a methodology for testing the model are presented. The model posits that the effectiveness of visual problem-structuring aids in formulating or understanding complex problems is influenced by specific cognitive skills (i.e., visual-thinking, verbal, and logical-reasoning skills). The results indicate GISMO use is related to higher levels of problem-structure understanding, regardless of verbal and logical-reasoning skills. Also, the results indicate a strong interaction effect between GISMO use and visual-thinking skill. Subjects classified as high visual thinkers benefited more from the use of GISMO than the low visual-thinking subjects. The results of the experiment lend support for the findings of a previous study of GISMO reported in the literature, and the image theorist view that the ability to create and use visual, mental images is related to better problem-solving performance.     

The Effectiveness of Graphical Presentation for Information Extraction: A Cumulative Experimental Approach*

Three laboratory experiments were conducted to assess the relative strengths and weaknesses of bar, symbol, and line graphs for performing a variety of elementary information extraction tasks using two dependent variables, time and accuracy. The findings indicate that the degree of support provided by a particular graph format for a particular data extraction task depends on the matching of format and task in terms of their anchoring characteristics. Anchoring, in this context, refers to the phenomenon that specific and diverse parts of a graph are segmented by the reader to act as salient and relevant cues, or anchors, when different classes of information are to be extracted from the graph. A data extraction task has high x-value (y-value) anchoring if the x-axis (y-axis) component is represented in the question as either a given value or an unknown value. Conversely, a task has low x-value (y-value) anchoring if the x-axis (y-axis) component is not represented in the question as either a given value or as an unknown value. Data extraction accuracy was not significantly affected by presentation format. Bars provided the best time performance for data extraction tasks having high anchoring on both axes but were not appropriate for tasks having low anchoring on either the y axis or both the x and y axes. Line graphs tended to be worse in terms of time performance for tasks having high anchoring on both axes although they were as fast or better than other representations for tasks having low anchoring on both axes. Symbol plots appeared to possess anchoring characteristics associated with both bars and line graphs. Symbols (as with bars) tended to produce a time performance superior to that of line graphs for tasks having high anchoring on both axes; and (as with line graphs) symbols allowed faster results than bar graphs for tasks having low anchoring on either the y axis or both the x and y axes.     

EFFECTS OF CONFLICTING ANALYSES ON MANAGERIAL DECISION MAKING: A LABORATORY EXPERIMENT

Researchers in a variety of disciplines have recommended the use of multiple conflicting data analyses to improve managerial decision making through the challenging of assumptions. This study deals with the effects of single data analyses and conflicting analyses on managers' solutions to a case analysis task. Results showed that managers who received conflicting analyses produced solutions with higher expected profits than those who received single analyses. Implications for the use of analytic decision aids are noted.     

Decisional Conflict and User Acceptance of Multicriteria Decision-Making Aids*

Despite the development of increasingly sophisticated and refined multicriteria decision-making (MCDM) methods, an examination of the experimental evidence indicates that users most often prefer relatively unsophisticated methods. In this paper, we synthesize theories and empirical findings from the psychology of judgment and choice to provide a new theoretical explanation for such user preferences. Our argument centers on the assertion that the MCDM method preferred by decision makers is a function of the degree to which the method tends to introduce decisional conflict. The model we develop relates response mode, decision strategy, and the salience of decisional conflict to user preferences among decision aids. We then show that the model is consistent with empirical results in MCDM studies. Next, the role of decisional conflict in problem formulation aids is briefly discussed. Finally, we outline future research needed to thoroughly test the theoretical mechanisms we have proposed.     

Fading Frames: Data Presentation and Framing Effects*

This article describes three experiments that investigate the impact of data presentation on framing, which is the differential evaluation of objectively equivalent information given different frames of reference. The experiments were designed to vary the amount of information and the format of presentation (tables versus graphs) provided to the decision maker to gain an understanding of the mechanism underlying framing. The inferred information model proposed by Johnson and Levin [11] is utilized to explain the results of the three experiments in terms of the redistribution of weights attached by the decision makers to the attributes of the prospective outcomes. The results of the three experiments indicate that framing effects are observed only when the presentation of data facilitates the shifting of weights among the attributes of the available alternatives. The results further indicate that framing is eliminated as more information is displayed. However, presentation of additional information does not eliminate framing when a graphical format is utilized. Finally, no framing effects are observed when uneven amounts of information are used to describe two alternatives in the same decision task.     

A Computerized Approach to Decision Process Tracing for Decision Support System Design*

We propose the use of computerized process tracing (CPT) tools as an appropriate approach for monitoring the information acquisition and evaluation phase of specific decision processes. CPT tools are unobtrusive and seem particularly relevant for evaluating certain decision tasks that may be supported by decision support systems (DSS). CPT tools can be an important component of DSS development. An information systems research taxonomy developed by previous researchers [29] [36] is used to position research work involving the methodology of CPT. Using a critique suggested by Libby [28], CPT tools are evaluated and compared to alternative process tracing tools. A brief empirical example using CPT is provided, and future uses relative to DSS are suggested. The appendix includes an example of a specific CPT tool.     

DUAL-METRIC MEASUREMENT OF THE IMPACT OF COMPLEXITY ON DECISION QUALITY*

Both sophisticated and unsophisticated subjects were provided either a simple or a complex cue set at random on which to base decisions. A dual-metric approach that consisted of measuring interrater consistency and cross-item variance for each subject category was used to compare the decision quality of the subjects. Although complexity and decision quality were inversely related, the sophisticated subjects performed relatively better using the complex cue set; the unsophisticated subjects performed better using the simple cue set. These findings support the notion that decision-maker and task compatibility are important to the assessment of decision quality.     

THE PREDICTIVE ABILITY CRITERION IN EXPERIMENTAL JUDGMENT RESEARCH IN ACCOUNTING*

Previous experimental judgment research in accounting has been interpreted as supportive of the linear model as an appropriate representation of decision-making behavior in nearly all of the tasks investigated. Moreover, nonlinear models have come to be viewed as adding relatively little predictive power over that provided by the linear model, even in tasks considered to be inherently nonlinear. These conclusions were based largely upon evaluating the predictive ability of the linear model in terms of statistics measuring the proportion of variance accounted for by the model. In the present paper it is argued that, since these statistics are not independent of the experimental design, it is not clear whether the high correlations are indicative of the model's success in representing decision-making processes or instead are more the result of various features of the experimental design. It is suggested that correlational tests be supplemented with qualitative tests of the predictive ability of a model. Implications for accounting are discussed.     

COMPUTER GRAPHICS AS DECISION AIDS: DIRECTIONS FOR RESEARCH*

Computer-generated graphics are becoming increasingly available to decision makers. Despite claims on the part of vendors that the use of graphics will improve decision speed and quality over traditional methods of data display, the available evidence is far from supportive. Initial studies show graphics to be no more effective in communicating information than tables. Correct interpretation of graphical displays appears to require training, which most users lack. Furthermore, there is evidence that those features that make a graph visually attractive—such as color, design complexity, and realism—may actually detract from accurate comprehension. This paper summarizes the literature dealing with the human use of graphics, develops several propositions based on persistent trends in the literature, and suggests directions for future research.     

An Examination of Two Screen/Report Design Variables in an Information Recall Context*

Screen/report design for output organization is still very much a common sense-driven activity. This paper identifies a preliminary set of factors beyond display format for examination as screen/report design variables. A laboratory experiment was conducted to assess the influence of one of these factors, spatial layout of information, along with display format in an information recall context. Individual imagery orientation (verbalizer/visualizer) was used as a moderating variable in the experimental design. The findings indicate that spatial layout of information significantly influenced performance when the recall task involved pattern detection. In fact, a central location of data in a document was the most conducive to pattern recall performance. Further, terminal items facilitated pattern recall better than early input items. With respect to display format, the results support previous findings that a tabular display enhances performance for point-value recall tasks while graphs exhibit an edge over tables for pattern types of recall tasks. Overall, the verbalizers demonstrated a complete supremacy in performance. Interaction between individual imagery orientation and display format indicates that, while the verbalizers and the mixed type were immune to display format variations, the graphical stimulus enhanced the pattern recall performance of the visualizers.     

Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature*

A considerable amount of research has been conducted over a long period of time into the effects of graphical and tabular representations on decision-making performance. To date, however, the literature appears to have arrived at few conclusions with regard to the performance of the two representations. This paper addresses these issues by presenting a theory, based on information processing theory, to explain under what circumstances one representation outperforms the other. The fundamental aspects of the theory are: (1) although graphical and tabular representations may contain the same information, they present that information in fundamentally different ways; graphical representations emphasize spatial information, while tables emphasize symbolic information; (2) tasks can be divided into two types, spatial and symbolic, based on the type of information that facilitates their solution; (3) performance on a task will be enhanced when there is a cognitive fit (match) between the information emphasized in the representation type and that required by the task type; that is, when graphs support spatial tasks and when tables support symbolic tasks; (4) the processes or strategies problem solvers use are the crucial elements of cognitive fit since they provide the link between representation and task; the processes identified here are perceptual and analytical; (5) so long as there is a complete fit of representation, processes, and task type, each representation will lead to both quicker and more accurate problem solving. The theory is validated by its success in explaining the results of published studies that examine the performance of graphical and tabular representations in decision making.     

Data and Model Management in a Generalized MCDM-DSS*

Drawing upon the choice models developed in the multiple criteria decision making (MCDM) area, this paper proposes an architecture for designing an intelligent decision support system (DSS) that is intended to aid in making choices among multiple alternatives along multiple dimensions. It argues that effective support can be provided to the decision maker when the knowledge-based DSS is capable of dynamically selecting choice models appropriate to the domain and context of a particular problem being specified by the decision maker, and of properly applying them to the problem solution. Development of a prototype intended to partially represent application of the architecture is described. The paper concludes with suggestions for research extensions.     

The Determinants of DSS Success: An Integrated Model*

The results of a field study investigating the determinants of decision support systems (DSS) success are presented. A multivariate model was developed and tested using multiple regression hierarchical analysis on responses from 118 DSS users. Several specific hypotheses are also proposed and tested. Data analyses indicate that DSS success, as measured by DSS satisfaction and perceived benefits, depends on several factors: previous user experience with DSS, user involvement, user training, top management support, information sources, the level of managerial activity, and task structure, difficulty and interdependence. Based on the research results, several suggestions are made to improve the likelihood of successful DSS implementation.     

Tax Professionals Belief Revision: The Effects of Information Presentation Sequence,Client Preference,and Domain Experience

Based on a belief revision model recently proposed by Hogarth and Einhorn [35], the present research investigates the effects of information presentation orders on tax professionals’ belief revisions about ambiguous tax treatments. Also examined in the study are the impacts of the tax professionals’ experience and the client's tax preference on the belief revision. The results show that the experienced tax professionals’ belief revisions are affected by the presentation order manipulations, but unaffected by the client preference manipulations. In contrast, the inexperienced tax professionals’ belief revisions display a reverse pattern.     

PERCEIVED INFORMATION STRUCTURE: IMPLICATIONS FOR DECISION SUPPORT SYSTEM DESIGN

Top-level decision making in business organizations is characterized by high degrees of uncertainty, incomplete information, and conflicting objectives. To support top-level decision making effectively, decision support systems (DSSs) have been proposed. Information supplied by a DSS must be selective in that not all possible information sets may be feasibly or economically represented in the data base. This study suggests that discovery of perceptual complexity (dimensionality) of information items, and the subsequent categorization of decision makers having the same perceptions of those information items, is a first step in the ultimate design of an effective DSS. Through the use of multidimensional scaling in a field setting, this study shows the feasibility of creating relatively homogeneous groups of decision makers according to the content and number of dimensions associated with various information items. Further results of the research suggest that information can be tailored to classes of users, which has cost-benefit implications as well as the potential to improve the quality of the resultant decisions.