Validating Expert Systems: A Demonstration Using Personal Choice Expert,A Flexible Employee Benefit System*

A method for validating expert systems, based on validation approaches from psychology and Turing's “imitation game,” is demonstrated using a flexible employee benefits expert system. Psychometric validation has three aspects: the extent to which the system and expert decisions agree (criterionrelated validity), the inputs and processes used by experts compared to the system (content validity), and differences between expert and novice decisions (construct validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies the Turing Test. Personal Choice Expert (PCE) was designed to help employees of a Fortune 500 firm choose benefits in their flexible benefits system. Its recommendations do not significantly differ from those given by independent experts. Hence, if the system-independent expert agreement (criterion-related validity) were the only standard, PCE could be considered valid. However, construct analysis suggests that re-engineering may be required. High intra-expert agreement exists only for some benefit recommendations (e.g., dental care and long-term disability) and not for others (e.g., short-term disability, accidental death and dismemberment, and life insurance). Insights offered by these methods are illustrated and examined.     

A COMPARISON OF UNIT,SUBJECTIVE, AND REGRESSION MEASURES OF SECOND-LEVEL VALENCES IN EXPECTANCY THEORY

Two different decision-making exercises are compared with unit weights and with traditional Likert scales as measures of second-level valences in expectancy theory. Second-level valences are measured across different levels of outcomes in each exercise and compared to the traditional subjective- and unit-weight measures. In the first experiment, the job-preference decisions of 21 undergraduates are examined using a decision-making exercise involving 24 hypothetical jobs described in terms of three intrinsic instrumentalities at two levels. In the second experiment, a different 21 undergraduates make job-preference decisions in a decision-making exercise involving 27 hypothetical jobs described in terms of three extrinsic instrumentalities at three levels. The β-weight (standardized regression coefficient) measures of the second-level valences are found to be more powerful predictors of job-preference decisions than either subjective-weight or unit-weight measures, both before and after shrinkage. They also had higher cross validations, exhibited higher test-retest reliability, and allowed testing of a quadratic motivational effect in the three-level exercise.     

A Simulation of Partial Information Use in Decision Making: Implications for DSS Design

Information matrices are often the output produced by a decision support system. These matrices are a common method for expressing a decision situation under different decision-making scenarios. The decisions involved in designing a decision support system to generate the information matrix are important and involve several cost and benefit components. A designer needs guidance in making effective design decisions in this context. Such guidance can be provided by considering the relationships among specific design decisions, costs, and benefits. The general objective of this study is to provide a comprehensive framework for this purpose. This study is the first to develop and present a comprehensive cost-benefit framework for evaluating design decisions for a variety of scenarios. The specific objective of this research is to provide guidance regarding the number of available information dimensions to incorporate in a computer-based decision aid. Simulation experiments are conducted with a completely specified model based on the cost-benefit framework (including needed assumptions) to evaluate how many information dimensions to include for a specific information matrix size to achieve a balance between information use costs and decision quality. Based upon extensive simulation analyses for a hypothetical decision maker, the practical guideline found for designers is to include only the top half of the relevant information dimensions in any specific decision support system. Over a large number of repeating choice decisions, the savings in cognitive effort and information gathering costs clearly offset relatively minor losses in decision quality.     

Decision approach to workload distribution

Abstract. This paper deals with decisions of workload temporal distribution in scheduling discrete and diversified productions. A new way of formulating the scheduling problem is proposed, from which some concepts and tools are presented. The notion of time resource interval objects, TRIs, allows the management of technical (time and resource) aspects at the different levels of a hierarchical structuring of the set of decisions taken in the workshop, from ‘load distribution’ types, to ‘effective realization of the operations’ types. Constraint-based reasoning handles different TRIs corresponding to given kinds of decisions. It helps to highlight the bounds or limits to be respected while deciding, to remain consistent with an initial set of constraints, issued for example from an upper level of decisions. Decisions of load temporal distribution consist in readjustments of some time constraints on a set of planned operations, by taking into account the (or some more detailed) constraints on the resource(s) on which they have been planned, such as finite capacity and/or minimal profitability. The analysis on temporal proximities of the planned operations involves some particular structuring of the time axis into successive time intervals: these structures are associated with sets of temporal bounds, and are called adjacent decompositions of the time axis. Such a decomposition introduces some specific TRIs, associated with load constraints (coming from the planned operations), and resource constraints (coming from limited quantities of resource, or profitability concerns). By respecting the given time and resource constraints, they can ‘exchange’ some quantities of load according to communicating vessels processes. These phenomena have been modelled as bounded flows in a temporal network, and offer new flexible curves of load with finite capacities, to help the decision.     

TOWARD UNDERSTANDING THE CONTENTS OF THE “BLACK BOX” FOR PREDICTING COMPLEX DECISION-MAKING OUTCOMES

The purpose of this research is to show the usefulness of three relatively simple nonlinear classification techniques for policy-capturing research where linear models have typically been used. This study uses 480 cases to assess the decision-making process used by 24 experienced national bank examiners in classifying commercial loans as acceptable or questionable. The results from multiple discriminant analysis (a linear technique) are compared to those of chi-squared automatic interaction detector analysis (a search technique), log-linear analysis, and logit analysis. Results show that while the four techniques are equally accurate in predicting loan classification, chi-squared automatic interaction detector analysis (CHAID) and log-linear analysis enable the researcher to analyze the decision-making structure and examine the “human” variable within the decision-making process. Consequently, if the sole purpose of research is to predict the decision maker's decisions, then any one of the four techniques turns out to be equally useful. If, however, the purpose is to analyze the decision-making process as well as to predict decisions, then CHAID or log-linear techniques are more useful than linear model techniques.     

A comparison of simplified value function approaches for treating uncertainty in multi-criteria decision analysis

Uncertainty is present in many decisions where an action's consequences are unknown because they depend on future events. Multi-attribute utility theory (MAUT) offers an axiomatic basis for choice, but practitioners may prefer to use simpler decision models for transparency, ease of use, or other practical reasons. We identify some ‘simplified’ models currently in use and use a simulation experiment to evaluate their ability to approximate results obtained using MAUT. Our basic message is that avoiding assessment errors in the application of a simplified model is more important than the choice of a particular type of model, but that the best performance over a range of decision problems is from a model using a small number of quantiles.     

Understanding Organizational Safety Using Value-Focused Thinking

Decision analysis is recognized as the right way to make risk management decisions, using probabilistic techniques to assess the accident risk. It is also accepted that the decisions that individuals in the organization make affect the likelihood of an accident and thus managerial and organizational factors should be included in the risk modeling process. However, decision analytic techniques have not been used to understand the decisions that are made by these individuals. The initial domain for this research is marine transportation. We use the framework of value-focused thinking in order to understand safety decisions made within our research partner organization, a major domestic oil tanker operator. We describe the results of interviews held with managers and employees from this organization. Through these interviews, we sought to understand the values these experts apply in their roles within the organization and the objectives they seek to achieve to contribute to its overall safety performance. The end result is a framework that not only portrays the fundamental objectives of safe operations for various roles in the organization, but also interconnects these different decision contexts. We believe that this approach is fundamentally different from those used in previous work and that this is an interesting application of value-focused thinking.     

Risk,Ambiguity, and Decision Choice: Some Additional Evidence*

This paper extends prior research by jointly assessing the roles of risk attitude and tolerance for ambiguity in predicting choice. An experiment examined the effects of these variables on decisions made in four different scenarios. The four scenarios (treatment combinations) were generated by manipulating risk and ambiguity into two levels (high and low). The context was defined in terms of a sample size selection problem. The second issue explored was the effect of attitudes toward risk and ambiguity on decision confidence. The results indicate that (1) both risk attitude and ambiguity intolerance determined choice behavior, (2) the roles of these individual attitudes depend on the levels of the two treatment variables of risk and ambiguity, (3) the presence of ambiguity accentuates the perception of risk in individual subjects, and (4) decision makers who are less risk averse, and have more tolerance for ambiguity, display greater confidence in their choice. The paper discusses some of the managerial implications of the results.     

Toward Computerizing a Causal Modeling Approach to Strategic Problem Framing*

This paper proposes an object-oriented approach to the development of interactive software for the purpose of managerial problem solving. A prototype is being developed using CSM causal mapping to represent each manager's perceptions of the relationships between key variables of a firm's strategic situation. This paper suggests the design of GDSS that would enable a group of managers to discuss, learn from each other, and possibly develop consensus about decisions or their causes. Issues involving future development are discussed.     

PROJECT IPMA: APPLYING DECISION SUPPORT SYSTEM DESIGN PRINCIPLES TO BUILDING EXPERT-BASED SYSTEMS*

There currently is no widely accepted design framework for building expert systems. Knowledge engineers describe the process in terms of a set of heuristics or as a series of phases. Most paradigms are based on post-development reflections. This paper explores an expert-based system constructed using decision support system design techniques. An actual system in experimental use at the Department of Entomology, Oregon State University, was designed using the approach presented. The system, Integrated Pest Management Assistant (IPMA), will be used by extension agents to help orchard owners make decisions regarding pear tree pest infestations and to offer remedial strategies for eradicating potentially destructive insects.     

Subordinate Participation in Audit Budgeting Decisions: A Comparison of Decisions Influenced by Organizational Factors to Decisions Conforming with the Vroom-Jago Model*

This study examines the role of subordinate participation in the effectiveness of audit hours budget decisions in accounting firms. The study compares decisions influenced by organizational factors (such as organizational policies or the preferences of the superior) to decisions that conform to the Vroom-Jago (VJ) model. Our results indicate that actual level of participation used in budget decisions appears to be based primarily on perceived preferences of superiors and participation styles available under the circumstancesof the individual audit. However, results also indicate that decisions consistent with the Vroom-Jago model are characterized by higher decision quality and increased subordinatedevelopment. Since managers are apparently reluctant to use decision styles that conflictwith organizational factors, accounting firms may wish to gather information regarding the attributes of the decision context and use the VJ model to establish their available decision styles and determine the preferences of superiors. Doing so has the advantage of considering decision-specific attributes while enhancing the probability of manager compliance with a desired decision style.     

Some Comments on the Validity of EOQ Formula under Inflationary Conditions

In an earlier issue of Decision Sciences, Jesse, Mitra, and Cox [1] examined the impact of inflationary conditions on the economic order quantity (EOQ) formula. Specifically, the authors analyzed the effect of inflation on order quantity decisions by means of a model that takes into account both inflationary trends and time discounting (over an infinite time horizon). In their analysis, the authors utilized two models: Current-dollars model and Constant-dollars model. These models were derived, of course, by setting up a total cost equation in the usual manner then finding the optimum order quantity that minimizes the total cost. Jesse, Mitra, and Cox [1] found that EOQ is approximately the same under both conditions; with or without inflation. However, we disagree with the conclusion drawn by [2] and show that EOQ will be different under inflationary conditions, provided that the inflationary conditions are properly accounted for in the formulation of the total cost model.     

The Effects of Task Information and Outcome Feedback on Individuals' Insight into Their Decision Models

When making business decisions, people generally receive some form of guidance. Often, this guidance might be in the form of instructions about which inputs to the decision are most important. Alternatively, it might be outcome feedback concerning the appropriateness of their decisions. When people receive guidance in making difficult judgments, it is important that they do not confuse this guidance with insight into their own decision models. This study examined whether people confuse their actual decision model with task information and outcome feedback. Subjects predicted the likelihood that various hypothetical companies would experience financial distress and then reported the decision models they believed they had used. Their reported models were compared with their actual models as estimated by a regression of the subjects' predictions on the inputs to their decisions. In a 2times2 factorial design, some subjects were provided with task information regarding the relative importance of each input to their decisions while others were not. Some subjects were provided with outcome feedback regarding the quality of their decisions while others were not. The subjects tended to confuse the task information and outcome feedback with their actual decision models. Implications for the results are discussed.     

The Relationship between Procedural Rationality and Political Behavior in Strategic Decision Making

Despite the central place of rationality and political behavior in the decision-making literature, we know little about the relationship between these two dimensions. Can decisions be made using both rational and political methods, or must managers use one approach or the other? These questions were addressed in a study of 61 strategic decisions in 24 companies using a multiple-informant, structured interview protocol. Results indicate that procedural rationality and political behavior are independent dimensions of the strategic decision-making process. The implications of our findings for future strategic decision-making research are discussed.     

The Effects of Process and Outcome Similarity on Users' Evaluations of Decision Aids*

Decision aids (DA) used in online shopping contexts have been shown to improve users' product choices. Given that previous research (e.g., Byrne & Griffitt, 1973 ) has demonstrated the positive effects of perceived similarity on an individual's evaluation of others, this study investigates the effects of users' perceived similarity with a DA on their evaluations of that DA. More specifically, we investigate the effect of users' perceptions of the similarity between their own decision process and that followed by the DA to arrive at a recommendation (decision process similarity), as well as the similarity between the recommendations made by the DA and users' initial choices (outcome similarity), on their evaluations of the DA's usefulness and trustworthiness. The results of this study show that perceived process similarity exerts positive and significant effects on users' perceptions of the DA's usefulness and trustworthiness. However, the effects of perceived outcome similarity on trust are completely mediated by perceived process similarity. It is also observed that the level of the user's domain knowledge moderates the effects of perceived decision process similarity on both perceived usefulness and trustworthiness. These results have implications for DA design. It is important that designers consider the process by which users make decisions for themselves and align the DA's decision process with those of the user's, especially for the novice user. The full mediation of the effect of outcome similarity on trust by process similarity highlights how a similar decision process can mitigate some of the negative effects of outcome dissimilarity.     

Determining attribute weights using mathematical programming

Group decisions are an important element of successful knowledge management in organizations. Such decisions are difficult to make, however, especially when they involve a large set of attributes that require decision-makers to develop rankings. This paper presents a goal programming model for determining constrained regression estimates of attribute weights. The model is developed using pair-wise comparison ratings that are derived by using triads of the attributes. In addition, metrics are presented for measuring individual and group consensus. A specific application to the health care industry is presented to illustrate results that are obtained from the model.     

An Empirical Investigation of Information Search Strategies with Implications for Decision Support System Design

Understanding how people search through and combine information before making decisions is an important concern in the study of decision making and in the design of decision support systems (DSS). The purpose of this study is to examine DSS search strategies in relation to the body of empirical research on information load. Prior research is examined in relation to basic empirically-testable hypotheses and compared to the results of this study as a way of validating the procedures used here. The transfer of information load empirical research to a DSS scenario is confirmed statistically. Then, results of an analysis of information search patterns under DSS conditions are described. Finally, implications for DSS design are discussed in terms of possible search support mechanisms for specific search strategies.