The Influences of User Characteristics on Performance in a Logistics DSS Application

Decision Support Systems (DSS) are widely used in logistics decision applications, and a large number and variety of systems are commercially available. We investigate the contributions of user characteristics including experiences, data preferences, intuition, and effort to decision performance in a logistics DSS context. The study includes a laboratory experiment in which decision makers with varied experiences used a DSS to make facility network design decisions for problems of varying complexity. Two variants of the DSS are utilized in order to examine the interactions of a DSS decision aid with user characteristics. We find that intuition and effort are associated with decision-making performance. High analytic ability is not related to intuition, however. Education and previous experience are associated with performance. Yet these characteristics are also unrelated to intuition. Decision makers who highly value disaggregated data provided by the DSS tend to perform poorly. Also, the results suggest that the effects of users' experiences and preferences on performance are influenced by an analytical decision aid.     

A COMPARISON OF UNIT,SUBJECTIVE, AND REGRESSION MEASURES OF SECOND-LEVEL VALENCES IN EXPECTANCY THEORY

Two different decision-making exercises are compared with unit weights and with traditional Likert scales as measures of second-level valences in expectancy theory. Second-level valences are measured across different levels of outcomes in each exercise and compared to the traditional subjective- and unit-weight measures. In the first experiment, the job-preference decisions of 21 undergraduates are examined using a decision-making exercise involving 24 hypothetical jobs described in terms of three intrinsic instrumentalities at two levels. In the second experiment, a different 21 undergraduates make job-preference decisions in a decision-making exercise involving 27 hypothetical jobs described in terms of three extrinsic instrumentalities at three levels. The β-weight (standardized regression coefficient) measures of the second-level valences are found to be more powerful predictors of job-preference decisions than either subjective-weight or unit-weight measures, both before and after shrinkage. They also had higher cross validations, exhibited higher test-retest reliability, and allowed testing of a quadratic motivational effect in the three-level exercise.     

Evaluating and Refining Expert Critiquing Systems: A Methodology*

Expert critiquing systems are a rapidly growing class of intelligent decision support systems that apply artificial intelligence techniques to criticize a user's proposed solution to a problem. Critic programs now exist in the medical, engineering, programming, knowledge acquisition, word processing, and other domains. Critic refinement is a nontrivial activity that, even when done well, consumes a sizable fraction of the complete effort to build and deploy a critiquing system. To ease that effort, it is important to adopt a rigorous approach that allows one to reproducibly measure the degree of success of the current critic version and to predict which refinements will improve the critic further. The current article presents one such approach with actual case studies that illustrate its usage and elaborate selected aspects of the refinement process.