Insights from Attempts to Validate a Multi-Attribute Model of Problem Definition Quality

Problem definitions are often used as part of the problem structuring process in decision sciences. We report an attempt to validate a multi-attribute model of problem definition quality in the context of an experiment based upon the definition of the ‘alcohol problem’ on a campus. Three independent raters assessed values for the attributes of the model for 57 written problem definitions, and their rating was compared to three subjective ‘expert’ ratings made by individuals with some responsibility for alcohol policy. Our study shows that the experts exhibited considerable stakeholder bias in their assessments of the definitions, suggesting that a definition that is evaluated as good may not be subjectively acceptable to others. We present some evidence that the best definitions are those that provide direction with regard to solution, neither suggesting firm solutions to the problem nor being too open-ended. It is shown that the multi-attribute model is a reasonable tool for eliminating poorer definitions. Further, we relate our work to recent conceptual developments in understanding problem definition.     

DECISION SUPPORT SYSTEMS FOR ILL-STRUCTURED PROBLEMS: AN EMPIRICAL STUDY

Decision support systems (DSSs) are more complex than most other traditional decision-aid systems. For what types of problems are they more effective, and what design characteristics make them more effective? The laboratory experiment reported here examined the effect of three design characteristics of these systems in the context of decision makers faced with ill-structured problems. The characteristics were presence or absence of decision-aid heuristics, degree of interaction between the user and the system, and whether or not the system was computerized. The dependent variables were (1) quality of user performance, (2) user productivity of ideas, (3) user confidence in the quality of his/her performance, (4) user satisfaction with the decision aid or support system, (5) changes in user attitude toward the problem addressed, and (6) changes in user attitude toward computers. Use of heuristics and increased interaction had positive effects on decision quality, user productivity, and attitude toward computers; they had negative effects on user confidence, satisfaction, and attitude toward the problem addressed. Whether or not the system was computerized did not have a significant effect on any dependent variable. The findings concerning negative effects, in particular, suggest the need for research on the design of heuristics for addressing ill-structured problems—heuristics that will deliver the positive but not the negative effects observed in this study. The findings also suggest the need for research on how to benefit from computers in the context of solving ill-structured problems.     

A New Approach to the Group Ranking Problem: Finding Consensus Ordered Segments from Users’ Preference Data

The group ranking problem involves constructing coherent aggregated results from users’ preference data. The goal of most group ranking problems is to generate an ordered list of all items that represents the user consensus. There are, however, two weaknesses to this approach. First, a complete list of ranked items is always output even when there is no consensus or only a slight consensus. Second, due to similarity of performance, in many practical situations, it is very difficult to differentiate whether one item is really better than another within a set. These weaknesses have motivated us to apply the clustering concept to the group ranking problem, to output an ordered list of segments containing a set of similarly preferred items, called consensus ordered segments. The advantages of our approach are that (i) the list of segments is based on the users’ consensuses, (ii) the items with similar preferences are grouped together in the same segment, and (iii) the relationships between items can be easily seen. An algorithm is developed to construct the consensus of the ordered segments from the users’ total ranking data. Finally, the experimental results indicate that the proposed method is computationally efficient, and can effectively identify consensus ordered segments.     

A Credit Limit Decision Model for Inventory Floor Planning and Other Extended Trade Credit Arrangements*

This paper analyzes the dealership credit limit problem in terms of the valuation of a Markov process of cash flows with sequential credit decisions over an infinite planning horizon. The formulation distinguishes between the upper bound on credit applicable at the account formation stage and the upper bound applicable to periodic reorders. The result is a closed form solution to the problem which serves as a criterion function for approving or denying credit on a customer-by-customer basis. Data for a sample of manufacturing firms are employed to estimate typical ranges for criterion function parameters. Upper bounds on credit limits are then calculated and graphically presented for median parameter values as well as for values at the 5th and 95th percentiles for the sample data. Finally, an empirical study is conducted of actual trade credit extended by firms. The results support the hypothesis that the variables in the decision model are important determinants of the amount of trade credit outstanding.     

Data Driven Modeling: An Application in Scheduling

This paper presents a data driven modeling (DDM) approach to certain types of optimization problems. DDM relinquishes control of the completed model to the user department rather than the operations research (OR) staff. The approach emphasizes development of models that are dependent on data maintained and understood by the users. The data base consists of coded user rules which describe when changes will occur in the problem structure and data which captures the generalization of the problem. Both the rules and data can be updated by user department personnel. These data drive a matrix generator controlled by the rules which uses the data base as input to generate the specific model formulation. This DDM system is designed by OR consultants or staff to allow independence of use along with low-cost and minimal-effort maintenance. The DDM approach is illustrated with an application to a real-world medical scheduling problem.     

Decisional Conflict and User Acceptance of Multicriteria Decision-Making Aids*

Despite the development of increasingly sophisticated and refined multicriteria decision-making (MCDM) methods, an examination of the experimental evidence indicates that users most often prefer relatively unsophisticated methods. In this paper, we synthesize theories and empirical findings from the psychology of judgment and choice to provide a new theoretical explanation for such user preferences. Our argument centers on the assertion that the MCDM method preferred by decision makers is a function of the degree to which the method tends to introduce decisional conflict. The model we develop relates response mode, decision strategy, and the salience of decisional conflict to user preferences among decision aids. We then show that the model is consistent with empirical results in MCDM studies. Next, the role of decisional conflict in problem formulation aids is briefly discussed. Finally, we outline future research needed to thoroughly test the theoretical mechanisms we have proposed.     

Toward Computerizing a Causal Modeling Approach to Strategic Problem Framing*

This paper proposes an object-oriented approach to the development of interactive software for the purpose of managerial problem solving. A prototype is being developed using CSM causal mapping to represent each manager's perceptions of the relationships between key variables of a firm's strategic situation. This paper suggests the design of GDSS that would enable a group of managers to discuss, learn from each other, and possibly develop consensus about decisions or their causes. Issues involving future development are discussed.     

A Decision Model for the Robot Selection Problem Using Robust Regression*

Industrial robots are increasingly used by many manufacturing firms. The number of robot manufacturers has also increased with many of these firms now offering a wide range of models. A potential user is thus faced with many options in both performance and cost. This paper proposes a decision model for the robot selection problem. The proposed model uses robust regression to identify, based on manufacturers' specifications, the robots that are the better performers for a given cost. Robust regression is used because it identifies and is resistant to the effects of outlying observations, key components in the proposed model. The robots selected by the model become candidates for testing to verify manufacturers' specifications. The model is tested on a real data set and an example is presented.     

An application of catastrophe theory to management science process

Practicing management scientists often complain of the alleged irrational behavior on the part of the managers they are trying to serve, particularly when a manager suddenly and unexpectedly shifts behavior from one of support to one of resistance. The authors feel such behavior is only believed to be irrational because the management scientist is using the wrong mental model when projecting past behavioral patterns into the future. This paper attempts to solve this problem by using the newly developed catastrophe theory to develop a different model where sudden shifts in behavior are considered perfectly rational and explainable. The basic implication that is drawn from this new model is that successful implementation of management science depends on a sequence of interactions with the ultimate model user and that care in the structuring of these interactions can greatly enhance the probability of eventual user acceptance.     

An Application of Expectancy Theory to Explain a Manager's Intention to Use a Decision Support System

A continuing gap exists between the capabilities of sophisticated computer-based information systems and the extent to which these systems are used by individuals. Studies which have examined the relationship between system utilization and various user, system, implementation, and organizational variables have provided few consistent findings. A new approach to this topic is suggested by a recent study by Davis, Bagozzi, and Warshaw [11], which indicates that individuals' intentions to use a system determine subsequent use. A large body of psychology-based research also supports this relationship between behavioral intentions and subsequent behavior. This study employs expectancy theory, which has often been used to examine behavioral intentions, to explain managers' intentions to use a decision support system (DSS). The results imply that the variables of the expectancy force model are determinants of a manager's behavioral intentions to use a DSS, and the variables of the expectancy valence model are determinants of the attractiveness of using a DSS to a manager.     

The impact of training on the formulation of ill-structured problems

Problem structuring heuristics and creative thinking techniques have been advanced as useful approaches for solving ill-structured problems. Unfortunately, little controlled experimentation has been done to test the effectiveness of these techniques. This paper describes an experiment in which the effects of training in the use of a problem-structuring heuristic and creative thinking on the quality and quantity of problem statements are investigated. The experiment illustrates that such training does have a positive impact on problem formulation quality and quantity.     

The Effectiveness of Decisional Guidance: An Empirical Evaluation

Decisional guidance is defined as how a decision support system (DSS) influences its users as they structure and execute the decision‐making process. It is assumed that decisional guidance has profound effects on decision making, but these effects are understudied and empirically unproven. This paper describes an empirical, laboratory‐experiment‐based evaluation of the effectiveness of deliberate decisional guidance and its four types. We developed and used a comprehensive model consisting of four evaluation criteria: decision quality, user satisfaction, user learning, and decision‐making efficiency. On these criteria, we compared decisional guidance versus no guidance, informative versus suggestive decisional guidance, and predefined versus dynamic decisional guidance. We found that deliberate decisional guidance was more effective on all four criteria; suggestive guidance was more effective in improving decision quality and user satisfaction, and informative guidance was more effective in user learning about the problem domain, whereas dynamic guidance was more effective than predefined guidance in improving decision quality and user learning; and both suggestive guidance and dynamic guidance reduced the decision time.     

Stochastic Judgments in the AHP: The Measurement of Rank Reversal Probabilities

This paper presents a methodology for analyzing Analytic Hierarchy Process (AHP) rankings if the pairwise preference judgments are uncertain (stochastic). If the relative preference statements are represented by judgment intervals, rather than single values, then the rankings resulting from a traditional (deterministic) AHP analysis based on single judgment values may be reversed, and therefore incorrect. In the presence of stochastic judgments, the traditional AHP rankings may be stable or unstable, depending on the nature of the uncertainty. We develop multivariate statistical techniques to obtain both point estimates and confidence intervals of the rank reversal probabilities, and show how simulation experiments can be used as an effective and accurate tool for analyzing the stability of the preference rankings under uncertainty. If the rank reversal probability is low, then the rankings are stable and the decision maker can be confident that the AHP ranking is correct. However, if the likelihood of rank reversal is high, then the decision maker should interpret the AHP rankings cautiously, as there is a subtantial probability that these rankings are incorrect. High rank reversal probabilities indicate a need for exploring alternative problem formulations and methods of analysis. The information about the extent to which the ranking of the alternatives is sensitive to the stochastic nature of the pairwise judgments should be valuable information into the decision-making process, much like variability and confidence intervals are crucial tools for statistical inference. We provide simulation experiments and numerical examples to evaluate our method. Our analysis of rank reversal due to stochastic judgments is not related to previous research on rank reversal that focuses on mathematical properties inherent to the AHP methodology, for instance, the occurrence of rank reversal if a new alternative is added or an existing one is deleted.     

A TIME-SERIES APPROACH FOR CONSTRUCTING EXPECTATIONS MODELS*

This article presents an efficient way of dealing with adaptive expectations models—a way that makes use of all the information available in the data. The procedure is based on multiple-input transfer functions (MITFs): by calculating lead and lag cross correlations between innovations associated with the variables in the model, it is possible to determine which periods have the greatest effects on the dependent variable. If information about k periods ahead is required, fitted values for the expectation variables are used to generate k-period-ahead forecasts. These in turn can be used in the estimation of the transfer function equation, which not only contains the usual lagged variables but also allows for incorporation of lead-fitted values for the expectation variables. The MITF identification and estimation procedures used are based on the corner method. The method is contrasted with the Almon distributed-lag approach using a model relating stock market prices to interest rates and expected corporate profits.     

Organizational constraints: a meta-analysis of a major stressor

The stressor organizational constraints, reflecting aspects of the work environment that inhibit or interfere with an individual's performance of job tasks, is prevalent in the workplace. This meta-analysis summarizes findings concerning the relationships of organizational constraints with other variables. Using five search methods, we located 84 research reports that provided data from 119 independent samples that contained 33,998 employed individuals. Fitting a stressor–strain framework, constraints had significant relationships with behavioural (counterproductive work behaviour), physical (somatic symptoms), and psychological (e.g. emotions and job dissatisfaction) strains and with well-being variables. Relationships were also found with all other work environment variables, and most personal characteristics (demographics and personality) except for gender. Moreover, a series of regression and relative weights analyses based on meta-analytic correlations showed that constraints had a unique contribution to strain beyond other stressors. The current meta-analysis shows that many commonly studied variables are related to organizational constraints, and it should more often be the centre of empirical research and theoretical development.     

Estimation of Nonlinear Models in the Presence of Measurement Error*

Techniques used in decision sciences and business research to estimate interactions between latent variables are limited in controlling for measurement error. This article uses a latent structure modeling approach that substantially controls for measurement error in nonlinear relationships. The results of this technique are compared to the results obtained applying hierarchical regression analysis and the impact of measurement error is assessed. The paper provides a unique assessment of the validity of the multi-attribute attitude model. The validity of the multiplicative rule in the model is supported.     

The Impact of a Structured-Argument Approach on Group Problem Formulation*

Despite advances in decision analysis and decision support systems, few formulaic procedures exist for undertaking problem formulation, particularly in group settings. This leaves managers with little procedural support for the important task of carefully structuring problems. In a laboratory experiment of 29 intact student teams, we contrasted two problem formulation methodologies: a structured argument approach (based on application of formal reasoning) and a group process approach (based on private idea generation prior to public sharing and evaluation of ideas). The structured argument approach took more time to use and failed to bring about more information search and equivocality reduction in group discussions. On the positive side, however, the structured argument approach led to a greater combination of both coverage of critical issues and consensus on those issues. Use of the structured argument approach also resulted in higher satisfaction with the problem definition and commitment to implementing results of the group meeting. Overall, the results reveal cost/benefit tradeoffs associated with developing argumentation-based procedures for problem formulation.     

Decision approach to workload distribution

Abstract. This paper deals with decisions of workload temporal distribution in scheduling discrete and diversified productions. A new way of formulating the scheduling problem is proposed, from which some concepts and tools are presented. The notion of time resource interval objects, TRIs, allows the management of technical (time and resource) aspects at the different levels of a hierarchical structuring of the set of decisions taken in the workshop, from ‘load distribution’ types, to ‘effective realization of the operations’ types. Constraint-based reasoning handles different TRIs corresponding to given kinds of decisions. It helps to highlight the bounds or limits to be respected while deciding, to remain consistent with an initial set of constraints, issued for example from an upper level of decisions. Decisions of load temporal distribution consist in readjustments of some time constraints on a set of planned operations, by taking into account the (or some more detailed) constraints on the resource(s) on which they have been planned, such as finite capacity and/or minimal profitability. The analysis on temporal proximities of the planned operations involves some particular structuring of the time axis into successive time intervals: these structures are associated with sets of temporal bounds, and are called adjacent decompositions of the time axis. Such a decomposition introduces some specific TRIs, associated with load constraints (coming from the planned operations), and resource constraints (coming from limited quantities of resource, or profitability concerns). By respecting the given time and resource constraints, they can ‘exchange’ some quantities of load according to communicating vessels processes. These phenomena have been modelled as bounded flows in a temporal network, and offer new flexible curves of load with finite capacities, to help the decision.     

INFLUENCE OF MISSING ATTRIBUTES ON SELECTING A BEST MULTIATTRIBUTED ALTERNATIVE

Alternatives involving many factors arc difficult to evaluate because of multiple underlying competing objectives. If evaluation is based on an incomplete set of factors, and if the purpose of the evaluation is to select a single overall best alternative, inferior alternatives may be selected with surprising frequency and/or severe negative impact. At the same time, sensitivity analysis of evaluation scores based on statistical criteria can easily mask the impact and the frequency of selection of inferior alternatives. In this paper, criteria appropriate to reflect the decision impact are developed and both the frequency and impact of the selection of inferior alternatives are demonstrated empirically. Previous studies based on statistical criteria [1] [9] indicated minimal impact on overall evaluation and selection. This paper demonstrates that high statistical criterion values coexist with frequent and/or serious errors of selection.