Does IO Economics Really Make the Case for Investments in IT: A Rejoinder

The preceding critique [1] of the paper “The Information Problems in Organizations: A Research Model for the Value of Information and Information Systems” [3], reaches general conclusions with respect to the original paper and the applicability of economics to MIS research. These conclusions are based on arguments developed at a different level of analysis than that used in the original paper, and without considering the goals and intent of the original paper. The differences between the critique and the original paper are highlighted.     

Test-Retest Reliability of User Information Satisfaction: A Comment on Galletta and Lederer's Paper

Galletta and Lederer [10] concluded that the short-form measure of user information satisfaction (UIS) lacks sufficient test-retest reliability. We question the validity of this conclusion since the method used by the authors for assessing test-retest reliability is not grounded in classical reliability theory. Analysis of their data suggests that the UIS instrument has adequate test-retest reliability.     

Developing Measures for Assessing the Organizational Impact of Information Technology: A Comment on Mahmood and Soon's Paper

In a recent study, Mahmood and Soon [26] identified and operationalized a set of variables that can be potentially impacted by information technology. The lack of factor analysis to verify the dimensionality of different variables is a notable omission in the study. This paper describes and employs the limited-information factor analysis approach and the minimum average partial criterion for testing the unidimensionality of different variables in the Mahmood and Soon study. While reconfirming the construct validity of eight variables, our results question the unidimensionality of two original variables. Further testimony is provided in favor of our results by citing previous research studies.     

The Effectiveness of Graphical Presentation for Information Extraction: A Cumulative Experimental Approach*

Three laboratory experiments were conducted to assess the relative strengths and weaknesses of bar, symbol, and line graphs for performing a variety of elementary information extraction tasks using two dependent variables, time and accuracy. The findings indicate that the degree of support provided by a particular graph format for a particular data extraction task depends on the matching of format and task in terms of their anchoring characteristics. Anchoring, in this context, refers to the phenomenon that specific and diverse parts of a graph are segmented by the reader to act as salient and relevant cues, or anchors, when different classes of information are to be extracted from the graph. A data extraction task has high x-value (y-value) anchoring if the x-axis (y-axis) component is represented in the question as either a given value or an unknown value. Conversely, a task has low x-value (y-value) anchoring if the x-axis (y-axis) component is not represented in the question as either a given value or as an unknown value. Data extraction accuracy was not significantly affected by presentation format. Bars provided the best time performance for data extraction tasks having high anchoring on both axes but were not appropriate for tasks having low anchoring on either the y axis or both the x and y axes. Line graphs tended to be worse in terms of time performance for tasks having high anchoring on both axes although they were as fast or better than other representations for tasks having low anchoring on both axes. Symbol plots appeared to possess anchoring characteristics associated with both bars and line graphs. Symbols (as with bars) tended to produce a time performance superior to that of line graphs for tasks having high anchoring on both axes; and (as with line graphs) symbols allowed faster results than bar graphs for tasks having low anchoring on either the y axis or both the x and y axes.     

Preference-Order Recursion for Finding Relevant Pure,Admissible and Optimal Statistical Decision Functions*

A preference-order recursion algorithm for obtaining a relevant subset of pure, admissible (non-dominated, efficient) decision functions which converges towards an optimal solution in statistical decision problems is proposed. The procedure permits a decision maker to interactively express strong binary preferences for partial decision functions at each stage of the recursion, from which an imprecise probability and/or utility function is imputed and used as one of several pruning mechanisms to obtain a reduced relevant subset of admissible decision functions or to converge on an optimal one. The computational and measurement burden is thereby mitigated significantly, for example, by not requiring explicit or full probability and utility information from the decision maker. The algorithm is applicable to both linear and nonlinear utility functions. The results of behavioral and computational experimentation show that the approach is viable, efficient, and robust.     

Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature*

A considerable amount of research has been conducted over a long period of time into the effects of graphical and tabular representations on decision-making performance. To date, however, the literature appears to have arrived at few conclusions with regard to the performance of the two representations. This paper addresses these issues by presenting a theory, based on information processing theory, to explain under what circumstances one representation outperforms the other. The fundamental aspects of the theory are: (1) although graphical and tabular representations may contain the same information, they present that information in fundamentally different ways; graphical representations emphasize spatial information, while tables emphasize symbolic information; (2) tasks can be divided into two types, spatial and symbolic, based on the type of information that facilitates their solution; (3) performance on a task will be enhanced when there is a cognitive fit (match) between the information emphasized in the representation type and that required by the task type; that is, when graphs support spatial tasks and when tables support symbolic tasks; (4) the processes or strategies problem solvers use are the crucial elements of cognitive fit since they provide the link between representation and task; the processes identified here are perceptual and analytical; (5) so long as there is a complete fit of representation, processes, and task type, each representation will lead to both quicker and more accurate problem solving. The theory is validated by its success in explaining the results of published studies that examine the performance of graphical and tabular representations in decision making.     

The Design of a Knowledge-Based Decision Support System to Support the Information Analyst in Determining Requirements

Decision support system (DSS) researchers and designers continue to look for unstructured organizational tasks where there is a critical need for intelligent computer-based support. One such decision task is information requirements determination. Requirements determination is recognized as the most crucial phase of the systems development life cycle. Unfortunately, most methodologies and CASE tools focus only on how to specify the requirements once they are determined. There is very little computer support for the process of determining requirements. This paper discusses the conceptual design and development of a knowledge-based DSS to support information analysts in the critical decision task of determining requirements for the design of effective information systems. The expert modeling support system has the expertise to assist the analyst in studying the organization as a whole and in modeling the system under study in the context of the overall organization's goals and needs. The focus of the paper is on the problems associated with building the knowledge base component of the intelligent decision support system. A prototype implementation of the system is described.     

Data Envelopment Analysis for Managerial Control and Diagnosis*

Data envelopment analysis (DEA) is a linear programming-based technique that converts multiple input and output measures into a single comprehensive measure of productive efficiency. This is accomplished via the construction of an empirically based production frontier and by the identification of peer groups. Each unit is evaluated by comparison against a composite unit that is constructed as a convex combination of other units in its peer group. DEA has now been applied in a variety of managerial contexts. In this paper we draw on theories of decision making, measurement and control, the mathematical properties of DEA, prior reported applications, and our own experience, to assess the potential of DEA as a general management tool. We first make the distinction between managerial diagnosis and control. We show how measurement requirements differ for these two managerial decision contexts, and argue that DEA has the potential to provide support in each context. Measurement and decision support criteria for each activity are then developed by reference to the literature on diagnosis and control. Based on its mathematical definition and properties, the relevant attributes of DEA are then derived. The technique is evaluated in each capacity by comparison to the appropriate set of criteria. This evaluation is supported with evidence from our experience with DEA in a large public-sector organization. We argue that the structural properties of DEA, critical managerial choices in its application, and situationally specific factors, interact to determine the strengths and limitations of DEA in each decision context. Implications for research and practice are discussed.     

Decision Processes for Developing Strategic Applications of Information Systems: A Contingency Approach*

Strategic applications of information systems (IS) are considered by IS researchers to be determined by contextual factors such as environmental uncertainty, and influenced by attributes of the processes preceding them, such as planning and top management support. For better management of the process leading to these applications, it is essential to understand the relationship between process attributes and contextual factors. Utilizing a contingency approach, this article takes a step toward such an understanding. Based on successful strategic IS applications from 81 large companies, it examines the relationship between the context of these applications and the decision-making process leading to them. The results indicate that the external environment is related to whether a rational or political decision-making process is used. The IS function seems to influence the use of the decision process implied by IS researchers, one in which the top management champions the strategic application, while the IS managers contribute by conducting in-depth analysis. However, the organization structure was not related to any decision process attribute. The implications of these findings for future research and practice are also discussed.     

Some Cautions on the Measurement of User Information Satisfaction*

User information satisfaction (UIS) is important because of its potential effects on MIS department goals, quality of user work life, and extent of voluntary usage of systems. Reliable measurement of UIS is important for providing evaluative information for both researchers and practitioners. This study used 92 managers and executives as subjects to compare the test/retest reliability of a widely used, 13-scale UIS instrument together with four summary questions under experimental and control conditions. The summary questions behaved more reliably than the detailed questions for all groups, perhaps because of problems with scale units and origins and with item heterogeneity. This suggests that researchers need more reliable measures of UIS and that practitioners need to exercise caution when collecting and interpreting UIS scores.     

Automated Discovery in Managerial Problem Formulation: Formation of Causal Hypotheses for Cognitive Mapping

Development of knowledge acquisition techniques known as automated discovery systems has occurred in deep and narrow domains of knowledge. Automated discovery is the generation of new knowledge by a computer system on its own, without the help of another knowledge source. This paper describes research and validation of an automated discovery system for a wide and shallow domain—business management. The system continues recent advances in expert systems research which have enhanced cognitive mapping, a problem formulation tool. The system perceives the behavior of distal variables in the environment through probabilistic cues-to-causality, and generates previously unknown hypotheses by aggregating the probabilities into a single criterion of causal relatedness. The system is validated against the source code of a simulated managerial environment, and causal relationships posited by decision makers experienced in the play of the gaming simulator.     

The Information Problems in Organizations: A Research Model for the Value of Information and Information Systems*

Growing recognition of the need for better management information system (MIS) theory and practice centers around problems with identifying the value of information and information systems in organizations. This paper addresses this issue by showing that existing theory in organizational economics and industrial organization directly relates to information problems in organizations (their nature, and their effects in terms of organization structure and performance). The paper categorizes these problems and associated theories into four distinct classes and shows that MIS research and practice can be mapped to information problems of each type. Because the economic theory pertinent to specific information problems includes the nature of the organizational response to information constraints, it also provides insight into the nature of the improvement to be realized if information constraints can be overcome. These improvements can then be interpreted as the value of the associated information systems which suggests using the resultant classification scheme to map between information problems and system value. The paper shows that this methodology provides a more appropriate level of focus than either traditional normative or business value methods, especially because many improvements have structural as well as performance consequences.     

A Comprehensive Model for Measuring the Potential Impact of Information Technology on Organizational Strategic Variables*

The existing literature on the impact of information technology (IT) does not include rigorous theory building or empirical studies. This research seems to be the first comprehensive investigation towards the development of an empirically validated comprehensive model for understanding the potential impact of IT on organizational strategic variables. More specifically, organizational and industrial variables that appeared to be affected by IT are identified, measured, and operationalized in the form of a comprehensive model. This study is based on structured interviews with a carefully selected sample of 31 strategic managers who had experience using IT for strategic decisions. In addition, the variables included in the model are well grounded in the information systems literature. The variables are then empirically validated and their reliabilities critically tested. A comprehensive model is derived from these validated variables. The model is a first step towards measuring the overall potential impact of IT on an organization. The model can also be used to gauge IT's potential impact on individual strategic variables. A set of hypotheses is also presented for future research. The hypotheses primarily relate to the impact of IT on organizational strategic performance. The model provides an empirically validated foundation for testing of such hypotheses.     

A Framework for Examining the Interface between Operations and Information Systems: Implications for Research in the New Millennium*

Even though much research has been published in operations and information systems, both functional areas find their roots in other disciplines. While operations management evolved from operations research in the 1960s, the field of information systems is of more recent vintage and traces its original roots to computer science. Both disciplines now naturally have come closer together as information and process-technology-based changes force manufacturing firms to become more efficient and customer focused. Market and technology-driven e-commerce initiatives that are likely to dominate business strategies in the future cannot be successfully achieved without a successful integration of operations and information systems. In this paper, we present a unifying framework that can be used to better understand the management of the functional interface between operations and information systems. We also categorize and highlight the contributions of the articles that appear in this special research focus issue. Finally, research directions that emerge from our understanding of this interface are outlined in an effort to stimulate further thinking and research that can advance our knowledge of this interface area.     

Production System Development for Expert Systems Using a Recursive Partitioning Induction Approach: An Application to Mortgage,Commercial, and Consumer Lending

The concepts of expert systems and decision support systems have received considerable attention recently. While systems have been proposed for various problem areas in business, difficulties still exist in the knowledge acquisition phase of development. This paper presents a recursive partitioning analysis (RPA) approach to knowledge acquisition. The RPA production system approach was applied to data sets representing the mortgage, commercial, and consumer lending problems. Comparison of the classification rates across these problems to the results of a generalized inductive inference production system (Quinlan's ID3 algorithm) and across the mortgage and commercial lending problems to traditional statistical modeling approaches indicated that the RPA approach provided superior results while using fewer variables.     

Swiftness and Intensity of Administrative Innovation Adoption: An Empirical Study of TQM in Information Systems

Considerable attention has been directed toward developing a more complete understanding of innovation adoption by Information Systems (IS) departments. Much of this research has focused on the adoption of technological innovations, and limited research has focused on the adoption of administrative innovations. This paper focuses on an administrative innovation that is increasingly becoming popular among IS departments, namely Total Quality Management (TQM) in systems development. A synthesis of the IS innovation and TQM literatures was conducted to identify environmental, organizational, and task‐related factors that should relate to both the swiftness and the intensity of TQM adoption. The relationships between the identified variables and TQM adoption were examined using data collected from 123 IS departments in Fortune 1000 firms and large government agencies in the U.S. The results indicate that TQM adoption in systems development is influenced by the host organization's quality orientation and factors internal to the IS department including IS management support for quality, the presence of a separate quality assurance function, and the structural complexity of the IS department. Implications of this study for theory, future research, and practice are discussed.     

A Simulation of Partial Information Use in Decision Making: Implications for DSS Design

Information matrices are often the output produced by a decision support system. These matrices are a common method for expressing a decision situation under different decision-making scenarios. The decisions involved in designing a decision support system to generate the information matrix are important and involve several cost and benefit components. A designer needs guidance in making effective design decisions in this context. Such guidance can be provided by considering the relationships among specific design decisions, costs, and benefits. The general objective of this study is to provide a comprehensive framework for this purpose. This study is the first to develop and present a comprehensive cost-benefit framework for evaluating design decisions for a variety of scenarios. The specific objective of this research is to provide guidance regarding the number of available information dimensions to incorporate in a computer-based decision aid. Simulation experiments are conducted with a completely specified model based on the cost-benefit framework (including needed assumptions) to evaluate how many information dimensions to include for a specific information matrix size to achieve a balance between information use costs and decision quality. Based upon extensive simulation analyses for a hypothetical decision maker, the practical guideline found for designers is to include only the top half of the relevant information dimensions in any specific decision support system. Over a large number of repeating choice decisions, the savings in cognitive effort and information gathering costs clearly offset relatively minor losses in decision quality.     

Stochastic Judgments in the AHP: The Measurement of Rank Reversal Probabilities

This paper presents a methodology for analyzing Analytic Hierarchy Process (AHP) rankings if the pairwise preference judgments are uncertain (stochastic). If the relative preference statements are represented by judgment intervals, rather than single values, then the rankings resulting from a traditional (deterministic) AHP analysis based on single judgment values may be reversed, and therefore incorrect. In the presence of stochastic judgments, the traditional AHP rankings may be stable or unstable, depending on the nature of the uncertainty. We develop multivariate statistical techniques to obtain both point estimates and confidence intervals of the rank reversal probabilities, and show how simulation experiments can be used as an effective and accurate tool for analyzing the stability of the preference rankings under uncertainty. If the rank reversal probability is low, then the rankings are stable and the decision maker can be confident that the AHP ranking is correct. However, if the likelihood of rank reversal is high, then the decision maker should interpret the AHP rankings cautiously, as there is a subtantial probability that these rankings are incorrect. High rank reversal probabilities indicate a need for exploring alternative problem formulations and methods of analysis. The information about the extent to which the ranking of the alternatives is sensitive to the stochastic nature of the pairwise judgments should be valuable information into the decision-making process, much like variability and confidence intervals are crucial tools for statistical inference. We provide simulation experiments and numerical examples to evaluate our method. Our analysis of rank reversal due to stochastic judgments is not related to previous research on rank reversal that focuses on mathematical properties inherent to the AHP methodology, for instance, the occurrence of rank reversal if a new alternative is added or an existing one is deleted.     

The Significance of Information Systems Research on Emerging Technologies: Seven Information Technologies that Promise to Improve Managerial Effectiveness*

Emerging information technologies are those excitingly novel advances that decision makers are just beginning to notice. These emerging products or processes may have been invented some time ago, but are only now beginning to reveal practical applications. This paper examines this collection of seven studies exploring emerging technologies, classifying them so the context of their results can be related to other emerging technologies. Once the research is classified and the contribution and managerial significance of the research is identified, it becomes apparent that certain barriers exist, holding back the technology from wider use and appreciation. Common obstacles include resistance to, or difficulty with, using the technology; uncertainty concerning the value of the technology; and the complexities involved with implementation. This paper concludes with recommendations for future research aimed at breaking down the barriers in order to move from the technological emergence phase to the technological sublime phase in which decision makers and other end users understand the technology, realize and appreciate its value, and put the technology to its best use.     

Decision Making in an Automated Environment: The Effects of Anonymity and Proximity with a Group Decision Support System*

Recent advances in information systems technology have made it possible to support the work of interacting groups using networked personal computers. A laboratory experiment was conducted using a group decision support system to evaluate effects of anonymity and proximity on group process in automated group problem solving. Twenty groups of four persons each performed an idea-generating task using an interactive electronic brainstorming program. This experiment's main findings were: (1) Group members working anonymously and apart generated more comments. (2) Working in the same room increased satisfaction. (3) Highest levels of perceived system effectiveness were reported under anonymity.