Development and Measurement Validity of a Task-Technology Fit Instrument for User Evaluations of Information System

Although many researchers have raised concerns about the lack of theoretical underpinnings for the user evaluation construct and the lack of measurement validity for specific instruments measuring it, the construct is still widely used in IS research. This paper reports on the development and measurement validity of a diagnostic tool used in recently published research to evaluate an organization's overall information systems and services. A distinctive feature of this instrument is that it is conceptually based on the task-technology fit theory in which the correspondence between information systems functionality and task requirements leads to positive user evaluations, and positive performance impacts. Specifically, the instrument development was guided by a task model of managerial decision making using recorded organizational information. This model suggested the different information systems functionalities required by users for that task, which then serve as the basis for a “task-technology fit” (TTF) instrument. The instrument thus measures the degree to which an organization's information systems and services meet the information needs of its managers. An extensive test of the measurement validity of the instrument is conducted using a sample of 357 users in 10 companies. It is found to have excellent reliability and discriminant validity for 12 dimensions of TTF, and also exhibits strong predictive validity. Finally, the instrument is compared to two other well-known user evaluation instruments. Though no single instrument can meet all needs, the instrument presented here should be considered an attractive option for researchers and practitioners seeking to measure the effectiveness of organizational information systems.     

Decisions to Outsource Information Systems Functions: Testing a Strategy-Theoretic Discrepancy Model

In recent years, the decision to outsource information systems (IS) functions has become a viable strategic alternative in managing the increasingly complex IS functions. In this study, the IS outsourcing phenomenon is conceptualized as a strategic decision in the organization. Drawing on resource-based theories, resource dependence theories, and other theories of strategic management, a discrepancy model of this decision is developed. Relationships between a number of strategy-theoretic factors and the IS outsourcing decision are hypothesized. These factors include IS resource performance discrepancies manifested in the form of gaps in information quality, IS support quality, IS cost effectiveness and financial performance, as well as the strategic orientation of the firm. Results of the study indicate that, while cost consideration and the firm's financial performance are not associated with the IS outsourcing decision, difficulties in providing good information outputs and IS support services are associated with the decision. Overall, the findings suggest that the current trend toward outsourcing represents a continuing evolution of the IS function as it attempts to fulfill its traditional mission of providing high quality information resource to the firm. When the performance of the delivered resource begins to slip in the current environment of rising expectation and technological complexity, outsourcing may become a strategic response of necessity. The paper concludes with a discussion of the implications of the results for practicing managers and suggestions for future research.     

Swiftness and Intensity of Administrative Innovation Adoption: An Empirical Study of TQM in Information Systems

Considerable attention has been directed toward developing a more complete understanding of innovation adoption by Information Systems (IS) departments. Much of this research has focused on the adoption of technological innovations, and limited research has focused on the adoption of administrative innovations. This paper focuses on an administrative innovation that is increasingly becoming popular among IS departments, namely Total Quality Management (TQM) in systems development. A synthesis of the IS innovation and TQM literatures was conducted to identify environmental, organizational, and task‐related factors that should relate to both the swiftness and the intensity of TQM adoption. The relationships between the identified variables and TQM adoption were examined using data collected from 123 IS departments in Fortune 1000 firms and large government agencies in the U.S. The results indicate that TQM adoption in systems development is influenced by the host organization's quality orientation and factors internal to the IS department including IS management support for quality, the presence of a separate quality assurance function, and the structural complexity of the IS department. Implications of this study for theory, future research, and practice are discussed.     

Planning Implications of the Information Systems Strategic Grid: An Empirical Investigation

In recent years, information systems (IS) planning has come to represent a key IS management tool to practitioners and researchers. Concurrently, there has been increasing recognition by organizations that their IS can be viewed as a strategic resource. This research is an attempt to explore the relationship between these two aspects of IS. It analyzes, in an empirical setting, the contingent nature of some IS planning-related variables in the context of the strategic relevance of an organization's IS. The basis for the empirical test is the IS strategic grid framework developed by Cash, McFarlan, McKenney, and Vitale [6]. A questionnaire survey method is employed to elicit information on the strategic orientation of IS and IS planning issues within respondent organizations. The difference in emphasis on various planning aspects in organizations with different IS environments is analyzed. Results of the analysis indicate that there is empirical support for the strategic grid framework and that there are differences in planning aspects among organizations, depending on their location in the grid. The implications of these findings for IS and IS planning are then discussed.     

Dogmatism and Systems Development Personnel Performance

This research explores the relationship between the personal variable, dogmatism, and the performance evaluations of systems analysts within a Fortune 500 company. The Rokeach Dogmatism Scale was used to measure the analysts’ level of dogmatism. The analysts’ performance measurement was the product of the company's annual evaluation process. The results of the research indicate that personnel with the highest dogmatism scores generally receive the extreme performance ratings. The results of this study provide further support to consider dogmatism as an important personal variable influencing systems analyst job performance.     

Task-Structure Relationship of Information Systems Development Subunit: A Congruence Perspective*

Several studies have examined the influence of organizational context (e.g., organization size, environment, strategy) on information systems (IS) performance/success. But very few studies have examined IS structure, and those studies have focused on a particular element of IS structure (e.g., decentralization). Thus, the multidimensional structural configuration of IS as an organizational subunit has escaped research attention. Further, the nature of the task to be performed by the IS subunit as a contextual factor has been virtually ignored in the IS research literature. In this paper, we first develop a congruence model for the task-structure relationship in an information systems development (ISD) subunit setting. Then we test this model as a multivariate relationship followed by subordinate bivariate analyses. Analysis of data from 41 hospitals’ ISD subunits reveals that a congruence relationship between task context and the structure of an ISD subunit is present. Our findings also show that an exclusive association exists between uncertainty in the task environment and the decision-making structure. Likewise, the association between equivocality of task content and the control structure of the ISD subunit is also exclusive. Implications of these findings for decision makers and researchers are discussed.     

An Instrument for Assessing the Organizational Benefits of IS Projects

This paper reports the development of an instrument to measure the organizational benefits of IS projects. The basis for this instrument was a published framework that suggests three categories of such benefits: strategic, informational, and transactional. In a cross-sectional study of 178 IS projects proposed and approved for development, this framework was operationalized and empirically tested using the measurement model of LISREL. The analysis culminated in the validation and refinement of the these categories. The final instrument offers items under three separate subdimensions of strategic benefits: competitive advantage, alignment, and customer relations. Informational benefits are similarly comprised of information access, information quality, and information flexibility. Finally, transactional benefits are also shown to be of three types: communications efficiency, systems development efficiency, and business efficiency. Implications of this multidimensional instrument for IS practitioners and researchers are discussed.     

Decision Processes for Developing Strategic Applications of Information Systems: A Contingency Approach*

Strategic applications of information systems (IS) are considered by IS researchers to be determined by contextual factors such as environmental uncertainty, and influenced by attributes of the processes preceding them, such as planning and top management support. For better management of the process leading to these applications, it is essential to understand the relationship between process attributes and contextual factors. Utilizing a contingency approach, this article takes a step toward such an understanding. Based on successful strategic IS applications from 81 large companies, it examines the relationship between the context of these applications and the decision-making process leading to them. The results indicate that the external environment is related to whether a rational or political decision-making process is used. The IS function seems to influence the use of the decision process implied by IS researchers, one in which the top management champions the strategic application, while the IS managers contribute by conducting in-depth analysis. However, the organization structure was not related to any decision process attribute. The implications of these findings for future research and practice are also discussed.     

Developing Measures for Assessing the Organizational Impact of Information Technology: A Comment on Mahmood and Soon's Paper

In a recent study, Mahmood and Soon [26] identified and operationalized a set of variables that can be potentially impacted by information technology. The lack of factor analysis to verify the dimensionality of different variables is a notable omission in the study. This paper describes and employs the limited-information factor analysis approach and the minimum average partial criterion for testing the unidimensionality of different variables in the Mahmood and Soon study. While reconfirming the construct validity of eight variables, our results question the unidimensionality of two original variables. Further testimony is provided in favor of our results by citing previous research studies.     

Integration Between Business Planning and Information Systems Planning: Validating a Stage Hypothesis

This paper proposes and empirically validates a stages of growth model for the evolution of Information Systems Planning (ISP). A questionnaire survey of senior IS executives is used to gather information pertaining to the stages of growth model, which includes measurement of the nature and level of integration between business planning (BP) and ISP. The del test is used to validate empirically benchmark variables for each stage of BP-ISP integration. The results support the stages of growth model of BP-ISP integration and the benchmark variables are generally found to be successful in predicting the stage of integration.     

Information Systems Management Issues: Reporting and Relevance*

Reports of key information systems issues have been published over the last two decades in many journals. Leading IS journals (e.g., MIS Quarterly, Information & Management, among others) have published key IS management issue reports every three or four years over the last 15 years, and will probably continue to provide such reports in the future. Although these reports claim to provide decisional guidance to practitioners, researchers, consultants, etc., the authors in their experience have noted concerns about their usefulness. While not questioning the validity of the methods and analysis conducted in these studies, we address two important questions in this article: the manner of reporting of the key issue results (which might be misleading), and the relevance of the results (are they providing what they intended to?). We hope that our discussion will provide new perspectives in making resource allocation decisions to both readers and authors of key issue articles.     

The Information Problems in Organizations: A Research Model for the Value of Information and Information Systems*

Growing recognition of the need for better management information system (MIS) theory and practice centers around problems with identifying the value of information and information systems in organizations. This paper addresses this issue by showing that existing theory in organizational economics and industrial organization directly relates to information problems in organizations (their nature, and their effects in terms of organization structure and performance). The paper categorizes these problems and associated theories into four distinct classes and shows that MIS research and practice can be mapped to information problems of each type. Because the economic theory pertinent to specific information problems includes the nature of the organizational response to information constraints, it also provides insight into the nature of the improvement to be realized if information constraints can be overcome. These improvements can then be interpreted as the value of the associated information systems which suggests using the resultant classification scheme to map between information problems and system value. The paper shows that this methodology provides a more appropriate level of focus than either traditional normative or business value methods, especially because many improvements have structural as well as performance consequences.     

System Development Project Effectiveness: Problem-Solving Competence as a Moderator Variable*

In spite of end-user programming, for the foreseeable future users and information systems (IS) specialists will build systems together. A better understanding of this partnership (which has a troubled history) will help IS managers and users improve their management skills. This study investigates the relative contributions to project effectiveness of problem-solving competence, the quality of the working relationship, and the quality of the development process. Seventy-five medium-sized projects in 16 organizations were investigated using individual, one hour, structured interviews with the IS project leader and with the primary user. Results suggest that more complex models should be used to study system development. Moreover, problem-solving competence demonstrated a complex effect on project effectiveness.     

An Empirical Test of the Strategic-Grid Model of Information Systems Planning*

Contingency models of information systems planning predict that no single planning approach will suit all organizations' needs. Little empirical research has been undertaken, however, to evaluate this prediction. Accordingly, we used McFarlan, McKenney, and Pyburn's (1983) strategic-grid model to study the information systems planning problems encountered by 49 governmental agencies. Twenty-seven agencies were required to follow a planning approach best suited to organizations that had a high level of dependence on both their existing and proposed systems. We predicted that agencies not having these characteristics would encounter the most problems with the approach. The remaining 22 agencies could choose their own planning approach. We studied this latter group to determine whether the problems encountered by the first group could be attributed to the mandated approach. Overall, the empirical results obtained were equivocal. Some results indicated that more planning problems were encountered by agencies in which the mandated approach was not appropriate to their position in the strategic grid. Other results were not supportive of this proposition. More work needs to be undertaken, therefore, to evaluate the predictive and explanatory power of contingency models of information systems planning. In addition, our research indicates a need to develop more rigorous theories of information systems planning behaviors, to improve the instruments needed to measure these behaviors, to explore the relationship between information systems planning behaviours and organizational effectiveness, to investigate how organizational learning impacts planning behaviors, and to determine the types of information systems planning problems that diffuse through organizations and those that remain localized.     

Toward Development of the End User Computing Construct in a University Setting*

The authors identify and define the end user computing (EUC) construct and operationalize the construct through development of a valid, reliable measure called the EUC activities scale. This scale is designed to distinguish specific, discrete computer activities performed on the job. The research analyzes the validity and reliability of the instrument based on data obtained from 776 university employees who participated in the initial, exploratory application. Exploratory factor analysis of the scale resulted in five factors. In order of increasing complexity of job-related computer activities, these factors are: beginning EUC activities, intermediate EUC activities, advanced EUC activities, EUC facilitation activities, and EUC infrastructure activities. The factor analysis, reliabilities, and known-groups analysis provide support for the construct validity of the instrument when used in a university setting. Suggestions are provided for further use of the scale in business environments to increase generalizability, refine the instrument, and help establish a coherent theoretical basis for further research in EUC.     

A process-oriented perspective of IS success: Examining the impact of IS on operational cost

Effective measurement of information system (IS) success continues to be a top concern for both practitioners and researchers. Labeled the “productivity paradox”, previous attempts to quantify the benefits of IS have often resulted in inconclusive or inconsistent results. The present study contributes to IS success research through the development and empirical testing of a process-oriented model of IS success. Using an operational perspective and combining primary and secondary measures, this study examines the influence of lower-level intangible IS and information technology (IT) benefits on higher-level financial measures. Additionally, we introduce IS plan quality as an antecedent to the model's input variables. Primary survey data from CIOs of primarily Fortune 2000 size firms are combined with secondary data, total operational cost, from Compustat to facilitate the examination of the IS benefits. This study also addresses shortcomings cited in other studies that might have contributed to the productivity paradox by including control variables in the model, defining a research perspective—the operational success perspective—and clearly articulating the level of analysis as the organizational level. The results supported a process-oriented view of the benefits from IS and showed how the effects of IS along a path can lead to better organizational performance, in this case, lower overall costs. These results are discussed and future research implications are presented.     

PERCEIVED USEFULNESS OF INFORMATION: A PSYCHOMETRIC EXAMINATION*

The perceived usefulness of information is an important construct for the design of management information systems. Yet an examination of existing measures of perceived usefulness shows that the instruments developed have not been validated nor has their reliability been verified. In this paper a new instrument for measuring two dimensions of perceived usefulness is developed. The results of an empirical study designed to test the reliability and construct validity of this instrument in a capital-budgeting setting are presented.     

Critical Dimensions of Strategic Information Systems Planning

Strategic information systems planning (SISP) is the process whereby an organization establishes a long-range plan of computer-based applications in order to achieve its goals. A number of problems can potentially impede information systems planners as they carry out the process. A survey of 80 firms who completed the SISP process revealed that the problems constitute five factors: the organization, implementation, database, hardware, and cost. A discriminant analysis showed that three factors—the organization, implementation, and database—best distinguished satisfied from dissatisfied information systems planners. A research application of the five factors suggested that organizations with more sophisticated business planning endure significantly less severe hardware and implementation problems.     

Information Systems Adaptation and the Successful Implementation of Advanced Manufacturing Technologies*

Few plants have initial success with advanced manufacturing technologies (AMTs), and adaptations are commonly required after installing new technologies. We especially know little about the outcomes of adaptations that manufacturers regularly employ in the context of AMTs. Sociotechnical theory and the technology literature suggest that AMTs commonly require four types of adaptations including those to process technologies, human resources, operational structures, and information systems. This study investigates which of these four adaptations and their interactions lead to improved operational performance. Data were collected on 104 plants that implemented a single type of manufacturing AMT. Findings indicate that information systems (IS) adaptation during the course of AMT implementation was the most important action. Surprisingly, human resource and process technology adaptations were linked to lower performance, and no interactions had any clear effects. The results also show that practitioners underestimated the relevance of IS adaptations during contemporary or “second-generation” AMT implementation projects and overemphasized the need for shop floor changes.     

Replication of Measures in Information Systems Research: The Case of IS SERVQUAL*

The Information Systems (IS) discipline should recognize potential pitfalls in replication of research studies. This response to a note appearing in this issue by Van Dyke, Prybutok, and Kappelman (1999) aims to serve two purposes. First, this note will address the problems we have with the Van Dyke et al. (1999) attempted replication of our own previous work. Second, we hope it serves to establish more generally an awareness in the IS field for the potential to create and accept the “illusion of replicability” when this has not really been established.