Development and Measurement Validity of a Task-Technology Fit Instrument for User Evaluations of Information System

Although many researchers have raised concerns about the lack of theoretical underpinnings for the user evaluation construct and the lack of measurement validity for specific instruments measuring it, the construct is still widely used in IS research. This paper reports on the development and measurement validity of a diagnostic tool used in recently published research to evaluate an organization's overall information systems and services. A distinctive feature of this instrument is that it is conceptually based on the task-technology fit theory in which the correspondence between information systems functionality and task requirements leads to positive user evaluations, and positive performance impacts. Specifically, the instrument development was guided by a task model of managerial decision making using recorded organizational information. This model suggested the different information systems functionalities required by users for that task, which then serve as the basis for a “task-technology fit” (TTF) instrument. The instrument thus measures the degree to which an organization's information systems and services meet the information needs of its managers. An extensive test of the measurement validity of the instrument is conducted using a sample of 357 users in 10 companies. It is found to have excellent reliability and discriminant validity for 12 dimensions of TTF, and also exhibits strong predictive validity. Finally, the instrument is compared to two other well-known user evaluation instruments. Though no single instrument can meet all needs, the instrument presented here should be considered an attractive option for researchers and practitioners seeking to measure the effectiveness of organizational information systems.     

Cautions on the Use of the SERVQUAL Measure to Assess the Quality of Information Systems Services

The SERVQUAL questionnaire (Parasuraman, Zeithaml, & Berry, 1988) is one of the preeminent instruments for measuring the quality of services as perceived by the customer. In a recent Decision Sciences article, Kettinger and Lee (1995) suggested the use of a modified SERVQUAL instrument to assess the quality of the services supplied by an information services provider. However, a number of problems with the SERVQUAL instrument are discussed in the literature. This article provides an illustrative example utilizing data collected from 138 executive and information systems professional customers of a multibillion dollar information services provider in order to examine the validity and reliability of Kettinger and Lee's (1995) modified SERVQUAL instrument. Results of analyses do not confirm the findings of Kettinger and Lee. Moreover, it appears that the use of difference scores in calculating SERVQUAL contributes to problems with the reliability, discriminant validity, convergent validity, and predictive validity of the measure. These findings suggest that caution should be exercised in the use of SERVQUAL scores and that further work is needed in the development of measures for assessing the quality of information services.     

On the Repeated Test-Retest Reliability of the End-User Computing Satisfaction Instrument: A Comment*

This comment extends the test-retest reliability of the end-user computing satisfaction (EUCS) instrument by Torkzadeh and Doll [10]. Whereas Torkzadeh and Doll [10] reportedstability for two hour and two week EUCS test-retest reliability, we investigate the test-retest reliability of the EUCS instrument at two points in time, separated by a two yearinterval. We assess the end user computing satisfaction of personal computer, as well as mainframe, administrative end users in a large public organization. The results of the repeated test-retest using differing application platforms add further support for the reliability of the EUCS measure and highlight some areas of concern for managers of information systems.     

Global Measures of Information Service Quality: A Cross-National Study*

Maintaining consistently high quality information services (IS) is a powerful means of increasing the overall efficiency and effectiveness of a global enterprise. This study introduces a global Information Services Function (ISF) Quality Framework that outlines the importance of both behavioral and procedural dimensions in planning, implementing and evaluating global IS quality. Based on this framework, the study investigates the cross-national psychometric properties of a behavioral measure of service quality in the IS context. Using a cross-national survey of IS customers from Korea, Hong Kong, the United States and the Netherlands, perceived ISF service quality was measured using a service quality measure (SERVQUAL) to determine cultural affects. Based on confirmatory factor analysis, support was found for four of the original five SERVQUAL quality dimensions in the U.S.A. and the Netherlands. However, the same four-dimensional measurement model did not fit the Hong Kong and Korean samples. Further factor analysis showed that the Hong Kong and Korean samples shared a somewhat similar factor structure that differs from the shared U.S.A. and Netherlands structure. These findings support previous research that has found an “Asian factor'’with differing definitions of IS Service quality. These findings suggest that the feasibility of standardized global ISF measurement depends heavily on the relative magnitude of cultural effects. Rather than merely applying the U.S.A. ISF/SERVQUAL measure, a localized version of the instrument may need to be developed that captures the unique nature of ISF service perceptions in internationally based subsidiaries or companies.     

The FAIRSERV Model: Consumer Reactions to Services Based on a Multidimensional Evaluation of Service Fairness

The current dominant conceptualization of consumer reactions to services is the SERVQUAL model. This article proposes the FAIRSERV model as an alternative or additional conceptualization of consumer reactions to services. FAIRSERV involves seeing service evaluation through the lens of organizational fairness (justice) theory applied to the relationship between the service consumer and the service provider. FAIRSERV is premised on the claim that, especially in relational service contexts, consumers are interested in service fairness as well as service quality (service favorableness) as represented by SERVQUAL. Service fairness or justice is a multidimensional construct based on equity theory. In this article, the FAIRSERV model is tested with the SERVQUAL model in the context of information system services. The two models are used to predict service satisfaction and repatronage intention. The FAIRSERV model appears to add a significant new set of predictors of service satisfaction and repatronage intention that should be considered in the future by service providers.     

Some Cautions on the Measurement of User Information Satisfaction*

User information satisfaction (UIS) is important because of its potential effects on MIS department goals, quality of user work life, and extent of voluntary usage of systems. Reliable measurement of UIS is important for providing evaluative information for both researchers and practitioners. This study used 92 managers and executives as subjects to compare the test/retest reliability of a widely used, 13-scale UIS instrument together with four summary questions under experimental and control conditions. The summary questions behaved more reliably than the detailed questions for all groups, perhaps because of problems with scale units and origins and with item heterogeneity. This suggests that researchers need more reliable measures of UIS and that practitioners need to exercise caution when collecting and interpreting UIS scores.     

A Psychometric Evaluation of the Expectations,Perceptions, and Difference‐Scores Generated by the IS‐Adapted SERVQUAL Instrument

This article describes the results of a study assessing the psychometric properties of the expectations and perceptions‐of‐performance instruments and the difference‐score data contained within the information systems (IS)‐Adapted SERVQUAL measurement paradigm. The central claim of this study is: In order for rational inferences to be made about service expectations, service performance perceptions, or the gap between them, each of the two instruments must exhibit reasonable psychometric properties in isolation before difference‐scores are taken. Analysis of data from a field study (N= 401) through structural equation modeling (SEM) techniques produces empirical evidence indicating that both of the instruments exhibit low psychometric quality and yet the difference‐scores exhibit “psychometric inflation.” That is, the quality of the difference‐score data is in many ways apparently superior to the raw data from both instruments. Negative conclusions are reached as to the efficacy of either individual instrument and, thus, the full IS‐Adapted SERVQUAL paradigm. Questions and prospects for further research in this important area of service quality measurement/management are presented, and a potentially rich future for IS service quality is outlined. It is strongly suggested that future IS service quality research be based on development of a new instrument, grounded in attributes endemic to IS services and developed using the best available development techniques.     

卷烟物流服务质量评估指标的实证遴选及其应用

SERVQUAL量表作为一个基本框架,被广泛应用于竞争性行业的服务质量评估,本文尝试将其应用到具有垄断特征的卷烟物流行业,初步遴选出了34个指标作为卷烟物流服务质量的理论评估指标.基于行业专家的两次较大规模的问卷调查数据,采用隶属度分析和重要性分析方法,对理论评估指标进行实证遴选,最终形成了由硬件保障性、配送可靠性、客户导向性和成本经济性4个构面共12个指标构成的卷烟物流服务质量评估体系,并采用层次分析法确定了指标权重.该指标体系体现了主观性与客观性、科学性与可操作性的结合.以浙江省9个地市卷烟物流服务质量评估为例,测试了评估体系的可行性.     

Computer Simulation Success: On the Use of the End-User Computing Satisfaction Instrument: A Comment

This comment is part of a comprehensive study to develop a contingency model of simulation success. The current study focuses on the psychometric stability of the end-user computing satisfaction (EUCS) instrument by Doll and Torkzadeh (1988) when applied to users of computer simulation. Using a survey of 411 users, the researchers provide evidence that the EUCS instrument is a valid and reliable measure of computer simulation success. Given this evidence, managers and simulation software product developers can confidently apply the instrument in the investigation of competing tools, features, and technologies.     

Test-Retest Reliability of the End-User Computing Satisfaction Instrument

The development of reliable and valid measures of user-information satisfaction is becoming increasingly important to both practitioners and researchers. Measures of reliability include internal consistency and stability. Recent research has raised concerns about the stability of user information satisfaction measures. Using the test-retest correlation method, this article assesses short- and long-range stability of a 12-item scale for measuring end-user computing satisfaction. It examines the stability of individual items and subscales as well as the 12-item instrument. The results suggest that the instrument is internally consistent and stable.     

User involvement during information systems development: a comparison of analyst and user perceptions of system acceptance

A sample of 150 users and analysts from multiple organizations and many system projects provided field questionnaire data to test hypotheses about the differences in their perceptions of user involvement (UI) and system acceptance. The Franz and Robey [Franz, C.R., Robey, D., 1986. Organizational context, user involvement, and the usefulness of information systems. Decision Sciences, Vol. 17, No. 3, pp. 329–356.] instrument was used for data collection and factor analyzed, resulting in more focused and specific measures. Analysis of the data showed that users and analysts did not agree on the user's involvement nor did they agree on their perceptions of the acceptability of the system to the user. Relationships of self-ratings of UI with system usage and system acceptance by the user demonstrated high correlations, which were attributed to the narrow focus of the UI and system acceptance measures rather than the original more global measure. Analyst perceptions of UI showed no correlation with a user's perceptions of system acceptance.     

A SIMULATION GAME FOR TEACHING SERVICE‐ORIENTED SUPPLY CHAIN MANAGEMENT: DOES INFORMATION SHARING HELP MANAGERS WITH SERVICE CAPACITY DECISIONS?*

For decades, the Beer Game has taught complex principles of supply chain management in a finished good inventory supply chain. However, services typically cannot hold inventory and can only manage backlogs through capacity adjustments. We propose a simulation game designed to teach service‐oriented supply chain management principles and to test whether managers use them effectively. For example, using a sample of typical student results, we determine that student managers can effectively use end‐user demand information to reduce backlog and capacity adjustment costs. The game can also demonstrate the impact of demand variability and reduced capacity adjustment time and lead times.     

Political Exhortation, Patient Expectation and Professional Execution: Perspectives on the Consumerization of Health Care

A central theme underpinning the reform of public-sector services in western economies since the 1980s has been the emphasis on reorienting service provision around the user. Public-sector organizations have been forced to reappraise the design of the service delivery process, in particular the service encounter, to take account of the resultant changes in service users' expectations. Such focus on the service user has fundamental implications for public-sector professionals, specifically challenging the dominance of service professionals in the design and delivery of services. Based on a survey of patient attitudes towards service provision in the National Health Service in Scotland (NHSiS) and in-depth interviews with senior hospital clinicians and managers, the paper critically examines the reaction of both patients and professionals to politically driven initiatives to reorientate the delivery of health-care services. Specifically the paper explores the emerging perspectives of both patients and professionals towards the consumerization of health care and the changing nature of the relationship between patients and professionals within a publicly funded health-care system.     

A Staged Service Innovation Model*

Drawing from the new product development (NPD) literature, service quality literature (SERVQUAL), and empirically grounded research with 53 service innovation decision makers, we develop a staged service innovation model (SIM) for decision makers. We tested the model using empirical data from 329 firms across five industries. The empirical results show that integrating prelaunch service quality training into new service development process leads to successful service innovation. The model developed in this article can be used as a decision support tool and diagnostic model for assessing service innovation ideas, evaluating performance of ongoing service innovations, allocating resources, and improving success rate of service innovations. Decision makers can use the measures developed in this study as a checklist to identify their strengths in delivering service quality to their own customers as well as areas of improvement. This article extends service innovation research by combining NPD and service quality development into a single study and opens the door to further work that could help improve the success rate of service innovations. The model can serve as a base model for future research extensions in service innovation research. A major takeaway for the academic reader is that the SIM demonstrates the value of using the SERVQUAL literature to understand how best to provide excellent quality that results in more fully satisfied customers and, ultimately, improved service performance.     

一个园区网上的用户信息管理系统

作为一个连入Internet的大型园区网络AMMSNet,用户通过MSProxy2.0作为代理服务器对Internet进行访问,以MSExchange5.5提供邮件服务。在这种模式下,如何对用户的基本信息进行维护管理,对用户的访问实施监控并对其访问信息进行长期保存,完成用户数据流量的统计,实现用户管理的自动化,一直是Internet服务商和企业级网络管理员需要解决的首要问题。本文以某大型园区网络为例,介绍了利用C Builder与SQLServer数据库开发的Internet用户信息管理系统,以实现对用户访问Internet的一系列自动化管理。     

Using the End‐User Computing Satisfaction (EUCS) Instrument to Measure Satisfaction with a Web Site

The purpose of this study is to revise and revalidate the End‐User Computing Satisfaction (EUCS) instrument to measure satisfaction with a Web site from a usability perspective. This study is especially important given the increased significance of the Web and the uniqueness of the Web as a computing environment. A total of 176 students participated in a lab simulation that involved a usability evaluation of the Lands' End Web site ( http://www.landsend.com ). Students were asked to complete a set of tasks, record their answers, and then complete the EUCS instrument. Confirmatory factor analysis and invariance analyses were conducted to test the reliability, validity, and generalizability of the revised EUCS. The results show that the EUCS is a valid and robust instrument in the Web environment but that one of the subfactors, timeliness, will need further refinement in the future. Usability practitioners can use the EUCS to measure end‐user satisfaction with a Web site and use the feedback for improving Web‐site design. We describe a case study of an actual usability application that utilized the revised EUCS effectively to support the design of building supply Web sites involving two types of end users, homeowners and contractors. We also propose a typology that researchers can use as a starting point to judge when it is necessary to revalidate an instrument like the EUCS. Finally, we discuss the limitations of our study and present avenues for future research.     

An Entropy Measure of Flow Dominance for Predicting Operations Performance

The flow of jobs within a system is an important operating characteristic that influences system performance. While the majority of previous studies on manufacturing performance consider product flows only as an implicit parameter of the design, we introduce an explicit measure of flow dominance based on entropy and test its efficacy in predicting the performance of manufacturing systems. In computing entropy flow dominance (EFD), we aggregate information embedded in the routings of all products within a system into a single measure. EFD is designed to indicate on a 0–1 scale the level of flow dominance, where 1 represents a pure flow shop and 0 represents a pure job shop. The result is a simple measure that provides managers a way to explain and predict complex phenomena. Our experimental results indicate that EFD is a statistically significant determinant of manufacturing system performance. Furthermore, the model including EFD as an independent variable accurately predicts manufacturing system performance as measured by job flow time, flow time standard deviation, and work in process. We note that the same results can also apply to service systems, such as the “back‐room” low‐contact type systems, that have similar characteristics as manufacturing systems.     

Test-Retest Reliability of User Information Satisfaction: A Comment on Galletta and Lederer's Paper

Galletta and Lederer [10] concluded that the short-form measure of user information satisfaction (UIS) lacks sufficient test-retest reliability. We question the validity of this conclusion since the method used by the authors for assessing test-retest reliability is not grounded in classical reliability theory. Analysis of their data suggests that the UIS instrument has adequate test-retest reliability.     

Technological innovation and operational effectiveness: their role in achieving performance improvements

The purpose of this article is to examine the role of the alignment between technological innovation effectiveness and operational effectiveness after the implementation of enterprise information systems, and the impact of this alignment on the improvement in operational performance. Confirmatory factor analysis was used to examine structural relationships between the set of observed variables and the set of continuous latent variables. The findings from this research suggest that the dimensions stemming from technological innovation effectiveness such as system quality, information quality, service quality, user satisfaction and the performance objectives stemming from operational effectiveness such as cost, quality, reliability, flexibility and speed are important and significantly well-correlated factors. These factors promote the alignment between technological innovation effectiveness and operational effectiveness and should be the focus for managers in achieving effective implementation of technological innovations. In addition, there is a significant and direct influence of this alignment on the improvement of operational performance. The principal limitation of this study is that the findings are based on investigation of small sample size.