Replication of Measures in Information Systems Research: The Case of IS SERVQUAL*

The Information Systems (IS) discipline should recognize potential pitfalls in replication of research studies. This response to a note appearing in this issue by Van Dyke, Prybutok, and Kappelman (1999) aims to serve two purposes. First, this note will address the problems we have with the Van Dyke et al. (1999) attempted replication of our own previous work. Second, we hope it serves to establish more generally an awareness in the IS field for the potential to create and accept the “illusion of replicability” when this has not really been established.     

Evaluating and Refining Expert Critiquing Systems: A Methodology*

Expert critiquing systems are a rapidly growing class of intelligent decision support systems that apply artificial intelligence techniques to criticize a user's proposed solution to a problem. Critic programs now exist in the medical, engineering, programming, knowledge acquisition, word processing, and other domains. Critic refinement is a nontrivial activity that, even when done well, consumes a sizable fraction of the complete effort to build and deploy a critiquing system. To ease that effort, it is important to adopt a rigorous approach that allows one to reproducibly measure the degree of success of the current critic version and to predict which refinements will improve the critic further. The current article presents one such approach with actual case studies that illustrate its usage and elaborate selected aspects of the refinement process.