Some Cautions on the Measurement of User Information Satisfaction*

User information satisfaction (UIS) is important because of its potential effects on MIS department goals, quality of user work life, and extent of voluntary usage of systems. Reliable measurement of UIS is important for providing evaluative information for both researchers and practitioners. This study used 92 managers and executives as subjects to compare the test/retest reliability of a widely used, 13-scale UIS instrument together with four summary questions under experimental and control conditions. The summary questions behaved more reliably than the detailed questions for all groups, perhaps because of problems with scale units and origins and with item heterogeneity. This suggests that researchers need more reliable measures of UIS and that practitioners need to exercise caution when collecting and interpreting UIS scores.     

Test-Retest Reliability of the End-User Computing Satisfaction Instrument

The development of reliable and valid measures of user-information satisfaction is becoming increasingly important to both practitioners and researchers. Measures of reliability include internal consistency and stability. Recent research has raised concerns about the stability of user information satisfaction measures. Using the test-retest correlation method, this article assesses short- and long-range stability of a 12-item scale for measuring end-user computing satisfaction. It examines the stability of individual items and subscales as well as the 12-item instrument. The results suggest that the instrument is internally consistent and stable.     

A Congruence Construct of User Involvement

The MIS literature has devoted considerable attention to the relationship between user involvement and MIS success; unfortunately, this research has produced conflicting results. Recent research on a discrepancy model of user involvement has provided a framework for reconciling studies showing positive, negative, or no impact of involvement on user satisfaction. In this discrepancy model, individual differences between perceived and desired levels of involvement define three conditional states or frames of reference that govern the relationship between involvement and end-user satisfaction. The discrepancy model indicates that studies of user involvement need to control for individual differences by dividing respondents into three groups corresponding to these conditional states. Building upon the discrepancy concept, this paper presents a congruence construct of user involvement (i.e., a measure of involvement “relative” to an individual's desire to get involved) as an alternative way of modeling this contingency relationship. The reliability and validity of the involvement congruence construct are assessed. Perceived and congruence constructs of involvement are compared as predictors of end-user computing satisfaction. The results suggest that involvement congruence is a better predictor than perceived involvement and may offer theoretical as well as empirical advantages over the use of component measures. Furthermore, the congruence method of modeling the discrepancy effect has research design advantages (i.e., it does not require dividing respondents by frame of reference).     

On the Repeated Test-Retest Reliability of the End-User Computing Satisfaction Instrument: A Comment*

This comment extends the test-retest reliability of the end-user computing satisfaction (EUCS) instrument by Torkzadeh and Doll [10]. Whereas Torkzadeh and Doll [10] reportedstability for two hour and two week EUCS test-retest reliability, we investigate the test-retest reliability of the EUCS instrument at two points in time, separated by a two yearinterval. We assess the end user computing satisfaction of personal computer, as well as mainframe, administrative end users in a large public organization. The results of the repeated test-retest using differing application platforms add further support for the reliability of the EUCS measure and highlight some areas of concern for managers of information systems.     

An Investigation of Equity as a Determinant of User Information Satisfaction

This paper reviews equity theory and proposes that perceived inequity in the allocation of management information systems (MIS) resources is likely to influence overall user information satisfaction (UIS). Therefore, in addition to currently identified factors such as quality of information products (QIP), user's knowledge and involvement level (KIL), and EDP staff and services (ESS), equity is also likely to be a relevant factor in determining overall UIS. The paper reports on a cross-sectional survey to obtain data for testing the relevance of equity related issues in the MIS context. Responses from 226 users showed a strong correlation (.67, p > .001) between equity and overall UIS. Further, the addition of equity to the previously identified factors that influence overall UIS (i.e., QIP, KIL, and ESS) significantly increased the explained variance (from .48 to .56). The results support the relevance of equity related issues in determining user attitudes. It is recommended that MIS researchers and professionals take into account the likely influence of equity perceptions on user attitudes and behavior, and strive to develop and implement equitable resource allocation mechanisms. Implications of the findings for future research in the areas of MIS, UIS construct, systems implementation, and management of MIS are also discussed.     

Developing Measures for Assessing the Organizational Impact of Information Technology: A Comment on Mahmood and Soon's Paper

In a recent study, Mahmood and Soon [26] identified and operationalized a set of variables that can be potentially impacted by information technology. The lack of factor analysis to verify the dimensionality of different variables is a notable omission in the study. This paper describes and employs the limited-information factor analysis approach and the minimum average partial criterion for testing the unidimensionality of different variables in the Mahmood and Soon study. While reconfirming the construct validity of eight variables, our results question the unidimensionality of two original variables. Further testimony is provided in favor of our results by citing previous research studies.     

PERCEIVED USEFULNESS OF INFORMATION: A PSYCHOMETRIC EXAMINATION*

The perceived usefulness of information is an important construct for the design of management information systems. Yet an examination of existing measures of perceived usefulness shows that the instruments developed have not been validated nor has their reliability been verified. In this paper a new instrument for measuring two dimensions of perceived usefulness is developed. The results of an empirical study designed to test the reliability and construct validity of this instrument in a capital-budgeting setting are presented.     

A Note on SERVQUAL Reliability and Validity in Information System Service Quality Measurement

Today's information system function includes a large service component. Recent research has examined the SERVQUAL instrument as a possible measure to assist managers and researchers in evaluating service quality. To further examine the appropriateness of the SERVQUAL measure, a large industry sample serves to verify the anticipated structure of the instrument. In addition, a high correlation with a common measure of user satisfaction indicates that the SERVQUAL metric may indeed represent accurate views of user perception. As such, the SERVQUAL instrument can serve as a useful indicator for information system managers attempting to identify areas of needed service improvement and to researchers seeking a success measure of information system services.     

The Career Advancement Prospects of Managers and Professionals: Are MIS Employees Unique?

The present study examined the career advancement prospects of MIS and non-MIS employees, as well as the relationships of career advancement prospects with job performance evaluations, job satisfaction, career satisfaction, and organizational commitment for MIS and non-MIS professionals and managers. Participants included 134 MIS professionals and managers and 397 non-MIS professionals and managers of a large communications company. The results provided no evidence that MIS employees experience more restricted career advancement prospects than non-MIS employees. In addition, job performance evaluations generally had positive effects on career advancement prospects; career advancement prospects had a number of positive effects on job satisfaction, career satisfaction, and organizational commitment; and job satisfaction and career satisfaction had positive effects on organizational commitment. These findings are related to prior research, suggestions for future research are offered, and implications for the management of MIS employees are identified.     

The Development of a Measure for End-User Computing Support

Various studies have argued and concluded that the most effective method of promotingend-user computing (EUC) in an organization consists of providing extensive end-user support. This paper describes the development of an instrument to measure EUC support A search of the EUC literature was conducted to identify items to include in the instrument. Theinstrument was validated by administering it in the form of a questionnaire to end-users in more man a hundred information centers across the United States. A factor analysis conducted on the data obtained from this survey yielded 12 factors of EUC support. Three factors were subsequently dropped from the instrument. The remaining factors were successfully tested for internal consistency. The instrument was tested for, and found to possess, convergent and discriminant validity. The instrument exhibited nomological validity. The resulting instrument, consisting of 42 items and 9 factors, is presented as a reliable, valid, and useful device for assessing the level of EUC support.     

Does IO Economics Really Make the Case for Investments in IT: Comment on West and Courtney

West and Courtney [18] rely on industrial organization economics in general, and the economics of uncertainty in particular, for insights into the benefits of overcoming information constraints through information systems (IS) and improved information technologies (IT). This Comment argues that the IS researcher who looks to the economics of uncertainty to make a case for improved IT will only find support from a selective reading of a constantly expanding literature whose aims and inferences are easily misunderstood.     

The Information Problems in Organizations: A Research Model for the Value of Information and Information Systems*

Growing recognition of the need for better management information system (MIS) theory and practice centers around problems with identifying the value of information and information systems in organizations. This paper addresses this issue by showing that existing theory in organizational economics and industrial organization directly relates to information problems in organizations (their nature, and their effects in terms of organization structure and performance). The paper categorizes these problems and associated theories into four distinct classes and shows that MIS research and practice can be mapped to information problems of each type. Because the economic theory pertinent to specific information problems includes the nature of the organizational response to information constraints, it also provides insight into the nature of the improvement to be realized if information constraints can be overcome. These improvements can then be interpreted as the value of the associated information systems which suggests using the resultant classification scheme to map between information problems and system value. The paper shows that this methodology provides a more appropriate level of focus than either traditional normative or business value methods, especially because many improvements have structural as well as performance consequences.     

Task-Structure Relationship of Information Systems Development Subunit: A Congruence Perspective*

Several studies have examined the influence of organizational context (e.g., organization size, environment, strategy) on information systems (IS) performance/success. But very few studies have examined IS structure, and those studies have focused on a particular element of IS structure (e.g., decentralization). Thus, the multidimensional structural configuration of IS as an organizational subunit has escaped research attention. Further, the nature of the task to be performed by the IS subunit as a contextual factor has been virtually ignored in the IS research literature. In this paper, we first develop a congruence model for the task-structure relationship in an information systems development (ISD) subunit setting. Then we test this model as a multivariate relationship followed by subordinate bivariate analyses. Analysis of data from 41 hospitals’ ISD subunits reveals that a congruence relationship between task context and the structure of an ISD subunit is present. Our findings also show that an exclusive association exists between uncertainty in the task environment and the decision-making structure. Likewise, the association between equivocality of task content and the control structure of the ISD subunit is also exclusive. Implications of these findings for decision makers and researchers are discussed.     

Information Systems Management Issues: Reporting and Relevance*

Reports of key information systems issues have been published over the last two decades in many journals. Leading IS journals (e.g., MIS Quarterly, Information & Management, among others) have published key IS management issue reports every three or four years over the last 15 years, and will probably continue to provide such reports in the future. Although these reports claim to provide decisional guidance to practitioners, researchers, consultants, etc., the authors in their experience have noted concerns about their usefulness. While not questioning the validity of the methods and analysis conducted in these studies, we address two important questions in this article: the manner of reporting of the key issue results (which might be misleading), and the relevance of the results (are they providing what they intended to?). We hope that our discussion will provide new perspectives in making resource allocation decisions to both readers and authors of key issue articles.     

The Selection of Computer-Aided Software Engineering Tools: A Multi-Criteria Decision Making Approach

The selection of the appropriate computer-aided software engineering (CASE) tools to suit the needs of an organization requires the systematic application of the multi-criteria decision methodology (MCDM). In order to make a case for the use of MCDM, the application of the ELECTRE I method to the selection of CASE tools from a possible set of six alternatives is demonstrated. The demonstration helps in gaining a grasp of the MCDM approach and the ELECTRE I method. There is potential for the application of the MCDM approach in other software engineering decisions, especially in the feasibility analysis of the systems life cycle.     

PERCEIVED INFORMATION STRUCTURE: IMPLICATIONS FOR DECISION SUPPORT SYSTEM DESIGN

Top-level decision making in business organizations is characterized by high degrees of uncertainty, incomplete information, and conflicting objectives. To support top-level decision making effectively, decision support systems (DSSs) have been proposed. Information supplied by a DSS must be selective in that not all possible information sets may be feasibly or economically represented in the data base. This study suggests that discovery of perceptual complexity (dimensionality) of information items, and the subsequent categorization of decision makers having the same perceptions of those information items, is a first step in the ultimate design of an effective DSS. Through the use of multidimensional scaling in a field setting, this study shows the feasibility of creating relatively homogeneous groups of decision makers according to the content and number of dimensions associated with various information items. Further results of the research suggest that information can be tailored to classes of users, which has cost-benefit implications as well as the potential to improve the quality of the resultant decisions.     

A Rejoinder to Forman on AHP,with Emphasis on the Requirements of Composite Ratio Scales*

In an earlier paper [11], the problems of rank reversals and invalid composite priorities in AHP were addressed by modifications to the AHP procedure. That solution was subsequently criticized [5]. In this paper, we rebut these criticisms, and we show how rank reversals in AHP can arise merely from the process of normalizing local priorities.     

The Effectiveness of Graphical Presentation for Information Extraction: A Cumulative Experimental Approach*

Three laboratory experiments were conducted to assess the relative strengths and weaknesses of bar, symbol, and line graphs for performing a variety of elementary information extraction tasks using two dependent variables, time and accuracy. The findings indicate that the degree of support provided by a particular graph format for a particular data extraction task depends on the matching of format and task in terms of their anchoring characteristics. Anchoring, in this context, refers to the phenomenon that specific and diverse parts of a graph are segmented by the reader to act as salient and relevant cues, or anchors, when different classes of information are to be extracted from the graph. A data extraction task has high x-value (y-value) anchoring if the x-axis (y-axis) component is represented in the question as either a given value or an unknown value. Conversely, a task has low x-value (y-value) anchoring if the x-axis (y-axis) component is not represented in the question as either a given value or as an unknown value. Data extraction accuracy was not significantly affected by presentation format. Bars provided the best time performance for data extraction tasks having high anchoring on both axes but were not appropriate for tasks having low anchoring on either the y axis or both the x and y axes. Line graphs tended to be worse in terms of time performance for tasks having high anchoring on both axes although they were as fast or better than other representations for tasks having low anchoring on both axes. Symbol plots appeared to possess anchoring characteristics associated with both bars and line graphs. Symbols (as with bars) tended to produce a time performance superior to that of line graphs for tasks having high anchoring on both axes; and (as with line graphs) symbols allowed faster results than bar graphs for tasks having low anchoring on either the y axis or both the x and y axes.     

Construct Measurement in Information Systems Research: An Illustration in Strategic Systems

This study seeks to highlight construct measurement issues in information systems (IS) research. It describes the normative process of construct measurement and identifies the difficult problems involved in measurement and some ways in which these difficulties may be overcome. An illustrative construct-operationalization study in the area of strategic systems outlines how the normative guidelines may be applied to IS. Some specific recommendations for IS include developing a preliminary model of the construct even if there is little previous measurement research, devoting greater attention to predictive validity because a lack of theories in IS precludes the examination of nomological validity, verifying the assumptions underlying the computation of an overall index, and examining the measurement properties of the index.