A FRAMEWORK FOR THE SYSTEM DESIGN PROCESS1

Much discussion and writing about “systems” is often superficial, glib, and of a non-operational nature. This paper presents a cohesive framework for the process of system design which is felt to be of operational value in the design and management of systems. The framework's central tenet is that all systems are made up of entities and the relationships among the attributes of those entities; and that in designing systems, two processes are used, inclusion and structuring. Inclusion is concerned with what entities to include in a system and structuring with how their attributes are to be related. The system design process is directed or driven by a set (system) of criteria. Systems must be viewed with a time dimension: evolution will and should take place, both in the systems themselves and, more importantly, in the criteria that drive the design and evolution of those systems. In this framework, all problem solvers can usefully regard themselves as system designers. Presentation of the framework is followed by examples, discussion of the criteria set, system design tools, and some implications for teachers and practitioners.     

A proposed methodology for the prediction of mental workload, based on engineering system parameters

In the design of engineering systems, mental workload is one of the most important factors in the allocation of cognitive tasks. Current methods of task allocation have criteria that are defined in only general terms and are thus not very useful in aiding detailed decision-making in system design. Whilst there are many quantitative criteria available to determine the physical space in human-machine interaction, system designers really require an explicit model and specific criteria for the following identification of the mental workload imposed by the system; prediction of both human and system performance; evaluation of the alternatives of system design; and the design of system components. It is argued that the available methods of workload or performance are either too domain-dependent to apply to the design of other systems, or subject-dependent and thus do not reflect the objective workload imposed by the system. The presented research adopts a new approach to cognitive task analysis in dynamic decision-making systems. Based on the characteristics derived from task analysis, a general conceptual model of the prediction of mental workload in system design is proposed. In the new model, workload is represented by a set of system parameters—task arrival rate, task complexity, task uncertainty, and performance requirements—which are considered to be the main sources of workload. In this context, workload becomes an objective demand of engineering systems, independent of any subjective factors. Whether an individual or population is overloaded depends upon their workload threshold with respect to the specified task and environment. It is hoped that this new model, after both laboratory and industrial validation, could be used by system designers to predict the workload imposed on people by systems.     

A Multi-disciplinary Identification of Issues Associated with ‘Contracting’ in Market-oriented Health Service Reforms

Multi-disciplinary perspectives from operational research, management information systems, purposeful activity systems, accounting and finance, transaction-cost economics and organization learning are discussed in relation to `contracting' in the NHS following the recent reforms, applied within the general framework of soft-systems methodology. These are then used to frame questions for collecting information about contracting practices and issues. The data so collected suggest that the issues and activities associated with contracting can be grouped into five interacting categories of: strategy formation, making enabling arrangements, operational management of contracted activities, identifying and relating to stakeholders, and carrying forward organizational learning from experience. Each of the disciplinary perspectives attributes significance to specific forms in these five activities. The possibility of some multi-disciplinary linking of theoretical perspectives is demonstrated. Information and purposeful systems are central to this, being on the one hand created by social processes which define relevant information and corporate alignments of purpose, and on the other hand patterns of activity that can be evaluated in terms of contribution to these purposes, with properties of greater or lesser inhibition of innovation to improve purpose achievement. As a case study of an attempt at inter-disciplinary research, it demonstrates that interdisciplinary linkage can be made, though certain epistemological issues are skated over in the process. On the basis of the case study, some of the richness and insight of the different perspectives is lost in the process. The critical debate about the neutrality of soft-systems methodology is commented on in the light of the study.     

An Integrated Approach to Oversight Assessment for Emerging Technologies

Analysis of oversight systems is often conducted from a single disciplinary perspective and by using a limited set of criteria for evaluation. In this article, we develop an approach that blends risk analysis, social science, public administration, legal, public policy, and ethical perspectives to develop a broad set of criteria for assessing oversight systems. Multiple methods, including historical analysis, expert elicitation, and behavioral consensus, were employed to develop multidisciplinary criteria for evaluating oversight of emerging technologies. Sixty‐six initial criteria were identified from extensive literature reviews and input from our Working Group. Criteria were placed in four categories reflecting the development, attributes, evolution, and outcomes of oversight systems. Expert elicitation, consensus methods, and multidisciplinary review of the literature were used to refine a condensed, operative set of criteria. Twenty‐eight criteria resulted spanning four categories: seven development criteria, 15 attribute criteria, five outcome criteria, and one evolution criterion. These criteria illuminate how oversight systems develop, operate, change, and affect society. We term our approach “integrated oversight assessment” and propose its use as a tool for analyzing relationships among features, outcomes, and tradeoffs of oversight systems. Comparisons among historical case studies of oversight using a consistent set of criteria should result in defensible and evidence‐supported lessons to guide the development of oversight systems for emerging technologies, such as nanotechnology.     

Evaluating qualitative management research: Towards a contingent criteriology

The term qualitative management research embraces an array of non‐statistical research practices. Here it is argued that this diversity is an outcome of competing philosophical assumptions which produce distinctive research perspectives and legitimate the appropriation of different sets of evaluation criteria. Some confusion can arise when evaluation criteria constituted by particular philosophical conventions are universally applied to this heterogeneous management field. In order to avoid such misappropriation, this paper presents a first step towards a contingent criteriology located in a metatheoretical analysis of three modes of qualitative management research which are compared with the positivist mainstream to elaborate different forms of evaluation. It is argued that once armed with criteria that vary accordingly, evaluation can reflexively focus upon the extent to which any management research consistently embraces the particular methodological principles that are sanctioned by its a priori philosophical commitments.     

From the Inside Out: Environmental Agency Views about Communications with the Public

Relatively few studies have examined risk communication with the public from the viewpoint of the staff of institutions attempting such communications. This paper reports results of interviews with managers and staff of the New Jersey Department of Environmental Protection, on their programs' current and ideal communications with the public. Q analysis revealed two orthogonal perspectives on current program communications, the Enthused and the Constrained views. The primary focus of their divergence was on the matter of commitment and support for such communications. The Enthused group felt that they had attitudinal support from program culture and managers; the Constrained group focused on the lack of concrete operational support in terms of time, money, and expertise. These differences between the two groups did not appear to be associated with gender, managerial status, education, communication training, or organizational unit. When ideal program communications were discussed, the focus was on the need for all kinds of commitment and support (i.e., culture, managers, time, money, building expertise through training), as well as on more proactive and responsive communication. Both perspectives agreed that communication with the public is essential to their programs' success, trying new ways to communicate is worthwhile, scientific bases for decisions are not compromised by communication, and communication is not delegated to specialists. While varying ideas were held of the public's capacity and interest in communication, citizens were largely not held responsible for communication problems.     

The Influences of User Characteristics on Performance in a Logistics DSS Application

Decision Support Systems (DSS) are widely used in logistics decision applications, and a large number and variety of systems are commercially available. We investigate the contributions of user characteristics including experiences, data preferences, intuition, and effort to decision performance in a logistics DSS context. The study includes a laboratory experiment in which decision makers with varied experiences used a DSS to make facility network design decisions for problems of varying complexity. Two variants of the DSS are utilized in order to examine the interactions of a DSS decision aid with user characteristics. We find that intuition and effort are associated with decision-making performance. High analytic ability is not related to intuition, however. Education and previous experience are associated with performance. Yet these characteristics are also unrelated to intuition. Decision makers who highly value disaggregated data provided by the DSS tend to perform poorly. Also, the results suggest that the effects of users' experiences and preferences on performance are influenced by an analytical decision aid.     

Systems‐Based Guiding Principles for Risk Modeling,Planning, Assessment,Management, and Communication

This article is grounded on the premise that the complex process of risk assessment, management, and communication, when applied to systems of systems, should be guided by universal systems‐based principles. It is written from the perspective of systems engineering with the hope and expectation that the principles introduced here will be supplemented and complemented by principles from the perspectives of other disciplines. Indeed, there is no claim that the following 10 guiding principles constitute a complete set; rather, the intent is to initiate a discussion on this important subject that will incrementally lead us to a more complete set of guiding principles. The 10 principles are as follows: First Principle: Holism is the common denominator that bridges risk analysis and systems engineering. Second Principle: The process of risk modeling, assessment, management, and communication must be systemic and integrated. Third Principle: Models and state variables are central to quantitative risk analysis. Fourth Principle: Multiple models are required to represent the essence of the multiple perspectives of complex systems of systems. Fifth Principle: Meta‐modeling and subsystems integration must be derived from the intrinsic states of the system of systems. Sixth Principle: Multiple conflicting and competing objectives are inherent in risk management. Seventh Principle: Risk analysis must account for epistemic and aleatory uncertainties. Eighth Principle: Risk analysis must account for risks of low probability with extreme consequences. Ninth Principle: The time frame is central to quantitative risk analysis. Tenth Principle: Risk analysis must be holistic, adaptive, incremental, and sustainable, and it must be supported with appropriate data collection, metrics with which to measure efficacious progress, and criteria on the basis of which to act. The relevance and efficacy of each guiding principle is demonstrated by applying it to the U.S. Federal Aviation Administration complex Next Generation (NextGen) system of systems.     

An investigation of project complexity's influence on team communication using Monte Carlo simulation

Research using empirical methods has established a curvilinear relationship between team communication and performance. We conduct virtual experiments to examine team communication and performance when teams work under varying types and levels of project complexity. Data samples, generated using Monte Carlo simulation, are based on the statistical characteristics of empirical data collected from 60 cross-functional project teams that communicated over multiple media (email, phone, and face-to-face) and were completing projects of varying complexity. Regression analysis indicates that project complexity influences the communication-performance relationship. Optimization shows that the communication frequencies at which teams maximize or minimize their performance are dependent upon media used.     

Development of a two-dimensional scale for evaluating technologies in high-tech companies: An empirical examination

Based on a sample of 463 technology evaluations, this paper demonstrates empirically the conceptual split between “technological attractiveness” – which is outside of the control of the company and “technological competitiveness” – which is within the company's control. The 16 “technological attractiveness” criteria produced by the literature review gave a set of six different factors (62.5% of variance) depicting potential value. The 16 “technological competitiveness” criteria derived of the literature review were summarized with only four questions (58% of variance) depicting accumulated value. As such, this research shows that managers assess technologies on the basis of a limited set of criteria. These results have practical implications as they enable us to target technology audits to a more workable set of questions at the operational level.     

中国A股市场价值成长效应的边际条件随机占优分析

利用边际条件随机占优(MCSD)方法对中国A股市场进行了价值成长效应的检验,选取2000～2007年申银万国全市场A股股价指数日收益率作为市场核心组合的基准,分别选取市盈率和市净率指数作为价值/成长型股票的划分指标,将核心组合划分为互斥的子组合.运用MCSD两两比较子组合,发现依据E/P指标划分的价值型股票占优于成长型股票,市场组合存在显著的价值成长效应,且该效应仅在下降市场中显著;而依据B/P指标划分的成长型股票占优于价值型股票,且仅在2005年之后的上升市场中显著.研究结果表明,上市公司财务特征可以部分地作为投资者考察股票收益的一个思路,在不同的市场条件下,投资者应根据不同的财务指标进行投资判断.     

Efficiency improvement from multiple perspectives: An application to Japanese banking industry

The current study focuses on efficiency improvement for banking systems from multiple perspectives, which have different definitions of input/output about various attributes of a banking system. In this research we utilize data envelopment analysis (DEA) and Nash bargaining game (NBG) theory to improve inefficient banks in order to: (1) Make the inefficient bank be the state of Pareto Optimality for multiple perspectives, which can avoid discontentment of some perspectives. (2) Improve a bank by changing its attributes and provide various improving schemes for decision makers. A numerical case study of Japanese banks is also given to show the results of equilibrium solution from multiple perspectives.     

Eliciting and Combining Decision Criteria Using a Limited Palette of Utility Functions and Uncertainty Distributions: Illustrated by Application to Pest Risk Analysis

Utility functions in the form of tables or matrices have often been used to combine discretely rated decision‐making criteria. Matrix elements are usually specified individually, so no one rule or principle can be easily stated for the utility function as a whole. A series of five matrices are presented that aggregate criteria two at a time using simple rules that express a varying degree of constraint of the lower rating over the higher. A further nine possible matrices were obtained by using a different rule either side of the main axis of the matrix to describe situations where the criteria have a differential influence on the outcome. Uncertainties in the criteria are represented by three alternative frequency distributions from which the assessors select the most appropriate. The output of the utility function is a distribution of rating frequencies that is dependent on the distributions of the input criteria. In pest risk analysis (PRA), seven of these utility functions were required to mimic the logic by which assessors for the European and Mediterranean Plant Protection Organization arrive at an overall rating of pest risk. The framework enables the development of PRAs that are consistent and easy to understand, criticize, compare, and change. When tested in workshops, PRA practitioners thought that the approach accorded with both the logic and the level of resolution that they used in the risk assessments.     

Dimensions and design criteria for developing training reaction evaluations

Although there is research on training program evaluation, there is little systematic research on the design, development and use of training reaction evaluations. In order to obtain meaningful information from a program evaluation, evaluation professionals must be rigorous in the design and development of all aspects of an evaluation study, including instrumentation. The purposes of the study were to classify the dimensions of information sought using reaction evaluations and to establish design criteria for developing training reaction instruments. There were three major phases of the study: (1) classification of the dimensions and questionnaire design criteria used in reaction evaluations; (2) validation of the classified dimensions and the questionnaire design criteria by subject matter experts; and (3) assessment of a sample of training reaction instruments currently used in training programs in US corporations. The research findings were: eleven dimensions for reaction evaluation were identified and classified by purpose. Five overall design criteria, each consisting of several sub-criteria, were judged important in the design of reactionnaires. These include: introduction and directions; question format; question construction; questionnaire layout; and data analysis. It was concluded that a well-designed training reaction instrument integrates the proper application of design criteria with appropriate reaction dimensions. Most training reaction instruments used by US corporations consisted of questions representing only a few dimensions. The instruments varied in form and length. Few of them properly utilized the established questionnaire design criteria.     

Identifying design criteria for urban system ‘last-mile’ solutions – a multi-stakeholder perspective

Abstract

This study presents a novel approach to design and evaluate ‘last-mile’ solutions – encompassing the social and economic perspectives of key stakeholders. While urban system initiatives have been implemented in practice, theoretical gaps remain at the operational design level. A theoretical framework is developed, based on design criteria identified from a critical synthesis of supply chain and operations management literature, and ‘operationalised’ using an in-depth case study demonstrating implementation of a Consumer Choice Portal-Package Consolidation Centre solution, within a densely populated urban geography. Findings suggest that there is a need to re-define the role of institutional actors beyond that of the traditional governance task, to one of being able to facilitate performance outcomes. Similarly, industrial efficiency dimensions need to be reorientated to include consumer participation, social considerations and multi-stakeholder service outcomes. Finally, implications for operations theory and practising managers in city logistics are highlighted, with suggested directions for future research.     

Monitoring the improvement of an overall industrial performance based on a Choquet integral aggregation

The design and use of performance measurement systems (PMSs) have received considerable attention in recent years. Indeed, industrial performances are now defined in terms of numerous criteria to be synthesized for overall improvement purposes. The analysis of the literature leads to the conclusion that most of the proposed approaches deal with a qualitative approach of this multi-criteria issue. But only a few quantitative models for PMSs have been proposed in order to better monitor the continuous improvement cycle. Among them, the one proposed by the authors, based on a Choquet integral aggregation operator, allows to express an overall performance according to subordination and transverse interactions between the criteria involved. But, as this model is nonlinear, it is useful to define pieces of information aimed at aiding the manager to improve the performance situation. Thus, this article is a contribution to the managers’ requirements for optimizing the improvement of the overall performance versus the allocated resources. In this view, indexes of efficiency and predictive improvement are proposed. The approach is applied to a case study submitted by a company manufacturing kitchen and bathroom furniture which wants to upgrade the monitoring of its “environment and quality improvement plan”.     

Design of a strategic planning management information system

Information systems that are to support strategic planning must be developed on bases that reflect the unique nature of strategic organizational decisions. This paper presents a systems design methodology which implements five basic criteria for the design of a strategic planning Management Information System (MIS). The methodology uses discriminant analyses of the perceived importance to managers of various categories of strategic information to delineate the most significant information requirements of various managerial groups. These results are augmented with similar analyses based on the personal characteristics of the potential system users in order to implement a design criterion related to the integrated nature of a planning MIS and a ‘teaching’ objective which is posited to be valid and feasible for such a system.     

The design of computer based planning and modeling systems

Today there are nearly 2000 corporations in North America and Europe either using, developing, or experimenting with some form of corporate planning model. With the emergence of this new and rather substantial interest in the methodology of corporate planning modeling, there appears to be a definite need for a conceptual framework which can be used to design and implement computer based planning and modeling systems.In this paper the authors describe a collection of elements which they believe to be of critical importance in designing a corporate planning model. Their objective is to develop a set of criteria for not only designing a planning and modeling system, but a set of criteria which can also be used to facilitate the evaluation and comparison of alternative planning and modeling systems.There are over 50 planning and modeling software packages on the market today. These include systems such as BUDPLAN, COMOS and SIMPLAN. This paper attempts to provide the reader with a convenient checklist of possible features to consider in either designing one's own system or selecting an appropriate software package.