A multicriteria decision analysis model for faculty evaluation

In the context of increasing demands for social and financial accountability of universities, the required implementation of transparent faculty evaluation systems constitutes a challenge and an opportunity for universities strategically aligning the activity of academic staff with the university goals. However, despite growing interest in the performance appraisal of faculty, only a few reported studies propose models that cover the full range of academic activities and the models in use are typically based on ad hoc scoring systems that lack theoretical soundness. This article approaches faculty evaluation from an innovative comprehensive perspective. Based on the concepts and methods of multiple criteria value measurement, it proposes a new faculty evaluation model that addresses the whole range of academic activities and can be applied within and across distinct scientific areas, while respecting their specificities. Constructed through a socio-technical process, the model was designed for and adopted by the Instituto Superior Técnico, the engineering school of the Technical University of Lisbon. The model has a two-level hierarchical additive structure, with top-level evaluation areas specified by second-level evaluation criteria. A bottom non-additive third level accounts for the quantitative and qualitative dimensions of academic activity related to each evaluation criterion. The model allows (a) the comparison of the performance of academic staff with performance targets reflecting the strategic policy concerns of university management; (b) the definition of the multicriteria value profile of each faculty member at the top level of the evaluation areas; (c) the computation of an overall value score for each faculty member, through an optimisation procedure that makes use of a flexible system of weights and (d) the assignment of faculty members to rating categories.     

Prioritization of bridges and tunnels in earthquake risk mitigation using multicriteria decision analysis: Application to Lisbon

This paper presents the development of a multicriteria value model enabling the prioritization of bridges and tunnels according to their structural vulnerability and strategic importance for the formulation and implementation of civil protection policies, both for retrofitting and emergency management, in face of seismic events. An interactive structuring process was developed with a group of key-players to carefully define the evaluation criteria and the MACBETH approach was extensively used (i) to facilitate the assessment from the group of the judgmental information necessary to build value functions and (ii) to establish relative weights for the criteria. The model was subsequently explored to prioritize the bridges and tunnels of a zone in Lisbon with high seismic hazard.     

A comparison of simplified value function approaches for treating uncertainty in multi-criteria decision analysis

Uncertainty is present in many decisions where an action's consequences are unknown because they depend on future events. Multi-attribute utility theory (MAUT) offers an axiomatic basis for choice, but practitioners may prefer to use simpler decision models for transparency, ease of use, or other practical reasons. We identify some ‘simplified’ models currently in use and use a simulation experiment to evaluate their ability to approximate results obtained using MAUT. Our basic message is that avoiding assessment errors in the application of a simplified model is more important than the choice of a particular type of model, but that the best performance over a range of decision problems is from a model using a small number of quantiles.     

Elevator planning with stochastic multicriteria acceptability analysis

Modern elevator systems in high-rise buildings consist of groups of elevators with centralized control. The goal in elevator planning is to configure a suitable elevator group to be built. The elevator group must satisfy specific minimum requirements for a number of standard performance criteria. In addition, it is desirable to optimize the configuration in terms of other criteria related to the performance, economy and service level of the elevator group. Different stakeholders involved in the planning phase emphasize different criteria. Most of the criteria measurements are by nature uncertain. Some criteria can be estimated by using analytical models, while others, especially those related to the service level in different traffic patterns, require simulations.     

Multicriteria decision support for global e-government evaluation

E-government refers to the use of information and communication technologies (ICT) by governments to provide digital services to citizens and businesses over the Internet, at local, national or international level. Benchmarking and assessing e-government is therefore necessary to monitor performance and progress by individual countries and identify areas for improvement. Although such measurements have already been initiated by various organizations, they scarcely highlight the multidimensional nature of the assessment. This paper outlines a multicriteria methodology to evaluate e-government using a system of eight evaluation criteria that are built on four points of view: (1) infrastructures, (2) investments, (3) e-processes, and (4) users’ attitude. The overall evaluation is obtained through an additive value model which is assessed with the involvement of a single decision maker–evaluator and the use of a multicriteria ordinal regression approach. Specifically, the UTA II method is used, whose interactive application process is divided in two phases. Its implementation is supported by MIIDAS (multicriteria interactive intelligent decision aiding system). This research work aims at supporting potential stakeholders to perform a global e-government evaluation, based on their own viewpoints and preferences. Finally, 21 European countries are evaluated and ranked considering the latest criteria data.     

A multicriteria decision making model for reverse logistics using analytical hierarchy process

Product recovery activities such as recycling, refurbishing and direct reuse are becoming integral to manufacturing supply chains. This study presents a multicriteria decision making model for reverse logistics using analytical hierarchy process (AHP). The AHP model evaluates a hierarchy of criteria and subcriteria, including costs and business relations, for critical decisions regarding network design. Using sensitivity analysis with AHP, the work provides insights into the preference ordering among eight alternative network configurations. For instance, the choice of test sites is largely dependent on the potential for cost savings on testing procedures and transportation of scrap, and this decision is not sensitive to the importance of business relations. By contrast, the choice of collection sites is largely determined by the relative importance of business relations considerations vs. cost considerations. As well, the processing location decision favors a third-party reprocessor if there is little need to protect proprietary product knowledge and cost savings is very important. The model is demonstrated using three case studies of real-world applications.     

Group decision making on water resources based on analysis of individual rankings

In problems to do with managing water resources multiple decision makers are involved, each acting in their own right and using different value systems. In the literature on management science, several procedures are proposed in order to establish a collective preference based on the aggregation of different individual preferences. However, the well-known methods that focus on a single winner have some inconveniences that should be addressed. This paper is focused on a group decision making procedure based on the analysis of individual rankings with the aim of choosing an appropriate alternative for a water resources problem. This alternative is found to be the best compromise from the points of view of all actors involved in the decision problem. The structure of the method is set out as is its application to the water resources problem. A comparison with other methods is presented and discussed.     

Two ways to handle dependent uncertainties in multi-criteria decision problems

We consider multi-criteria group decision-making problems, where the decision makers (DMs) want to identify their most preferred alternative(s) based on uncertain or inaccurate criteria measurements. In many real-life problems the uncertainties may be dependent. In this paper, we focus on multicriteria decision-making (MCDM) problems where the criteria and their uncertainties are computed using a stochastic simulation model. The model is based on decision variables and stochastic parameters with given distributions. The simulation model determines for the criteria a joint probability distribution, which quantifies the uncertainties and their dependencies. We present and compare two methods for treating the uncertainty and dependency information within the SMAA-2 multi-criteria decision aid method. The first method applies directly the discrete sample generated by the simulation model. The second method is based on using a multivariate Gaussian distribution. We demonstrate the methods using a decision support model for a retailer operating in the deregulated European electricity market.     

Electric-Utility Resource Planning and Decision-Making: The Importance of Uncertainty

Uncertainty is a critical factor that pervades all aspects of electric-utility planning. Uncertainties about future load growth, about the continued performance of existing supply and demand resources, and about the costs, construction times, and operations of new resources greatly complicate utility resource acquisition. This paper discusses the factors that lead to uncertainty, reviews the methods that utilities use in planning and in acquiring resources, and suggests future research to help deal with these uncertainties. This review is based on assessments of the long-term resource plans prepared by 10 utilities and one Public Service Commission, telephone interviews with staff at these 11 organizations and with staff at three consulting firms, and reviews of many other related publications.     

On the influence of a biased decision support model on subjects’ behavior

Summary  The role of decision tools in managerial decisions can be crucial. Yet, there is very little research seeking to understand and measure the influence of planning models on decision-makers’ actual behavior or performance. We focus on investigating, in an experimental context, the influence of a decision model (time allocation model) on student subjects’ changes of mind and behavior. Of special interest is the effect of the articulated importance of criteria on subjects’ choices. Furthermore, we have also studied how the above mentioned model influences subjects’ actual (measurable) performance. The results are partly conflicting, though interesting: (1) the (biased) decision model did have a significant influence on the subjects’ choices, (2) when the model provided additional freedom to improve criterion values, the subjects’ choices were rather inconsistent with the articulated importance of criteria, (3) the interpretation is not clear, how plans influence actual behavior, and (4) the subjects did not fully recognize the bias of the model (when present). However, our findings very clearly demonstrate that model developers have a great responsibility. Biased models may guide managers to make dramatically erroneous decisions in bona fide. We hope that our findings generate additional research into this important, but under-researched topic.      

Scenario planning,cognition, and strategic investment decisions in a turbulent environment

We explore how scenario planning contributes to cognitive dynamics and strategic investment decisions in a changing environment. Our research is based on an in-depth, longitudinal case study of Shell's ventures in Russia between 1994 and 2016. We show that Shell's scenarios did not foresee some major events that occurred in the turbulent Russian energy market. However, the scenarios envisaged the rising role of gas, the strategic relevance of the gas transportation infrastructure, and the growing interventionism of the Russian government. The scenarios thus helped Shell's managers to adapt their strategic beliefs and ultimately enabled them to notice, assess, and respond successfully to external changes—even though these changes had not been foreseen in the scenarios. Shell has been the first (and, for almost a decade, the only) foreign company to produce and export gas from Russia.     

Supporting the allocation of software development work in distributed teams with multi-criteria decision analysis

In the fast moving software industry, projects have been increasingly developed by distributed teams, which are located in geographically remote offices and collaborate using information communication technologies. In such environments, project distribution presents specific challenges, as work in distributed teams increases project technical complexity, communication lines multiply and stakeholders’ interests may be divergent. Despite the importance and complexity of this type of problem, it seems that there is a lack of reports, in the literature, of systems that could support these decisions. This paper presents a real-world case study, where we developed a multi-criteria model for supporting the distributed team work allocation decision for a major global software company. It was developed with a group of software development project managers, using decision conferencing and multi-attribute value analysis. The model deals not only with software engineering attributes, but also “soft” and strategic issues, like team satisfaction and training opportunities. We also discuss some issues and challenges faced during this modelling process.     

Multiple criteria decision analysis of treatment and land-filling technologies for waste incineration residues

In this paper, we present a real-world case realized in Switzerland on behalf of the federal agency in charge of the waste policy. The incineration of urban wastes generates hazardous residues that need specific treatment and/or land-filling technologies. The variety of local situations and appraisals led to the choice of very different solutions. Moreover, proponents were keen to propose new technologies. The purpose of this study was to provide the background for a national policy that would apply to all future projects.     

决策分析的情景树方法及其应用

本文结合一个实例介绍了贝叶斯决策问题的情景树表示和求解.在情景树中,不需要计算条件概率,仅需要每条路径的联合概率.给出了求解情景树的方法--删除法,并将情景树方法与决策树方法进行了比较分析.     

Illustration of a Method to Incorporate Preference Uncertainty in Benefit–Cost Analysis

Benefit–cost analysis is widely used to evaluate alternative courses of action that are designed to achieve policy objectives. Although many analyses take uncertainty into account, they typically only consider uncertainty about cost estimates and physical states of the world, whereas uncertainty about individual preferences, thus the benefit of policy intervention, is ignored. Here, we propose a strategy to integrate individual uncertainty about preferences into benefit–cost analysis using societal preference intervals, which are ranges of values over which it is unclear whether society as a whole should accept or reject an option. To illustrate the method, we use preferences for implementing a smart grid technology to sustain critical electricity demand during a 24‐hour regional power blackout on a hot summer weekend. Preferences were elicited from a convenience sample of residents in Allegheny County, Pennsylvania. This illustrative example shows that uncertainty in individual preferences, when aggregated to form societal preference intervals, can substantially change society's decision. We conclude with a discussion of where preference uncertainty comes from, how it might be reduced, and why incorporating unresolved preference uncertainty into benefit–cost analyses can be important.     

The Design of Feature‐Limited Demonstration Software: Choosing the Right Features to Include

Today, software supports many important tasks in a variety of industries. In the specialized nature of these environments, a common problem faced by software vendors is to correctly signal the true value of a software product to the end users. For example, telecommunications equipment manufacturers design complex software for important functions like provisioning new users in the network. These software products automate various functions that would otherwise need to be done manually. In order to enable potential customers—telecommunications providers—to evaluate and recognize the full value of the product, equipment vendors often provide a free, feature‐limited version of the product to the customer. As the specific features included in the feature‐limited version influence whether the full product is purchased or not, it is essential that the features included in the feature‐limited version be selected judiciously. While the importance of identifying the best set of features has been well recognized, there has been little research to date that systematically addresses this fundamental business decision. This study fills this gap in the literature by providing an objective approach to the design of demonstration software. We illustrate the benefits of our approach through a case study involving the design of a feature‐limited demo for a wireless telecommunications equipment manufacturer.     

How Much Uncertainty is Too Much and How Do We Know? A Case Example of the Assessment of Ozone Monitor Network Options

Limited time and resources usually characterize environmental decision making at policy organizations such as the U.S. Environmental Protection Agency. In these climates, addressing uncertainty, usually considered a flaw in scientific analyses, is often avoided. However, ignoring uncertainties can result in unpleasant policy surprises. Furthermore, it is important for decisionmakers to know how defensible a chosen policy option is over other options when the uncertainties of the data are considered. The purpose of this article is to suggest an approach that is unique from other approaches in that it considers uncertainty in two specific ways-the uncertainty of stakeholder values within a particular decision context and data uncertainty in the light of the decision-contextual data-values relationship. It is the premise of this article that the interaction between data and stakeholder values is critical to how the decision options are viewed and determines the effect of data uncertainty on the relative acceptability of the decision options, making the understanding of this interaction important to decisionmakers and other stakeholders. This approach utilizes the recently developed decision analysis framework and process, multi-criteria integrated resource assessment (MIRA). This article will specifically address how MIRA can be used to help decisionmakers better understand the importance of uncertainty on the specific (i.e., decision contextual) environmental policy options that they are deliberating.     

Third-party logistics selection problem: A literature review on criteria and methods

This paper presents a literature review on Third-Party Logistics (3PL) selection decision in terms of criteria and methods. Based on the analysis of 67 articles published within 1994–2013 period, this review reveals that 3PL selection is empirical in nature and is related to a region/country, industrial sector, and logistics activities outsourced. In terms of 3PL selection criteria, 11 key criteria are identified; each one is defined by a set of attributes. Cost is the most widely adopted criterion, followed by relationship, services, and quality. In terms of methods for 3PL evaluation, they can be categorized in 5 groups, namely: MCDM techniques, statistical approaches, artificial intelligence, mathematical programming, and hybrid methods.