Variable Weighting via Multidimensional IRT Models in Composite Indicators Construction

One of the most controversial steps in Composite Indicators (CIs) construction is the selection of one (possibly) best weighting technique. In this paper, we introduce a new endogenous weighting methodology developed in an extended Item Response Theory (IRT) framework. As weighting is much more thorough when carried out by accounting for the dimensionality of a dataset rather than by ignoring it, we suggest to assign weights on the basis of the discrimination parameters estimated through a multidimensional two-parameter logistic IRT model. Specifically, the procedure is developed through two consecutive steps. The first applies a hierarchical clustering algorithm to ascertain the number of dimensions measured by the data. The second estimates the discrimination parameters under the multidimensional two-parameter logistic model selected at the first step. The discrimination parameters can then be used to compare and weight the items that refer to the same dimension. Besides, in order to make such discrimination indices comparable across dimensions, the distribution of the latent trait is standardised for each dimension. The potentialities of this novel weighting technique are illustrated through an application to educational data, which refer to a national standardised test developed and collected by the Italian National Institute for the Evaluation of the Education System.     

An Introduction to ‘Benefit of the Doubt’ Composite Indicators

Despite their increasing use, composite indicators remain controversial. The undesirable dependence of countries’ rankings on the preliminary normalization stage, and the disagreement among experts/stakeholders on the specific weighting scheme used to aggregate sub-indicators, are often invoked to undermine the credibility of composite indicators. Data envelopment analysis may be instrumental in overcoming these limitations. One part of its appeal in the composite indicator context stems from its invariance to measurement units, which entails that a normalization stage can be skipped. Secondly, it fills the informational gap in the ‘right’ set of weights by generating flexible ‘benefit of the doubt’-weights for each evaluated country. The ease of interpretation is a third advantage of the specific model that is the main focus of this paper. In sum, the method may help to neutralize some recurring sources of criticism on composite indicators, allowing one to shift the focus to other, and perhaps more essential stages of their construction. An abridged version of this paper was presented at the Workshop on European Indicators and Scoreboards, organised by DG Education and the Joint Research Centre within the auspices of CRELL, in Brussels, October 24–25, 2005.     

Comparing MCDA Aggregation Methods in Constructing Composite Indicators Using the Shannon-Spearman Measure

Composite indicators have been increasingly recognized as a useful tool for performance monitoring, benchmarking comparisons and public communication in a wide range of fields. The usefulness of a composite indicator depends heavily on the underlying data aggregation scheme where multiple criteria decision analysis (MCDA) is commonly used. A problem in this application is the determination of an appropriate MCDA aggregation method. Of the many criteria for comparing MCDA methods, the Shannon-Spearman measure (SSM) is one that compares alternative MCDA aggregation methods in constructing composite indicators based on the information loss concept. This paper assesses the effectiveness of the SSM using Monte Carlo approach-based uncertain analysis and variance-based sensitivity analysis techniques. It is found that most of the variation in the SSM arises from the uncertainty in choosing an aggregation method. Therefore, the SSM can be considered as an effective measure for comparing MCDA aggregation methods in constructing composite indicators. We also use the SSM to evaluate five MCDA aggregation methods in constructing composite indicators and present the findings.     

Assessment of Composite Indicators Using the ANOVA Model Combined with Multivariate Methods

The synthesis of a number of single observed indicators into a unique composite indicator involves various subjective choices related, for instance, to the type of combination (linear, non-linear) and to the aggregation method (simple average, geometric average) used in its construction. Thus, it is clearly important to analyse the variability of a composite indicator according to all possible alternatives before employing it in any decision-making process. Within such a framework, in this paper, we present a new approach based on a combination of explorative and confirmative analyses aiming to investigate the impact of different subjective choices on the variability of composite indicators. This new approach also allows the analysis of the related individual differences among the statistical units and the use of external information on the same units to enhance the interpretation of the final results.     

Local Meanings of Wellbeing and the Construction of Wellbeing Indicators

There have been attempts to develop and employ tools for measuring development that shifted away from economic growth. In Thailand, a number of alternative indicators to measure the progress of development were created and employed. However, the indicators and measurements of development continued to be challenging, partly because of the dynamic changes of socio-economics, and partly due to the inconsistency between aggregated and local, or individual units of measurement. This paper aims to study meanings and develop wellbeing indicators of a community in the Northeast of Thailand. The data was derived from qualitative research conducted during 2013–2014. Focus group discussions with thirty-five formal and informal community leaders were used, and in-depth interviews were applied to collect data from thirty key informants. The results showed that the indicators of community wellbeing are classified into four groups, namely objective community wellbeing, community trust, community security, and community strength. The objective community wellbeing indicators were consistent with those that are used to guide policy and practice in Thailand. However, based on the local meaning of community wellbeing, new dimensions of indicators were found and proposed. This study recommended that these indicators should be developed further, to be employed as a tool to monitor progress, especially by local government.     

Interpreting the Dynamics of Social Indicators: Methodological Issues Related to Absolute,Relative, and Time Differences

In contemporary society, a growing demand for the accessibility of social indicators can be observed. This requires increased attention to their construction to avoid misleading interpretations, which can be the result of inadequate knowledge, or can even be an intentional choice to imply a specific desired outcome. This paper addresses this issue by first summarizing research regarding the perception of numbers, statistical thinking, and numerical literacy. The focus is then narrowed to the comparison of social indicators observed for two units in a time perspective. Three simple and popular measures of dynamics—most frequently used when social change is analyzed and interpreted—are addressed: absolute difference, relative difference, and time distance. In a corresponding experiment, respondents evaluated the direction of change of a certain social indicator in time (i.e., whether the differences increase, decrease, or stagnate) for a hypothetical case where the three measures implied contradictory interpretations. Each experimental group was exposed to one of these measures. The results indicate that interpretations basically followed the specific measures that respondents were exposed to. This effect was particularly strong regarding absolute difference, followed by relative difference, while the effect of exposure to time distance was somewhat weaker. When only data or graphical presentation was given, respondents tended to interpret dynamics according to absolute differences. The results indicate that extreme methodological rigor is needed when presenting social indicators in time, and some guidelines are provided for this purpose.     

On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting,Aggregation, and Robustness

Social Indicators Research - In recent times, composite indicators have gained astounding popularity in a wide variety of research areas. Their adoption by global institutions has further captured...     

A Common Weight Approach to Construct Composite Indicators: The Evaluation of Fourteen Emerging Markets

This study presents an ongoing project, emerging market (EM) evaluation project, of the Taiwan Institute of Economic Research (TIER). The purpose of this project is to construct a composite indicator (CI) named as growth potential index (GPI) for selecting the promising EMs, in which to begin new or expand existing business is attractive to governments, firms, and investors. However, weight determination is one of the most difficult tasks in the construction process of a CI. A new approach inspired by the Z score and rooted in data envelopment analysis (DEA) is proposed to objectively determine the common weights for constructing the GPI without requiring data normalisation beforehand. The same dataset is used to compare the proposed common weight approach with the equal weighting method (currently used by the TIER), the widely used DEA-CI model, and the first common weight DEA-CI model. Spearman’s rank correlation test revealed a high positive correlation between the GPIs obtained by the proposed approach and each considered method. The major findings include: (1) China is the most promising EM; (2) Argentina, China, Malaysia, Poland, and Russia are above-average EMs; (3) India, Indonesia, Saudi Arabia, South Africa, and Thailand are below-average EMs; and (4) of the so-called BRIC countries (Brazil, Russia, India, and China), China is the best EM, and India is the worst EM.     

关于长期稳定低生育水平的理论思考

《中共中央国务院关于加强人口与计划生育工作稳定低生育水平的决定》在实践上和理论上都有划时代的意义 ,也具有国际意义。稳定低生育水平必须加强人口统计学的研究 ,必须科学地掌握稳定低生育水平的数量关系和数量界限。低生育水平会有一定的负效应 ,要用科学的价值观全面衡量低生育水平的利害得失。总的来看利大于弊、利多弊少。低生育水平有利于人口、资源、环境的协调发展和社会的可持续发展。     

Time in the Development of Indicators on Sustainable Wellbeing: A Local Experiment in Developing Alternative Indicators

This paper is a theoretical and methodological study on the topics of distribution of and competition between time spans dedicated to social activities in the development of sustainable well-being indicators. This article seeks to answer the following question: why and how should we take into account social time in the development of alternative indicators? To bring to light the complex relationship between well-being, sustainability and people’s relationship to time, this article draws on an experiment aimed at developing Regional Sustainable Well-being Indicators (Indicateurs de Bien-être Soutenable Territorialisés—IBEST), which took place in the Grenoble urban area. This experiment was based on two methodologies; the first one being a quantitative survey, and the second one being a series of qualitative interviews coupled with a participatory approach. One of the datasheet indicators developed following this methodological crossover is the activity times balancing indicator, which allows us to take into consideration the pressure affecting time spent on social activities (work, leisure, families and civic engagement).     

Indicators of Democratization in Romania

The process of democratization of the Romanian society, which started in December 1989, following the removal of the communist regime, has several peculiarities. The new authorities managed to stay in power until the autumn of 1996, after their left wing party (The National Salvation Front – FSN) won the free multi-party elections of 1990 in which several other parties participated too, and the elections of 1992 (this time under the name of Party of Social Democracy). A new Constitution was adopted and the institutions of the state of the right were established. Although marked by tension and conflicts, especially during 1990–1991, the process of democratization progressed, however, steadily, satisfying the formal indicators of democracy. The victory of the center and right-winged opposition in the elections of November 1996, revealed the alternance of parties in power and by this the consolidation of the democratic institutions in Romania.     

The Stability of Rankings Derived From Composite Indicators: Analysis of the “IL Sole 24 Ore” Quality of Life Report

The calculation of composite indicators and the derivation of respective rankings is a common method used to benchmark countries or regions. However, although the statistical robustness of these rankings is often criticised, they often still spark off heated political debate. Here, we assess the sensitivity of the province ranking published by the Italian economics newspaper “Il Sole 24 Ore”. For this purpose, various weighting, normalising and aggregation schemes were compared. Furthermore, to assess the accuracy of the underlying model, confidence intervals were calculated for the indicator values. The dynamic properties of the rankings were also analysed by evaluating the short- and long-term forecast properties of the rankings.     

Social Indicators and Living Conditions in the Netherlands

The Dutch Social en Cultural Planning Office(SCP) monitors social changes and socialservices, evaluates social policies, givesinformation about expected developments in thefuture and gives recommendations for furtherpolicy. Key issues are the assessment ofeconomic, demographic and social changes andthe influence of these factors on the socialand living conditions of the population. TheSCP uses social and economic indicators toexamine these developments. Besidesresearching specific themes (such as socialexclusion, the consequences of long-termunemployment, the social position of theelderly and use, costs and productivity ofsocial services) the SCP has developed anoverall monitoring tool for the livingconditions: the living conditions index (LCI).Nowadays the index is composed of indicatorswhich reflect conditions in eight areas:housing, health, consumer durables, leisureactivity, sport activity, socialparticipation, mobility and holiday. In thefuture, the SCP will attempt to develop aconceptual model that will link the livingconditions index to other social indices likelivability (housing and level of services),poverty and socioeconomic deprivation.     

Using Ranked Weights and Acceptability Analysis to Construct Composite Indicators: A Case Study of Regional Sustainable Society Index

A variety of published composite indicators, i.e., Energy Trilemma Index and Sustainable Society Index, are commonly aggregated with equal weights. However, this plausible scheme is criticized as eclecticism and ignores the discriminating power of the different indicators. Differing from the traditional methods that assign weights to each indicator for the purpose of aggregation, this paper proposes a new mechanism to construct composite indicators using ranked weights and stochastic multicriteria acceptability analysis. More specifically, this study comprehensively consider all possible preferences among the indicators. Under each preference, we develop a sophisticated mathematical transformation to calculate the least and most favorable scores of each entity, which formulates the lower and upper bounds of the intervals. Then an interval decision matrix, alternatively described as a stochastic decision problem, is formulated to construct the composite indicators. Holistic acceptability indices are generated and regarded as a new composite indicator, which is capable of providing a comprehensive and robust composite indicator with more discriminating power. We apply the proposed method to modify the regional sustainable society index and present the obtained results and comparisons.     

Weighting and Aggregation in Composite Indicator Construction: a Multiplicative Optimization Approach

Composite indicators (CIs) have increasingly been accepted as a useful tool for benchmarking, performance comparisons, policy analysis and public communication in many different fields. Several recent studies show that as a data aggregation technique in CI construction the weighted product (WP) method has some desirable properties. However, a problem in the application of the WP method is the difficulty and subjectivity in determining the weights for sub-indicators. In this paper, we extend the WP method and propose a multiplicative optimization approach to constructing CIs. This approach requires no prior knowledge of the weights for sub-indicators. Instead, the weights are generated by solving a series of multiplicative data envelopment analysis type models that can be transformed into equivalent linear programs. Additional relevant information on the weights, if available, can be incorporated into the proposed models. We apply the proposed approach to the 2005 data of 27 economies in the Asia and the Pacific region in the United Nations’ Human Development Index study and present the results.     

Productivity Indicators for the Rural Poor in Developing Countries

Meeting the Millennium Development Goals will necessarily require a heavy focus on rural areas, where most of the world’s poor are to be found. More specifically, policy will need to raise the productivity of this group, which includes farmers, wage labourers and those suffering from disease and malnutrition. Yet, at present, no index exists which can assess countries on the basis of technological indicators that bear heavily on the productivity of the rural poor. A wide variety of unresolved problems notwithstanding, this paper constitutes the first attempt to construct an index that is designed specifically to assist those who are concerned with rural-specific policies towards meeting the MDGs.