Graphical methods for ranking data

Graphical methods are presented for the analysis of ranking data collected from g groups of rankers. The data provided by a single individual consist of the ranks of r objects. The sample space is the space of all permutations and has cardinality r! In order to reduce the dimensionality of the data and to study the interrelationships among rankers and items, a two-stage approach is proposed. First, transformations motivated by various metrics on permutations are defined. In particular, the Kendall metric gives rise to pairwise comparisons. Then, the transformed data are analyzed using results in connection with the generalized singular-value decomposition of a matrix. The methods are illustrated on two examples.     

A New Conjoint Analysis Procedure with Application to Marketing Research

Abstract

In this article we propose some extensions and applications of the nonparametric combination of dependent rankings (see Pesarin, F., Lago, A. (2000). Nonparametric combination of department rankings with applications to the quality assessment of industrial products. Metron LVIII (1–2):39–52.) This methodology is applied to Conjoint Analysis in order to aggregate (ex ante) preferences from a group of individuals. Furthermore, a new global association test (GAT) is introduced in order to test for the association of the global ranking with all attributes of interest. The GAT procedure allows the experimenter to have clear indications on significant attributes by considering the intensity of the optimal weights given by the procedure itself. This may help the experimenter in interpreting the usual analysis involving the normal plot for detecting active effects.     

New imperfect rankings models for ranked set sampling

Ranked set sampling is a sampling approach that leads to improved statistical inference in situations where the units to be sampled can be ranked relative to each other prior to formal measurement. This ranking may be done either by subjective judgment or according to an auxiliary variable, and it need not be completely accurate. In fact, results in the literature have shown that no matter how poor the quality of the ranking, procedures based on ranked set sampling tend to be at least as efficient as procedures based on simple random sampling. However, efforts to quantify the gains in efficiency for ranked set sampling procedures have been hampered by a shortage of available models for imperfect rankings. In this paper, we introduce a new class of models for imperfect rankings, and we provide a rigorous proof that essentially any reasonable model for imperfect rankings is a limit of models in this class. We then describe a specific, easily applied method for selecting an appropriate imperfect rankings model from the class.     

Estimation of Population Variance from Multi-ranker Ranked Set Sampling Designs

In this article, we develop an estimator for a population variance based on a multi-ranker ranked set sampling design. In a multi-ranker design, the units are ranked by more than one ranker allowing ties whenever the confidence level of the rankers is low. The ranking information of all rankers is then combined in a meaningful way to create a single measure. This measure is used to construct the sampling design and a new estimator for the population variance. The article investigates the bias and relative efficiency of the proposed variance estimator. It is shown that the new estimator performs as good as or better than its competitors in the literature.     

Are Overall Journal Rankings a Good Mapping for Article Quality in Specialty Fields?

Overall journal rankings, which are generated with sample articles in different research fields, are commonly used to measure the research productivity of academic economists. In this article, we investigate a growing concern in the profession that the use of the overall journal rankings to evaluate scholars’ relative research productivity may exhibit a downward bias toward researchers in some specialty fields if their respective field journals are under-ranked in the overall journals rankings. To address this concern, we constructed new journal rankings based on the intellectual influence of research in 8 specialty fields using a sample consisting of 26,401 articles published across 60 economics journals from 1998 to 2007. We made various comparisons between the newly constructed journal rankings in specialty fields and the traditional overall journal ranking. Our results show that the overall journal ranking provides a considerably good mapping for the article quality in specialty fields. Supplementary materials for this article are available online.     

Large Differences between Partial Correlation Coefficients

A plot of each ranking of N objects in N-dimensional space is shown to provide geometric interpretations of Kendall's tau and Spearman's rho and also of the relationship of rho to a sum of inversion weights. The computation of rho from a sum of inversion weights is shown to allow sequential calculation of rho.     

Concordance of Complete or Right-Censored Rankings Based on Spearman's Footrule

A right-censored ranking is what results when a judge ranks only the “top K” of M objects. Complete uncensored rankings constitute a special case. We present two measures of concordance among the rankings of N ≥ 2 such judges, both based on Spearman's footrule. One measure is unweighted, while the other gives greatest weight to the first rank, less to the second, and so on. We consider methods for calculating or estimating the P-values of the corresponding tests of the hypothesis of random ranking.     

A note on ranked-set sampling using a covariate

Ranked-set sampling (RSS) and judgment post-stratification (JPS) use ranking information to obtain more efficient inference than is possible using simple random sampling. Both methods were developed with subjective, judgment-based rankings in mind, but the idea of ranking using a covariate has received a lot of attention. We provide evidence here that when rankings are done using a covariate, the standard RSS and JPS mean estimators no longer make efficient use of the available information. We first show that when rankings are done using a covariate, the standard nonparametric mean estimators in JPS and unbalanced RSS are inadmissible under squared error loss. We then show that when rankings are done using a covariate, nonparametric regression techniques yield mean estimators that tend to be significantly more efficient than the standard RSS and JPS mean estimators. We conclude that the standard estimators are best reserved for settings where only subjective, judgment-based rankings are available.     

An empirical Bayes' procedure for ranking players in Ryder Cup golf

We describe a model to obtain strengths and rankings of players appearing in golf's Ryder Cup. Obtaining rankings is complicated because of two reasons. First, competitors do not compete on an equal number of occasions, with some competitors appearing too infrequently for their ranking to be estimated with any degree of certainty, and second, different competitors experience different levels of volatility in results. Our approach is to assume the competitor strengths are drawn from some common distribution. For small numbers of competitors, as is the case here, we fit the model using Monte-Carlo integration. Results suggest there is very little difference between the top performing players, though Scotland's Colin Montgomerie is estimated as the strongest Ryder Cup player.     

A Bayesian multiple comparison procedure for ranking the means of normally distributed data

We present a Bayesian decision-theoretic model for producing, via all pair-wise comparisons, a set of possible rankings for a given number of normal means. A simulation is performed to compare this constant loss model to popular frequentist methods used to rank normal means, including Tukeys method and the Benjamini–Hochberg procedure. The model will also be compared to a Bayesian model using a linear loss function. Properties to be compared include the probability of containing the true ranking and the expected number of possible rankings produced.     

Olympic rankings based on objective weighting schemes

In this paper, we propose an objective principal components weighting scheme for all-time Winter Olympic gold, silver and bronze medals based solely on the number of medals won. Our results suggest that the approximately equal weights be assigned (or the total medal counts be used regardless of color) if all of the three medal types are retained for ranking purposes. When the proposed methodology is tested against five alternative weighting schemes that have been suggested in the literature using the results for the 2010 Vancouver Winter Olympics, we find a significant agreement in the country rankings. Furthermore, our implementation of principal components variable reduction strategy results in the identification of silver as the best representative medal count for parsimonious Winter Olympics rankings.KEYWORDS: Olympic rankings, principal components analysis, variable reduction strategy, medal counts, objective weighting schemeJEL Classifications: C18, C38, C43     

Efficiency of tests based on weighted rankings for randomized blocks when the alternatives have an a prior1 ordering

Salama and Quade (1981) proposed a family of nonparametric tests based on a method of weighted within-block rankings, for testing the hypothesis of no treatment effects against a postulated ranking of the treatments in a complete randomized blocks layout.

These tests and others are compared with respect to asymptotic efficiency.     

The Value of South and Southeast Asian Studies Journal Rankings

Abstract

This article ranks South and Southeast Asian Studies journals along three categories—quality, efficiency, and impact. The ranking summarizes the results of a survey sent to an international sample of South and Southeast Asian Studies scholars. This study is valuable in three ways: as a decision making tool for the South and Southeast Asian Studies academic community, as an evaluation of methodology in journal ranking studies, and as a discussion of the value of journal rankings studies to assessment and evaluation in higher education generally.     

大学综合排名竞争策略研究——基于网大（www．netbig．com）大学排行榜的实证分析

关于提升大学综舍排名的竞争策略，可以运用累积Logit模型来研究各评价指标排名对大学综合排名的影响效应，通过提高那些对综合排名影响效应较大的指标排名，能更有效地促进综合排名的提高，从而确定有效的排名竞争策略。利用网大（www．netbig．com）“中国大学排行榜”的排名数据进行实证分析，得出排名越靠前的大学通过提升“声誉”和“学术成果”名次能最有效提升综合排名。而排名越靠后的大学则通过提升“学术资源”和“物资资源”名次能最有效提升综合排名等结论。     

Conditionally Efficient Estimation of Long-Run Relationships Using Mixed-Frequency Time Series

I analyze efficient estimation of a cointegrating vector when the regressand and regressor are observed at different frequencies. Previous authors have examined the effects of specific temporal aggregation or sampling schemes, finding conventionally efficient techniques to be efficient only when both the regressand and the regressors are average sampled. Using an alternative method for analyzing aggregation under more general weighting schemes, I derive an efficiency bound that is conditional on the type of aggregation used on the low-frequency series and differs from the unconditional bound defined by the full-information high-frequency data-generating process, which is infeasible due to aggregation of at least one series. I modify a conventional estimator, canonical cointegrating regression (CCR), to accommodate cases in which the aggregation weights are known. The correlation structure may be utilized to offset the potential information loss from aggregation, resulting in a conditionally efficient estimator. In the case of unknown weights, the correlation structure of the error term generally confounds identification of conditionally efficient weights. Efficiency is illustrated using a simulation study and an application to estimating a gasoline demand equation.     

The Clustering of Categorical Data: A Comparison of a Model-based and a Distance-based Approach

For clustering multivariate categorical data, a latent class model-based approach (LCC) with local independence is compared with a distance-based approach, namely partitioning around medoids (PAM). A comprehensive simulation study was evaluated by both a model-based as well as a distance-based criterion. LCC was better according to the model-based criterion and PAM was sometimes better according to the distance-based criterion. However, LCC had an overall good and sometimes better distance-based performance as PAM, although this was not the case in a real data set on tribal art items.     

A Non-parametric Method for Defining a Global Preference Ranking of Industrial Products

Although experimentation is a crucial stage in the process of research and development of industrial products, no satisfactory procedure is available to deal with the common but rather important industrial problem of defining a preference ranking among all the studied product prototypes on the basis of performances. In this paper we propose a two-stage non-parametric procedure in which we firstly perform a set of C-sample testing procedures, followed by multiple comparisons, in this way evaluating a set of partial preference rankings, and secondly synthesise the partial rankings by combining them into a global ranking that provides a general product preference rule. The proposed method is particularly useful in the context of industrial experimentation and offers several advantages such as effectiveness, high flexibility and practical adherence to real problems where preference ranking is a natural goal.     

Neurocognitive Effects in Welders Exposed to Aluminium: An Application of the NPC Test and NPC Ranking Methods

The aim of this study is to investigate the effects of exposure to aluminium on the cognitive sphere in a group of welders. Given the intrinsic complexity of the neurocognitive sphere, three different psychomotor variables were used for its investigation allowing the resulting problem to be naturally modelized into a multivariate framework and solved by a nonparametric combination of permutation tests (NPC test). In order to make the treated and control groups comparable, we also stratified the samples on the basis of a risk index that estimates the combined action of biological age and exposure time. Despite the fact that studies in the literature have reached conflicting results, our study highlights a significant drop in attention and memory performances in individuals exposed to aluminium. Finally, we identify a global criterion that summarises the information on the neurocognitive state by applying the nonparametric combination of dependent rankings method (NPC ranking).     

Clustering ranking data in market segmentation: a case study on the Italian McDonald's customers’ preferences†

Cluster analysis is often used for market segmentation. When the inputs in the clustering algorithm are ranking data, the intersubject (dis)similarities must be measured by matching-type measures, able to take account of the ordinal nature of the data. Among them, we used a Weighted Spearman's rho, suitably transformed into a (dis)similarity measure, in order to emphasize the concordance on the top ranks. This allows creating clusters grouping customers that place the same items (products, services, etc.) higher in their rankings. Also the statistical instruments used to interpret the clusters must be conceived to deal with ordinal data. The median and other location measures are appropriate but not always able to clearly differentiate groups. The so-called bipolar mean, with its related variability measure, may reveal some additional features. A case study on real data from a survey carried out in the Italian McDonald's restaurants is presented.     

Bayesian subset selection approach to ranking normal means

In this, article we consider a Bayesian approach to the problem of ranking the means of normal distributed populations, which is a common problem in the biological sciences. We use a decision-theoretic approach with a straightforward loss function to determine a set of candidate rankings. This loss function allows the researcher to balance the risk of not including the correct ranking with the risk of increasing the number of rankings selected. We apply our new procedure to an example regarding the effect of zinc on the diversity of diatom species.