A new weighted rank coefficient of concordance

There are many situations where n objects are ranked by b>2 independent sources or observers and in which the interest is focused on agreement on the top rankings. Kendall's coefficient of concordance [10 M. Kendall and B. Smith, The problem of m rankings, Ann. Math. Stat. 10 (1939), pp. 275–287. doi: 10.1214/aoms/1177732186[Crossref] , [Google Scholar]] assigns equal weights to all rankings. In this paper, a new coefficient of concordance is introduced which is more sensitive to agreement on the top rankings. The limiting distribution of the new concordance coefficient under the null hypothesis of no association among the rankings is presented, and a summary of the exact and approximate quantiles for this coefficient is provided. A simulation study is carried out to compare the performance of Kendall's, the top-down and the new concordance coefficients in detecting the agreement on the top rankings. Finally, examples are given for illustration purposes, including a real data set from financial market indices.     

Aspects of two group concordance

A definition of concordance is derived from Rao's concept of a perfect diversity measure in order to identify aspects about which two populations of judges agree. In the case where each judge independently ranks a fixed set of items, the class of concordance measures based on the marginal distributions of the ranks is characterized by bi-affine similarity functions that measure how well pairs of judges tend to agree. This class contains population versions of several familiar indices of concordance, including Kendall's W. Concordance between two populations, refered to as intergroup concordance, is also scaled against its corresponding intragroup measures. Small sample properties of estimators of the ratio of inter-to-intra group concordance are investigated in a Monte Carlo study. An example is given to illustrate components of concordance attributable to subsets of items, and to contrast the proposed methods with previous analyses.     

Detection of concordance for extreme ranks

When analysing ranking data from one or more groups of judges one may wish to allow for the possibility that the judges have paid more attention to the allocation of the extreme ranks, rather than to the intermediate ranks. In some cases they may have only worried about assigning the top ranks (1 and 2, say) while randomly allocating the remaining ones.

In another context, the analystmay wish to only take account of the agreement among judges with respect to extreme ranks (top or bottom, or both),

In such situations an analysis of concordance within, and between groups, if appropriate, should be able to deal with extreme ranks specifically. We propose a data analyticapproach, related to an analysis of diversity,which actyally permits an analysis of concordance for each rank separately.     

An approximated distribution of the gini's rank associatio coefficient

An approximate distribution is proposed for the Gini's rank association coefficient g which is, like Kendall's and Spearman's rank correlation coefficient, a statistic to test independence between two random variables. The purposed distribution can be simply transformed into a Student's T distribution; so, hypothesis testing is made much easier.     

Comparing the methods of measuring multi-rater agreement on an ordinal rating scale: a simulation study with an application to real data

Agreement among raters is an important issue in medicine, as well as in education and psychology. The agreement among two raters on a nominal or ordinal rating scale has been investigated in many articles. The multi-rater case with normally distributed ratings has also been explored at length. However, there is a lack of research on multiple raters using an ordinal rating scale. In this simulation study, several methods were compared with analyze rater agreement. The special case that was focused on was the multi-rater case using a bounded ordinal rating scale. The proposed methods for agreement were compared within different settings. Three main ordinal data simulation settings were used (normal, skewed and shifted data). In addition, the proposed methods were applied to a real data set from dermatology. The simulation results showed that the Kendall's W and mean gamma highly overestimated the agreement in data sets with shifts in data. ICC₄ for bounded data should be avoided in agreement studies with rating scales<5, where this method highly overestimated the simulated agreement. The difference in bias for all methods under study, except the mean gamma and Kendall's W, decreased as the rating scale increased. The bias of ICC₃ was consistent and small for nearly all simulation settings except the low agreement setting in the shifted data set. Researchers should be careful in selecting agreement methods, especially if shifts in ratings between raters exist and may apply more than one method before any conclusions are made.     

Tables of the exact quantiles of the top-down correlation coefficient for n - 3(1)14

Iman and Connver (1985, 1987) have suggested the top-down correlation coefficient as a measure of association when n objects are ranked by two or more independent sources and interest centers primarily on agreement in the top rankings, with disagreements on items at the bottom of the rankings being of little or no importance. The top-down correlation coefficient results from computing the ordinary Pearson correlation coefficient on Savage scores. Quantiles of the exact distribution of the top-down correlation coefficient based on the assumption of independent rankings are provided for n = 3(1)14.     

Nonparametric measures of intraclass correlation

Three nonparametric measures of intraclass correlation based on the notion of concordance are considered. Their unbiased estimators and nonparametric tests based on the estimators are studied and it is shown that an analogue of the Kendall's tau provides small variance estimator and relatively powerful test. Furthermore, the approximate variance of the estimator is given when the correlation is small in the normal model.     

Simple moment-based inferences of generalized concordance correlation

We proposed two simple moment-based procedures, one with (GCCC1) and one without (GCCC2) normality assumptions, to generalize the inference of concordance correlation coefficient for the evaluation of agreement among multiple observers for measurements on a continuous scale. A modified Fisher's Z-transformation was adapted to further improve the inference. We compared the proposed methods with U-statistic-based inference approach. Simulation analysis showed desirable statistical properties of the simplified approach GCCC1, in terms of coverage probabilities and coverage balance, especially for small samples. GCCC2, which is distribution-free, behaved comparably with the U-statistic-based procedure, but had a more intuitive and explicit variance estimator. The utility of these approaches were illustrated using two clinical data examples.     

Testing for concordance between several criteria

The problem of whether the rankings of some objects given by a set of criteria (or judges) show any agreement or are more or less independent is addressed. The most familiar measure for concordance is the Kendall W coefficient. Classical tests for concordance are the Friedman test and the F test. Legendre [Species associations: the Kendall coefficient of concordance revisited. J. Agric. Biol. Environ. Stat. 2005;10(2):226–245] compared via simulation the Friedman test and its permutation version. Unfortunately, the simulation study of Legendre was very limited because it considered neither the copula aspect nor the F test. Kendall W is a rank-based correlation measure, and therefore it is not affected by the marginal distributions of the underlying variables, but only by the copula of the multivariate distribution. In this article, the simulation study of Legendre is deeply extended by considering the copula aspect as well as the F test. It is shown that the Friedman test is too conservative and less powerful than both the F test and the permutation test for concordance which always have a correct size and behave alike. The F test should be preferred because it is computationally much easier. Surprisingly, the power function of the tests is not much affected by the type of copula.     

KENDALL'S TAU AND CONTINGENCY TABLES

Kendall's tau is a coefficient of concordance between two rankings of n objects. Its definition and large sample normal approximation are easily extended to the case where one of the rankings contains ties. In this paper, definition and normal approximation are extended further to the case where both rankings contain ties. The results are applied to give a fully distribution-free test for two-way contingency tables with ordered categories.     

A coefficient of multiple association based on ranks

A generalization of Kendall's tau is formulated for describing the association between a dependent variable and a collection of independent variables. The coefficient may be defined in terms of the proportional reduction in prediction errors obtained by predicting the ordering of pairs of observations on the dependent variable based on orderings of the pairs on the independent variables. The coefficient is formulated both for continuous and discrete variables. Approximate large-sample distributions are considered for both cases. Some of the properties of this coefficient are discussed and compared with those of other multiple measures of association based on ranks.     

A test of concordance based on Gini’s mean difference

A new rank correlation index, which can be used to measure the extent of concordance or discordance between two rankings, is proposed. This index is based on Gini’s mean difference computed on the totals ranks corresponding to each unit and it turns out to be a special case of a more general measure of the agreement of m rankings. The proposed index can be used in a test for the independence of two criteria used to rank the units of a sample, against their concordance/discordance. It can then be regarded as a competitor of other classical methods, such as Kendall’s tau. The exact distribution of the proposed test-statistic under the null hypothesis of independence is studied and its expectation and variance are determined; moreover, the asymptotic distribution of the test-statistic is derived. Finally, the implementation of the proposed test and its performance are discussed. Both the authors contributed equally to this work; however, the actual writing of the paper was as follows: Sects. 2 and 3 are due to C. G. Borroni, Sects. 1 and 4 are due to M. Zenga.     

On inference for Kendall's τ within a longitudinal data setting

Kendall's τ is a non-parametric measure of correlation based on ranks and is used in a wide range of research disciplines. Although methods are available for making inference about Kendall's τ, none has been extended to modeling multiple Kendall's τs arising in longitudinal data analysis. Compounding this problem is the pervasive issue of missing data in such study designs. In this article, we develop a novel approach to provide inference about Kendall's τ within a longitudinal study setting under both complete and missing data. The proposed approach is illustrated with simulated data and applied to an HIV prevention study.     

ESTIMATORS BASED ON KENDALL'S TAU IN MULTIVARIATE COPULA MODELS

The estimation of a real‐valued dependence parameter in a multivariate copula model is considered. Rank‐based procedures are often used in this context to guard against possible misspecification of the marginal distributions. A standard approach consists of maximizing the pseudo‐likelihood. Here, we investigate alternative estimators based on the inversion of two multivariate extensions of Kendall's tau developed by Kendall and Babington Smith, and by Joe. The former, which amounts to the average value of tau over all pairs of variables, is often referred to as the coefficient of agreement. Existing results concerning the finite‐ and large‐sample properties of this coefficient are summarized, and new, parallel findings are provided for the multivariate version of tau due to Joe, along with illustrations. The performance of the estimators resulting from the inversion of these two versions of Kendall's tau is compared in the context of copula models through simulations.     

Rank Correlation — an Alternative Measure

Within the bounds of a general theory of rank correlation two particular measures have been adopted widely: Spearman7apos;s rank correlation coefficient, ρ in which ranks replace variates in Pearson's product-moment correlation calculation; and Kendall's τ in which the disarray of x -ordered data due to a y -ordering is measured by counting the minimum number, s ; of transpositions (interchanges between adjacent ranks) of the y -ordering sufficient to recover the x-ordering. Based on insights from the calculation of Kendall's coefficient, this paper develops a graphical approach which leads to a new rank correlation coefficient akin to that of Spearman. This measure appears to stand outside general theorybut has greater power of discrimination amongst differing reorderings of the data whilst simultaneously being strongly correlated with both ρ and τ. The development is focused on situations where agreement over ordering is more important for top place getters than for those lower down the order as, for example, in subjectively judged Olympic events such as ice skating. The basic properties of the proposed coefficient are identified.     

A WEIGHTED RANK MEASURE OF CORRELATION

Spearman's rank correlation coefficient is not entirely suitable for measuring the correlation between two rankings in some applications because it treats all ranks equally. In 2000, Blest proposed an alternative measure of correlation that gives more importance to higher ranks but has some drawbacks. This paper proposes a weighted rank measure of correlation that weights the distance between two ranks using a linear function of those ranks, giving more importance to higher ranks than lower ones. It analyses its distribution and provides a table of critical values to test whether a given value of the coefficient is significantly different from zero. The paper also summarizes a number of applications for which the new measure is more suitable than Spearman's.     

Concordance of Complete or Right-Censored Rankings Based on Spearman's Footrule

A right-censored ranking is what results when a judge ranks only the “top K” of M objects. Complete uncensored rankings constitute a special case. We present two measures of concordance among the rankings of N ≥ 2 such judges, both based on Spearman's footrule. One measure is unweighted, while the other gives greatest weight to the first rank, less to the second, and so on. We consider methods for calculating or estimating the P-values of the corresponding tests of the hypothesis of random ranking.     

Sample size calculation for an agreement study

It is often necessary to compare two measurement methods in medicine and other experimental sciences. This problem covers a broad range of data. Many authors have explored ways of assessing the agreement of two sets of measurements. However, there has been relatively little attention to the problem of determining sample size for designing an agreement study. In this paper, a method using the interval approach for concordance is proposed to calculate sample size in conducting an agreement study. The philosophy behind this is that the concordance is satisfied when no more than the pre‐specified k discordances are found for a reasonable large sample size n since it is much easier to define a discordance pair. The goal here is to find such a reasonable large sample size n. The sample size calculation is based on two rates: the discordance rate and tolerance probability, which in turn can be used to quantify an agreement study. The proposed approach is demonstrated through a real data set. Copyright © 2009 John Wiley & Sons, Ltd.     

Smoothed ranks for two or multi-sample location problems

Smoothed ranks are proposed for two or multi-sample location problems. The regular ranks in Wilcoxon's two sample test are replaced with smoothed ranks, and the shift parameter is estimated. Asymptotic properties of the smoothed rank estimator are shown and a hypothesis test is proposed. Moreover, the smoothed ranks are applied in the Kruskal–Wallis's r-sample test and the power of the test is computed using regular and smoothed ranks. Examples and Monte Carlo simulations show that the smoothed ranks perform similarly to the traditional rank based estimators under contaminated normal or non-normal populations.