Tests for the multivariate two-sample problem based on empirical probability measures

Nonparametric tests are proposed for the equality of two unknown p-variate distributions. Empirical probability measures are defined from samples from the two distributions and used to construct test statistics as the supremum of the absolute differences between empirical probabilities, the supremum being taken over all possible events. The test statistics are truly multivariate in not requiring the artificial ranking of multivariate observations, and they are distribution-free in the general p-variate case. Asymptotic null distributions are obtained. Powers of the proposed tests and a competitor are examined by Monte Carlo techniques.     

Multivariate Kendall's tau for change‐point detection in copulas

Statistical procedures for the detection of a change in the dependence structure of a series of multivariate observations are studied in this work. The test statistics that are proposed are $L_1$ , $L_2$ , and $L_{\infty }$ distances computed from vectors of differences of Kendall's tau; two multivariate extensions of Kendall's measure of association are used. Since the distributions of these statistics under the null hypothesis of no change depend on the unknown underlying copula of the vectors, a procedure based on the multiplier central limit theorem is used for the computation of p‐values; the method is shown to be valid both asymptotically and for moderate sample sizes. Alternative versions of the tests that take into account possible breakpoints in the marginal distributions are also investigated. Monte Carlo simulations show that the tests are powerful under many scenarios of change‐point. In addition, two estimators of the time of change are proposed and their efficiency is carefully studied. The methodologies are illustrated on simulated series from the Canadian Regional Climate Model. The Canadian Journal of Statistics 41: 65–82; 2013 © 2012 Statistical Society of Canada     

A Family of Goodness‐of‐Fit Tests for Copulas Based on Characteristic Functions

A general class of rank statistics based on the characteristic function is introduced for testing goodness‐of‐fit hypotheses about the copula of a continuous random vector. These statistics are defined as L ₂ weighted functional distances between a nonparametric estimator and a semi‐parametric estimator of the characteristic function associated with a copula. It is shown that these statistics behave asymptotically as degenerate V ‐statistics of order four and that the limit distributions have representations in terms of weighted sums of independent chi‐square variables. The consistency of the tests against general alternatives is established and an asymptotically valid parametric bootstrap is suggested for the computation of the critical values of the tests. The behaviour of the new tests in small and moderate sample sizes is investigated with the help of simulations and compared with a competing test based on the empirical copula. Finally, the methodology is illustrated on a five‐dimensional data set.     

Linear and non-linear canonical correlation analysis:an exploratory tool for the analysis of group-structured data

Confronted with multivariate group-structured data, one is in fact always interested in describing differences between groups. In this paper, canonical correlation analysis (CCA) is used as an exploratory data analysis tool to detect and describe differences between groups of objects. CCA allows for the construction of Gabriel biplots, relating representations of objects and variables in the plane that best represents the distinction of the groups of object points. In the case of non-linear CCA, transformations of the original variables are suggested to achieve a better group separation compared with that obtained by linear CCA. One can detect which (transformed) variables are responsible for this separation. The separation itself might be due to several characteristics of the data (eg. distances between the centres of gravity of the original or transformed groups of object points, or differences in the structure of the original groups). Four case studies give an overview of an exploration of the possibilities offered by linear and non-linear CCA.     

Location‐invariant Multi‐sample U‐tests for Covariance Matrices with Large Dimension

For two or more multivariate distributions with common covariance matrix, test statistics for certain special structures of the common covariance matrix are presented when the dimension of the multivariate vectors may exceed the number of such vectors. The test statistics are constructed as functions of location‐invariant estimators defined as U‐statistics, and the corresponding asymptotic theory is used to derive the limiting distributions of the proposed tests. The properties of the test statistics are established under mild and practical assumptions, and the same are numerically demonstrated using simulation results with small or moderate sample sizes and large dimensions.     

An Optimal Semiparametric Method for Two‐group Classification

In the classical discriminant analysis, when two multivariate normal distributions with equal variance–covariance matrices are assumed for two groups, the classical linear discriminant function is optimal with respect to maximizing the standardized difference between the means of two groups. However, for a typical case‐control study, the distributional assumption for the case group often needs to be relaxed in practice. Komori et al. (Generalized t ‐statistic for two‐group classification. Biometrics 2015, 71: 404–416) proposed the generalized t ‐statistic to obtain a linear discriminant function, which allows for heterogeneity of case group. Their procedure has an optimality property in the class of consideration. We perform a further study of the problem and show that additional improvement is achievable. The approach we propose does not require a parametric distributional assumption on the case group. We further show that the new estimator is efficient, in that no further improvement is possible to construct the linear discriminant function more efficiently. We conduct simulation studies and real data examples to illustrate the finite sample performance and the gain that it produces in comparison with existing methods.     

Linear discriminant analysis for multiple functional data analysis

In multivariate data analysis, Fisher linear discriminant analysis is useful to optimally separate two classes of observations by finding a linear combination of p variables. Functional data analysis deals with the analysis of continuous functions and thus can be seen as a generalisation of multivariate analysis where the dimension of the analysis space p strives to infinity. Several authors propose methods to perform discriminant analysis in this infinite dimensional space. Here, the methodology is introduced to perform discriminant analysis, not on single infinite dimensional functions, but to find a linear combination of p infinite dimensional continuous functions, providing a set of continuous canonical functions which are optimally separated in the canonical space.KEYWORDS: Functional data analysis, linear discriminant analysis, classification     

An empirical goodness-of-fit test for multivariate distributions

An empirical test is presented as a tool for assessing whether a specified multivariate probability model is suitable to describe the underlying distribution of a set of observations. This test is based on the premise that, given any probability distribution, the Mahalanobis distances corresponding to data generated from that distribution will likewise follow a distinct distribution that can be estimated well by means of a large sample. We demonstrate the effectiveness of the test for detecting departures from several multivariate distributions. We then apply the test to a real multivariate data set to confirm that it is consistent with a multivariate beta model.     

On a new goodness‐of‐fit process for families of copulas

A goodness‐of‐fit procedure is proposed for parametric families of copulas. The new test statistics are functionals of an empirical process based on the theoretical and sample versions of Spearman's dependence function. Conditions under which this empirical process converges weakly are seen to hold for many families including the Gaussian, Frank, and generalized Farlie–Gumbel–Morgenstern systems of distributions, as well as the models with singular components described by Durante [Durante ( 2007 ) Comptes Rendus Mathématique. Académie des Sciences. Paris, 344, 195–198]. Thanks to a parametric bootstrap method that allows to compute valid P‐values, it is shown empirically that tests based on Cramér–von Mises distances keep their size under the null hypothesis. Simulations attesting the power of the newly proposed tests, comparisons with competing procedures and complete analyses of real hydrological and financial data sets are presented. The Canadian Journal of Statistics 37: 80‐101; 2009 © 2009 Statistical Society of Canada     

Goodness‐of‐fit Test for Directional Data

In this paper, we study the problem of testing the hypothesis on whether the density f of a random variable on a sphere belongs to a given parametric class of densities. We propose two test statistics based on the L² and L¹ distances between a non‐parametric density estimator adapted to circular data and a smoothed version of the specified density. The asymptotic distribution of the L² test statistic is provided under the null hypothesis and contiguous alternatives. We also consider a bootstrap method to approximate the distribution of both test statistics. Through a simulation study, we explore the moderate sample performance of the proposed tests under the null hypothesis and under different alternatives. Finally, the procedure is illustrated by analysing a real data set based on wind direction measurements.     

Some Applications of Order Statistics to Multivariate Analysis II

ABSTRACT

Harter (1979) summarized applications of order statistics to multivariate analysis up through 1949. The present paper covers the period 1950–1959. References in the two papers were selected from the first and second volumes, respectively, of the author's chronological annotated bibliography on order statistics [Harter (1978, 1983)]. Tintner (1950a) established formal relations between four special types of multivariate analysis: (1) canonical correlation, (2) principal components, (3) weighted regression, and (4) discriminant analysis, all of which depend on ordered roots of determinantal equations. During the decade 1950–1959, numerous authors contributed to distribution theory and/or computational methods for ordered roots and their applications to multivariate analysis. Test criteria for (i) multivariate analysis of variance, (ii) comparison of variance–covariance matrices, and (iii) multiple independence of groups of variates when the parent population is multivariate normal were usually derived from the likelihood ratio principle until S. N. Roy (1953) formulated the union–intersection principles on which Roy & Bose (1953) based their simultaneous test and confidence procedure. Roy & Bargmann (1958) used an alternative procedure, called the step–down procedure, in deriving a test for problem (iii), and J. Roy (1958) applied the step–down procedure to problem (i) and (ii), Various authors developed and applied distribution theory for several multivariate distributions. Advances were also made on multivariate tolerance regions [Fraser & Wormleighton (1951), Fraser (1951, 1953), Fraser & Guttman (1956), Kemperman (1956), and Somerville (1958)], a criterion for rejection of multivariate outliers [Kudô (1957)], and linear estimators, from censored samples, of parameters of multivariate normal populations [Watterson (1958, 1959)]. Textbooks on multivariate analysis were published by Kendall (1957) and Anderson (1958), as well as a monograph by Roy (1957) and a book of tables by Pillai (1957).     

Hypothesis Testing of Hazard Ratio Parameters in Marginal Models for Multivariate Failure Time Data

Marginal hazard models for multivariate failure time data have been studied extensively in recent literature. However, standard hypothesis test statistics based on the likelihood method are not exactly appropriate for this kind of model. In this paper, extensions of the three commonly used likelihood hypothesis test statistics are discussed. Generalized Wald, generalized score and generalized likelihood ratio tests for hazard ratio parameters in a marginal hazard model for multivariate failure time data are proposed and their asymptotic distributions examined. The finite sample properties of these statistics are studied through simulations. The proposed method is applied to data from Busselton Population Health Surveys.     

Robust inference from multiple test statistics via permutations: a better alternative to the single test statistic approach for randomized trials

Formal inference in randomized clinical trials is based on controlling the type I error rate associated with a single pre‐specified statistic. The deficiency of using just one method of analysis is that it depends on assumptions that may not be met. For robust inference, we propose pre‐specifying multiple test statistics and relying on the minimum p‐value for testing the null hypothesis of no treatment effect. The null hypothesis associated with the various test statistics is that the treatment groups are indistinguishable. The critical value for hypothesis testing comes from permutation distributions. Rejection of the null hypothesis when the smallest p‐value is less than the critical value controls the type I error rate at its designated value. Even if one of the candidate test statistics has low power, the adverse effect on the power of the minimum p‐value statistic is not much. Its use is illustrated with examples. We conclude that it is better to rely on the minimum p‐value rather than a single statistic particularly when that single statistic is the logrank test, because of the cost and complexity of many survival trials. Copyright © 2013 John Wiley & Sons, Ltd.     

Sharp bounds on the expectations of L-statistics expressed in the Gini mean difference units

We describe a method of calculating sharp lower and upper bounds on the expectations of arbitrary, properly centered L-statistics expressed in the Gini mean difference units of the original i.i.d. observations. Precise values of bounds are derived for the single-order statistics, their differences, and some examples of L-estimators. We also present the families of discrete distributions which attain the bounds, possibly in the limit.     

Interpretation of Canonical Discriminant Functions,Canonical Variates,and Principal Components

Canonical discriminant functions are defined here as linear combinations that separate groups of observations, and canonical variates are defined as linear combinations associated with canonical correlations between two sets of variables. In standardized form, the coefficients in either type of canonical function provide information about the joint contribution of the variables to the canonical function. The standardized coefficients can be converted to correlations between the variables and the canonical function. These correlations generally alter the interpretation of the canonical functions. For canonical discriminant functions, the standardized coefficients are compared with the correlations, with partial t and F tests, and with rotated coefficients. For canonical variates, the discussion includes standardized coefficients, correlations between variables and the function, rotation, and redundancy analysis. Various approaches to interpretation of principal components are compared: the choice between the covariance and correlation matrices, the conversion of coefficients to correlations, the rotation of the coefficients, and the effect of special patterns in the covariance and correlation matrices.     

A nonparametric allocation scheme for classification based on transvariation probabilities

In this paper, a nonparametric discriminant analysis procedure that is less sensitive than traditional procedures to deviations from the usual assumptions is proposed. The procedure uses the projection pursuit methodology where the projection index is the two-group transvariation probability. Montanari [A. Montanari, Linear discriminant analysis and transvariation, J. Classification 21 (2004), pp. 71–88] proposed and used this projection index to measure group separation but allocated the new observation using distances. Our procedure employs a method of allocation based on group–group transvariation probability to classify the new observation. A simulation study shows that the procedure proposed in this paper provides lower misclassification error rates than classical procedures like linear discriminant analysis and quadratic discriminant analysis and recent procedures like maximum depth and Montanari's transvariation-based classifiers, when the underlying distributions are skewed and/or the prior probabilities are unequal.     

A survey on multivariate chi-square distributions and their applications in testing multiple hypotheses

We are concerned with three different types of multivariate chi-square distributions. Their members play important roles as limiting distributions of vectors of test statistics in several applications of multiple hypotheses testing. We explain these applications and consider the computation of multiplicity-adjusted p-values under the respective global hypothesis. By means of numerical examples, we demonstrate how much gain in level exhaustion or, equivalently, power can be achieved with corresponding multivariate multiple tests compared with approaches which are only based on univariate marginal distributions and do not take the dependence structure among the test statistics into account. As a further contribution of independent value, we provide an overview of essentially all analytic formulas for computing multivariate chi-square probabilities of the considered types which are available up to present. These formulas were scattered in the previous literature and are presented here in a unified manner.     

Robust mixture modelling using the t distribution

Normal mixture models are being increasingly used to model the distributions of a wide variety of random phenomena and to cluster sets of continuous multivariate data. However, for a set of data containing a group or groups of observations with longer than normal tails or atypical observations, the use of normal components may unduly affect the fit of the mixture model. In this paper, we consider a more robust approach by modelling the data by a mixture of t distributions. The use of the ECM algorithm to fit this t mixture model is described and examples of its use are given in the context of clustering multivariate data in the presence of atypical observations in the form of background noise.     

EXACT DISTRIBUTION OF CERTAIN GENERAL TEST STATISTICS IN MULTIVARIATE ANALYSIS

In this paper, exact solution of Wilks' type-B integral equation has been obtained in its most general form as a series of weighted gamma distributions. This general result then gives the distributions of many test statistics in multivariate analysis. In particular the distributions of Wilks' Λ, the sphericity test criterion, and Bartlett's test statistic, are derived in easily computable form.