Simulation study of confidence regions for canonical variate analysis

Canonical variate analysis often involves the construction of confidence regions round points representing group means in a 2-dimensional plot. Traditionally circles have always been constructed, but some authors have recently advocated ellipses as being more appropriate. This paper describes a Monte Carlo study investigating the effect of a range of factors on the inclusion rates of true population means within both types of region for normal data. The traditional circles do not perform too badly within a restricted range, but they are nearly always under-included. The ellipses usually have higher inclusion rates, and so are often closer to the nominal rate, but are sometimes over-included.     

ON SELECTING VARIABLES AND ASSESSING THEIR PERFORMANCE IN LINEAR DISCRIMINANT ANALYSIS

Linear discriminant analysis between two populations is considered in this paper. Error rate is reviewed as a criterion for selection of variables, and a stepwise procedure is outlined that selects variables on the basis of empirical estimates of error. Problems with assessment of the selected variables are highlighted. A leave-one-out method is proposed for estimating the true error rate of the selected variables, or alternatively of the selection procedure itself. Monte Carlo simulations, of multivariate binary as well as multivariate normal data, demonstrate the feasibility of the proposed method and indicate its much greater accuracy relative to that of other available methods.     

Exploratory factor and principal component analyses: some new aspects

Exploratory Factor Analysis (EFA) and Principal Component Analysis (PCA) are popular techniques for simplifying the presentation of, and investigating the structure of, an (n×p) data matrix. However, these fundamentally different techniques are frequently confused, and the differences between them are obscured, because they give similar results in some practical cases. We therefore investigate conditions under which they are expected to be close to each other, by considering EFA as a matrix decomposition so that it can be directly compared with the data matrix decomposition underlying PCA. Correspondingly, we propose an extended version of PCA, called the EFA-like PCA, which mimics the EFA matrix decomposition in the sense that they contain the same unknowns. We provide iterative algorithms for estimating the EFA-like PCA parameters, and derive conditions that have to be satisfied for the two techniques to give similar results. Throughout, we consider separately the cases n>p and p≥n. All derived algorithms and matrix conditions are illustrated on two data sets, one for each of these two cases.     

The effects of data input errors on subsequent statistical inference

Data input errors can potentially affect statistical inferences, but little research has been published to date on this topic. In the present paper, we report the effect of data input errors on the statistical inferences drawn about population parameters in an empirical study involving 280 students from two Polish universities, namely the Warsaw University of Life Sciences – SGGW and the University of Information Technology and Management in Rzeszow. We found that 28% of the students committed at least one data error. While some of these errors were small and did not have any real effect, a few of them had substantial effects on the statistical inferences drawn about the population parameters.     

Data-based interval estimation of classification error rates

Leave-one-out and 632 bootstrap are popular data-based methods of estimating the true error rate of a classification rule, but practical applications almost exclusively quote only point estimates. Interval estimation would provide better assessment of the future performance of the rule, but little has been published on this topic. We first review general-purpose jackknife and bootstrap methodology that can be used in conjunction with leave-one-out estimates to provide prediction intervals for true error rates of classification rules. Monte Carlo simulation is then used to investigate coverage rates of the resulting intervals for normal data, but the results are disappointing; standard intervals show considerable overinclusion, intervals based on Edgeworth approximations or random weighting do not perform well, and while a bootstrap approach provides intervals with coverage rates closer to the nominal ones there is still marked underinclusion. We then turn to intervals constructed from 632 bootstrap estimates, and show that much better results are obtained. Although there is now some overinclusion, particularly for large training samples, the actual coverage rates are sufficiently close to the nominal rates for the method to be recommended. An application to real data illustrates the considerable variability that can arise in practical estimation of error rates.     

ASSESSING ERROR RATE ESTIMATORS: THE LEAVE-ONE-OUT METHOD RECONSIDERED

Many comparative studies of the estimators of error rates of supervised classification rules are based on inappropriate criteria. In particular, although they fix the Bayes error rate, their summary statistics aggregate a range of true error rates. This means that their conclusions about the performance of classification rules cannot be trusted. This paper discusses the general issues involved, and then focuses attention specifically on the leave-one-out estimator. The estimator is investigated in a simulation study, both in absolute terms and in comparison with a popular bootstrap estimator. An improvement to the leave-one-out estimator is suggested, but the bootstrap estimator appears to maintain superiority even when the criteria are adjusted.     

Antedependence models in the analysis of multi-group high-dimensional data

Antedependence modelling has previously been shown to be useful for twogroup discriminant analysis of high-dimensional data. In this paper, the theory of such models is extended to multi-group discriminant analysis and to canonical variate analysis for data display. The application of antedependence models of orders 1, 2 and 3 to spectroscopic analyses of rice samples is described, and the results are compared with those from standard methods based on principal component scores calculated from the data.     

A stopping rule for structure-preserving variable selection

A stopping rule is provided for the backward elimination process suggested by Krzanowski (1987a) for selecting variables to preserve data structure. The stopping rule is based on perturbation theory for Procrustes statistics, and a small simulation study verifies its suitability. Some illustrative examples are also provided and discussed.