ASSESSING ERROR RATE ESTIMATORS: THE LEAVE-ONE-OUT METHOD RECONSIDERED

Many comparative studies of the estimators of error rates of supervised classification rules are based on inappropriate criteria. In particular, although they fix the Bayes error rate, their summary statistics aggregate a range of true error rates. This means that their conclusions about the performance of classification rules cannot be trusted. This paper discusses the general issues involved, and then focuses attention specifically on the leave-one-out estimator. The estimator is investigated in a simulation study, both in absolute terms and in comparison with a popular bootstrap estimator. An improvement to the leave-one-out estimator is suggested, but the bootstrap estimator appears to maintain superiority even when the criteria are adjusted.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

ON SELECTING VARIABLES AND ASSESSING THEIR PERFORMANCE IN LINEAR DISCRIMINANT ANALYSIS

Linear discriminant analysis between two populations is considered in this paper. Error rate is reviewed as a criterion for selection of variables, and a stepwise procedure is outlined that selects variables on the basis of empirical estimates of error. Problems with assessment of the selected variables are highlighted. A leave-one-out method is proposed for estimating the true error rate of the selected variables, or alternatively of the selection procedure itself. Monte Carlo simulations, of multivariate binary as well as multivariate normal data, demonstrate the feasibility of the proposed method and indicate its much greater accuracy relative to that of other available methods.