VARIABLE SELECTION ALONG CANONICAL VECTORS

When describing and displaying inter-group relationships using canonical variate analysis, the question often arises: which variables contribute most to the group separation along particular canonical vectors? This problem is discussed in the context of an example which uses remotely sensed data to discriminate between cover classes. The appropriate between-groups matrix to be input into subset selection procedures is identified. When combined with subsequent simplification of the vectors, the approach leads to a simpler interpretation of the canonical variate display.     

Linear generalizations of various matric-t distributions

The concept of a matric-t variate is extended to cases where the positive (definite) part of the variate, which is usually Wishart distributed independently of the normal part, is a linear sum of positive (definite) variates with positive coefficients. These distributions and their quadratic forms are of importance i.a, for the exact solution to the multi¬variate Behrens-Fisher problem. A few useful identities con¬cerning the invariant polynomials with matrix arguments are derived     

Approximate optimal allocation in repeated sampling from a finite population

Samples of size n are drawn from a finite population on each of two occasions. On the first occasion a variate x is measured, and on the second a variate y. In estimating the population mean of y, the variance of the best linear unbiased combination of means for matched and unmatched samples is itself minimized, with respect to the sampling design on the second occasion, by a certain degree of matching. This optimal allocation depends on the population correlation coefficient, which previous authors have assumed known. We estimate the correlation from an initial matched sample, then an approximately optimal allocation is completed and an estimator formed which, under a bivariate normal superpopulation model, has model expected mean square error equal, apart from an error of order n^-2, to the minimum enjoyed by any linear, unbiased estimator.     

Estimating a distribution for each category of a sensitive variate

The use of a Randomized Response (RR) design makes it possible to estimate the distribution of a sensitive variate. In this paper, the estimation of the distribution of a non-sensitive variate for each category of a sensitive variate is considered for the case where data on the sensitive variate is obtained by use of an RR procedure. Simple estimators are developed without making any distributional assumptions about the non-sensitive variate. However, if distributional assumptions are made, it is shown that the EM algorithm may be used to compute Maximum Likelihood estimates. Computational comparisons of the estimators, using simulation, indicate that the simple estimators perform well, particularly for large sample sizes.     

On best affine prediction

The theory of best affine prediction (BAP) is extended to the vector case with possibly singular variance matrix of the predictor variable. The theory is then applied to derive Thomson’s classical predictor for factor scores, allowing for a singular variance matrix of the factors. The results are formulated in a free distribution setting. Further, Bartlett’s estimator is considered and compared with Thomson’s predictor. The authors are thankful to the two referees, one for a suggestion that led to the Addendum of the paper, and the other one for several very useful remarks. Research supported by the Spanish grant BEC2000-0983.     

CANONICAL VARIATE ANALYSIS WITH SPATIALLY-CORRELATED DATA

A general canonical variate model is derived when the observations are spatially correlated. For spatial covariance structures resulting from dependence of a pixel on its nearest neighbours, the solution reduces to an analysis of neighbour-corrected values. The usual analysis, in which spatial correlation is ignored, gives similar canonical vectors but over-estimates the canonical roots. A formula for approximating the reduction in the canonical roots to adjust for the spatial correlation is given.     

Performance of Variable Selection Methods in Regression Using Variations of the Bayesian Information Criterion

The Bayesian information criterion (BIC) is widely used for variable selection. We focus on the regression setting for which variations of the BIC have been proposed. A version that includes the Fisher Information matrix of the predictor variables performed best in one published study. In this article, we extend the evaluation, introduce a performance measure involving how closely posterior probabilities are approximated, and conclude that the version that includes the Fisher Information often favors regression models having more predictors, depending on the scale and correlation structure of the predictor matrix. In the image analysis application that we describe, we therefore prefer the standard BIC approximation because of its relative simplicity and competitive performance at approximating the true posterior probabilities.     

Simple least squares estimation versus best linear unbiased prediction

Necessary and sufficient conditions are developed for the simple least squares estimator to coincide with the best linear unbiased predictor. The conditions obtained are valid for a general linear model and are generalizations of the condition given by Watson (1972). Also, as a preliminary result, a new representation of the best linear unbiased predictor is established.     

Generalized f variates in the general linear model

In this paper the theory of the generalized F variate is examined. A large class of statistics is constructed for linear for linear models when the errors are distributed N(0, ∑), where ∑ is positive definite. These statistics are shown to have a generalized F distribution. The role of the generalized F variate is then studied in detail for a linear model of constant intraclass correlation.     

An improvement of a monte carlo technique using asymptotic moments with an application to the likelihood ratio statistic

The use of asymptotic moments to increase the precision of the control variate technique for Monte Carlo estimation is dis­cussed. An application is made to the estimation of the mean and variance of the likelihood ratio goodness–of–fit statistic with the Pearson statistic used as a control variate. Estimates of the variance reductions are given.     

The concept of risk unbiasedness in statistical prediction

This paper extends the concept of risk unbiasedness for applying to statistical prediction and nonstandard inference problems, by formalizing the idea that a risk unbiased predictor should be at least as close to the “true” predictant as to any “wrong” predictant, on the average. A novel aspect of our approach is measuring closeness between a predicted value and the predictant by a regret function, derived suitably from the given loss function. The general concept is more relevant than mean unbiasedness, especially for asymmetric loss functions. For squared error loss, we present a method for deriving best (minimum risk) risk unbiased predictors when the regression function is linear in a function of the parameters. We derive a Rao–Blackwell type result for a class of loss functions that includes squared error and LINEX losses as special cases. For location-scale families, we prove that if a unique best risk unbiased predictor exists, then it is equivariant. The concepts and results are illustrated with several examples. One interesting finding is that in some problems a best unbiased predictor does not exist, but a best risk unbiased predictor can be obtained. Thus, risk unbiasedness can be a useful tool for selecting a predictor.     

On certain unbiased product estimators

When data on an auxiliary variate is available on all the units of the population, negatively correlated with the study variate, Robson (1957) and Murthy (1964) proposed product method of estimation for the estimation of the population total (mean) of the study variate. In this paper, we discuss a method given in Rao (1983) and obtain a simpler dervation of the class of unbiased product estimators for the case of Simple Random Sampling WithOut Replacement design as well as for the case of interpenetrating subsamples design which follows as a limiting case. Finally, we shall illustrate the results by means of two simple numerical example from live data.     

Tolerance regions for random vectors and best linear predictors

In the case that vectors X and Y have a joint multivariate normal distribution, tolerance regions are found for the best linear predictor of Y using X if samples are used to estimate the regression coeffierante. Tolerance regions are also found for Y. In addition, simultaneous tolerance intervals for all linear functions of Y or of the best linear predictor of Y using X are found.     

Linear predictors of future order statistics

In this paper we establish an optimal asymptotic linear predictor which does not involve the finite-sample variance-covariance structure. Extensions to the problem of finding the best linear unbiased and simple linear unbiased predictors for k samples are given. Moreover, we obtain alternative linear predictors by modifying the covariance matrix by either an identity matrix or a diagonal matrix. For normal, logistic and Rayleigh samples of size 10, the alternative linear predictors with these modifications have high efficiency when compared with the best linear unbiased predictor.     

Point predictors of the latent failure times of censored units in progressively Type-II censored competing risks data from the exponential distributions

In this paper, we discuss the problem of predicting times to the latent failures of units censored in multiple stages in a progressively Type-II censored competing risks model. It is assumed that the lifetime distribution of the latent failure times are independent and exponential-distributed with the different scale parameters. Several classical point predictors such as the maximum likelihood predictor, the best unbiased predictor, the best linear unbiased predictor, the median unbiased predictor and the conditional median predictor are obtained. The Bayesian point predictors are derived under squared error loss criterion. Moreover, the point estimators of the unknown parameters are obtained using the observed data and different point predictors of the latent failure times. Finally, Monte-Carlo simulations are carried out to compare the performances of the different methods of prediction and estimation and one real data is used to illustrate the proposed procedures.     

ESTIMATION, PREDICTION AND INFERENCE FOR THE LASSO RANDOM EFFECTS MODEL

The least absolute shrinkage and selection operator (LASSO) can be formulated as a random effects model with an associated variance parameter that can be estimated with other components of variance. In this paper, estimation of the variance parameters is performed by means of an approximation to the marginal likelihood of the observed outcomes. The approximation is based on an alternative but equivalent formulation of the LASSO random effects model. Predictions can be made using point summaries of the predictive distribution of the random effects given the data with the parameters set to their estimated values. The standard LASSO method uses the mode of this distribution as the predictor. It is not the only choice, and a number of other possibilities are defined and empirically assessed in this article. The predictive mode is competitive with the predictive mean (best predictor), but no single predictor performs best across in all situations. Inference for the LASSO random effects is performed using predictive probability statements, which are more appropriate under the random effects formulation than tests of hypothesis.     

Multiple comparisons with ‘best’ for multivariate normal populations

Comparisons of multivariate normal populations are made using a mul-tivariate approach (instead of reducing the problem to a univariate one). A rather negative finding is that, for comparisons with the ‘best’ of each variate, repeated univariate comparisons appear to be almost as efficient as multivariate comparisons, at least for the bivariate case and, under certain circumstances, for higher dimensional cases. Investigations are done on comparisons with the ‘MAX-best’ population (that one having the largest maximum of the marginal means), the ‘MIN-best’ (having the largest minimum) and the ‘O-best’ (being closest to largest in all marginal means). Detailed results are given for the bivariate normal with extensions indicated for the multivariate.     

Nonparametric Random Variate Generation Using a Piecewise-Linear Cumulative Distribution Function

The standard approach to solving the interpolation problem for a trace-driven simulation involving a continuous random variable is to construct a piecewise-linear cdf that fills in the gaps between the data values. Some probabilistic properties of this estimator are derived, and three extensions to the standard approach (matching moments, weighted values, and right-censored data) are presented, along with associated random variate generation algorithms. The algorithm is a nonparametric blackbox variate generator requiring only observed data from the user.     

Riemann sums for MCMC estimation and convergence monitoring

This paper develops an extension of the Riemann sum techniques of Philippe (J. Statist. Comput. Simul. 59: 295–314) in the setup of MCMC algorithms. It shows that these techniques apply equally well to the output of these algorithms, with similar speeds of convergence which improve upon the regular estimator. The restriction on the dimension associated with Riemann sums can furthermore be overcome by Rao–Blackwellization methods. This approach can also be used as a control variate technique in convergence assessment of MCMC algorithms, either by comparing the values of alternative versions of Riemann sums, which estimate the same quantity, or by using genuine control variate, that is, functions with known expectations, which are available in full generality for constants and scores.     

Measures of fit in principal component and canonical variate analyses

Treating principal component analysis (PCA) and canonical variate analysis (CVA) as methods for approximating tables, we develop measures, collectively termed predictivity, that assess the quality of fit independently for each variable and for all dimensionalities. We illustrate their use with data from aircraft development, the African timber industry and copper froth measurements from the mining industry. Similar measures are described for assessing the predictivity associated with the individual samples (in the case of PCA and CVA) or group means (in the case of CVA). For these measures to be meaningful, certain essential orthogonality conditions must hold that are shown to be satisfied by predictivity.