Discriminant analysis under the common principal components model

For two or more populations of which the covariance matrices have a common set of eigenvectors, but different sets of eigenvalues, the common principal components (CPC) model is appropriate. Pepler et al. (2015 Pepler, P. T., Uys, D. W. and Nel, D. G. (2015). Regularised covariance matrix estimation under the common principal components model. Communications in Statistics: Simulation and Computation. (In press). [Google Scholar]) proposed a regularized CPC covariance matrix estimator and showed that this estimator outperforms the unbiased and pooled estimators in situations, where the CPC model is applicable. This article extends their work to the context of discriminant analysis for two groups, by plugging the regularized CPC estimator into the ordinary quadratic discriminant function. Monte Carlo simulation results show that CPC discriminant analysis offers significant improvements in misclassification error rates in certain situations, and at worst performs similar to ordinary quadratic and linear discriminant analysis. Based on these results, CPC discriminant analysis is recommended for situations, where the sample size is small compared to the number of variables, in particular for cases where there is uncertainty about the population covariance matrix structures.     

Recovery of inter-block information: extensions in a two variance component model

For a two variance component mixed linear model, it is shown that under suitable conditions there exists a nonlinear unbiased estimator that is better than a best linear unbiased estimator defined with respect to a given singular covariance matrix. It is also shown how this result applies to improving on intra-block estimators and on estimators like the unweighted means estimator in a random one-way model.     

Unbiased Estimator for a Covariance Matrix Under Two-Step Monotone Incomplete Sample

In this article, we consider an inference for a covariance matrix under two-step monotone incomplete sample. The maximum likelihood estimator of the mean vector is unbiased but that of the covariance matrix is biased. We derive an unbiased estimator for the covariance matrix using some fundamental properties of the Wishart matrix. The properties of the estimators are investigated and the accuracies are checked by a numerical simulation.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.     

On Necessary and Sufficient Conditions for Ordinary Least Squares Estimators to Be Best Linear Unbiased Estimators

Two often-quoted necessary and sufficient conditions for ordinary least squares estimators to be best linear unbiased estimators are described. Another necessary and sufficient condition is described, providing an additional tool for checking to see whether the covariance matrix of a given linear model is such that the ordinary least squares estimator is also the best linear unbiased estimator. The new condition is used to show that one of the two published conditions is only a sufficient condition.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

Approaches to regression analysis with multiple measurements from individual sampling units

Application of ordinary least-squares regression to data sets which contain multiple measurements from individual sampling units produces an unbiased estimator of the parameters but a biased estimator of the covariance matrix of the parameter estimates. The present work considers a random coefficient, linear model to deal with such data sets: this model permits many senses in which multiple measurements are taken from a sampling unit, not just when it is measured at several times. Three procedures to estimate the covariance matrix of the error term of the model are considered. Given these, three procedures to estimate the parameters of the model and their covariance matrix are considered; these are ordinary least-squares, generalized least-squares, and an adjusted ordinary least-squares procedure which produces an unbiased estimator of the covariance matrix of the parameters with small samples. These various procedures are compared in simulation studies using three examples from the biological literature. The possibility of testing hypotheses about the vector of parameters is also considered. It is found that all three procedures for regression estimation produce estimators of the parameters with bias of no practical consequence, Both generalized least-squares and adjusted ordinary least-squares generally produce estimators of the covariance matrix of the parameter estimates with bias of no practical consequence, while ordinary least-squares produces a negatively biased estimator. Neither ordinary nor generalized least-squares provide satisfactory hypothesis tests of the vector of parameter estimates. It is concluded that adjusted ordinary least-squares, when applied with either of two of the procedures used to estimate the error coveriance matrix, shows promise for practical application with data sets of the nature considered here.     

Admissible linear estimation in singular linear models

The admissibility results of Rao (1976), proved in the context of a nonsingular covariance matrix, are exteneded to the situation where the covariance matrix is singular. Admi.s s Lb Le linear estimators in the Gauss-Markoff model are characterized and admis-sibility of the best linear unbiased estimator is investigated.     

Estimation of the Length Distribution of Marine Populations in the Gaussian-multinomial Setting using the Method of Moments

In this paper, the problem of estimation of the length distribution of marine populations in the Gaussian-multinomial model is considered. For the purpose of the mean and covariance parameter estimation, the method of moments estimators are developed. That is, minimum variance linear unbiased estimator for the mean frequency vector is derived and a consistent estimator for the covariance matrix of the length observations is presented. The usefulness of the proposed estimators is illustrated with an analysis of real cod length measurement data.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Alternative estimators of the common regression matrix in two GMANOVA models under weighted quadratic losses

We consider the problem of estimating the common regression matrix of two GMANOVA models with different unknown covariance matrices under certain type of loss functions which include a weighted quadratic loss function as a special case. We consider a class of estimators, which contains the Graybill–Deal-type estimator proposed by Sugiura and Kubokawa (Ann. Inst. Statist. Math. 40 (1988) 119), and we give its risk representation via Kubokawa and Srivastava's (Ann. Statist. 27 (1999) 600; J. Multivariate Anal. 76 (2001) 138) identities when the error matrices follow the elliptically contoured distributions. Using the method similar to an approximate minimization of the unbiased risk estimate due to Stein (Studies in the Statistical Theory of Estimation, vol. 74, Nauka, Leningrad, 1977, p. 4), we obtain an alternative estimator to the Graybill–Deal-type estimator which was given under the normality assumption. However, it seems difficult to evaluate the risk of our proposed estimator analytically because of complex nature of its risk function. Instead, we conduct a Monte-Carlo simulation to evaluate the performance of our proposed estimator. The results indicate that our proposed estimator compares favorably with the Graybill–Deal-type estimator.     

Fisher information and maximum-likelihood estimation of covariance parameters in Gaussian stochastic processes

The inverse of the Fisher information matrix is commonly used as an approximation for the covariance matrix of maximum-likelihood estimators. We show via three examples that for the covariance parameters of Gaussian stochastic processes under infill asymptotics, the covariance matrix of the limiting distribution of their maximum-likelihood estimators equals the limit of the inverse information matrix. This is either proven analytically or justified by simulation. Furthermore, the limiting behaviour of the trace of the inverse information matrix indicates equivalence or orthogonality of the underlying Gaussian measures. Even in the case of singularity, the estimator of the process variance is seen to be unbiased, and also its variability is approximated accurately from the information matrix.     

Unbiased estimation of the MSE matrices of improved estimators in linear regression

Stein-rule and other improved estimators have scarcely been used in empirical work. One major reason is that it is not easy to obtain precision measures for these estimators. In this paper, we derive unbiased estimators for both the mean squared error (MSE) and the scaled MSE matrices of a class of Stein-type estimators. Our derivation provides the basis for measuring the estimators' precision and constructing confidence bands. Comparisons are made between these MSE estimators and the least squares covariance estimator. For illustration, the methodology is applied to data on energy consumption.     

Estimation of the MSE matrix of the stein estimator

Uniformly minimum-variance unbiased (UMVU) estimators of the total risk and the mean-squared-error (MSE) matrix of the Stein estimator for the multivariate normal mean with unknown covariance matrix are proposed. The estimated MSE matrix is helpful in identifying the components which contribute most to the total risk. It also contains information about the performance of the shrinkage estimator with respect to other quadratic loss functions.     

Predictions and estimations under a group of linear models with random coefficients

This article investigates the problem of establishing best linear unbiased predictors and best linear unbiased estimators of all unknown parameters in a group of linear models with random coefficients and correlated covariance matrix. We shall derive a variety of fundamental statistical properties of the predictors and estimators by using some matrix analysis tools. In particular, we shall establish necessary and sufficient conditions for the predictors and estimators to be equivalent under single and combined equations in the group of models by using the method of matrix equations, matrix rank formulas, and partitioned matrix calculations.     

Double shrinkage estimators for large sparse covariance matrices

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.     

Existence conditions of the uniformly minimum risk unbiased estimators in extended growth curve models

Necessary and sufficient existence conditions are derived for the uniformly minimum risk unbiased estimators of the parameters in extended growth curve models with the general covariance matrix or the uniform covariance structure or the serial covariance structure under convex losses and matrix losses, respectively.     

Accent on Teaching Materials

Estimation of covariance components in the multivariate random-effect model with nested covariance structure is discussed. There are two covariance matrices to be estimated, namely, the between-group and the within-group covariance matrices. These two covariance matrices are most often estimated by forming a multivariate analysis of variance and equating mean square matrices to their expectations. Such a procedure involves taking the difference between the between-group mean square and the within-group mean square matrices, and often produces an estimated between-group covariance matrix that is not nonnegative definite. We present estimators of the two covariance matrices that are always proper covariance matrices. The estimators are the restricted maximum likelihood estimators if the random effects are normally distributed. The estimation procedure is extended to more complicated models, including the twofold nested and the mixed-effect models. A numerical example is presented to illustrate the use of the estimation procedure.     

Wavelet threshold based on Stein's unbiased risk estimators of restricted location parameter in multivariate normal

In this paper, the problem of estimating the mean vector under non-negative constraints on location vector of the multivariate normal distribution is investigated. The value of the wavelet threshold based on Stein''s unbiased risk estimators is calculated for the shrinkage estimator in restricted parameter space. We suppose that covariance matrix is unknown and we find the dominant class of shrinkage estimators under Balance loss function. The performance evaluation of the proposed class of estimators is checked through a simulation study by using risk and average mean square error values.     

Use of Spacings in the Estimation of Common Scale Parameter of Several Distributions

In this article, we consider the problem of best linear unbiased estimation and best linear invariant estimation of the common scale parameter of several distributions using spacing of the pooled sample of all observations of individual samples. We derived conditions for the non negativity of the scale estimator obtained by the above methods. Further, we obtained necessary and sufficient conditions for the derived estimators to be constant multiples of the pooled sample range.