Reduced-rank estimation of the difference between two covariance matrices

We consider m×m$m\times m$ covariance matrices, _Σ1${\Sigma }_1$ and _Σ2${\Sigma }_2$, which satisfy _Σ2-_Σ1=Δ${\Sigma }_2-{\Sigma }_1=\Delta$, where Δ$\Delta$ has a specified rank. Maximum likelihood estimators of _Σ1${\Sigma }_1$ and _Σ2${\Sigma }_2$ are obtained when sample covariance matrices having Wishart distributions are available and rank(Δ)$\text{rank}(\Delta )$ is known. The likelihood ratio statistic for a test about the value of rank(Δ)$\text{rank}(\Delta )$ is also given and some properties of its null distribution are obtained. The methods developed in this paper are illustrated through an example.     

Multilevel maximum likelihood estimation with application to covariance matrices

The asymptotic variance of the maximum likelihood estimate is proved to decrease when the maximization is restricted to a subspace that contains the true parameter value. Maximum likelihood estimation allows a systematic fitting of covariance models to the sample, which is important in data assimilation. The hierarchical maximum likelihood approach is applied to the spectral diagonal covariance model with different parameterizations of eigenvalue decay, and to the sparse inverse covariance model with specified parameter values on different sets of nonzero entries. It is shown computationally that using smaller sets of parameters can decrease the sampling noise in high dimension substantially.     

A hierarchical eigenmodel for pooled covariance estimation

Summary.  Although the covariance matrices corresponding to different populations are unlikely to be exactly equal they can still exhibit a high degree of similarity. For example, some pairs of variables may be positively correlated across most groups, whereas the correlation between other pairs may be consistently negative. In such cases much of the similarity across covariance matrices can be described by similarities in their principal axes, which are the axes that are defined by the eigenvectors of the covariance matrices. Estimating the degree of across-population eigenvector heterogeneity can be helpful for a variety of estimation tasks. For example, eigenvector matrices can be pooled to form a central set of principal axes and, to the extent that the axes are similar, covariance estimates for populations having small sample sizes can be stabilized by shrinking their principal axes towards the across-population centre. To this end, the paper develops a hierarchical model and estimation procedure for pooling principal axes across several populations. The model for the across-group heterogeneity is based on a matrix-valued antipodally symmetric Bingham distribution that can flexibly describe notions of 'centre' and 'spread' for a population of orthogonal matrices.     

Multi-sample test for high-dimensional covariance matrices

In this paper, we propose a new test statistic for testing the equality of high-dimensional covariance matrices for multiple populations. The proposed test statistic generalizes the test of the equality of two population covariance matrices proposed by Li and Chen (2012).     

A permutation-based Bayesian approach for inverse covariance estimation

Abstract

Covariance estimation and selection for multivariate datasets in a high-dimensional regime is a fundamental problem in modern statistics. Gaussian graphical models are a popular class of models used for this purpose. Current Bayesian methods for inverse covariance matrix estimation under Gaussian graphical models require the underlying graph and hence the ordering of variables to be known. However, in practice, such information on the true underlying model is often unavailable. We therefore propose a novel permutation-based Bayesian approach to tackle the unknown variable ordering issue. In particular, we utilize multiple maximum a posteriori estimates under the DAG-Wishart prior for each permutation, and subsequently construct the final estimate of the inverse covariance matrix. The proposed estimator has smaller variability and yields order-invariant property. We establish posterior convergence rates under mild assumptions and illustrate that our method outperforms existing approaches in estimating the inverse covariance matrices via simulation studies.     

A note on homogeneity tests of covariance matrices

In this note we propose two procedures for testing homogeneity of co-variance matrices that are both extensions of Hartley's (1940) test for equality of variances. The first is a two-stage procedure where the first step is a simple test for equality of the largest eigenvalues, and corresponding eigenvectors, of the covariance matrices. The second is based on projection pursuit and seems harder to apply in practice.     

Double shrinkage estimators for large sparse covariance matrices

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.     

Averaging estimation for conditional covariance models

Estimating conditional covariance matrices is important in statistics and finance. In this paper, we propose an averaging estimator for the conditional covariance, which combines the estimates of marginal conditional covariance matrices by Model Averaging MArginal Regression of Li, Linton, and Lu. This estimator avoids the “curse of dimensionality” problem that the local constant estimator of Yin et al. suffered from. We establish the asymptotic properties of the averaging weights and that of the proposed conditional covariance estimator. The finite sample performances are augmented by simulation. An application to portfolio allocation illustrates the practical superiority of the averaging estimator.     

The influence in comparing covariance matrices

The influence measure for the likelihood ratio test for comparing two covariance matrices is derived using the influence curve approach under the normality assumption. The influence measure for testing the equality of covariance matrices against the arbitrariness of them is partitioned into three influence measures: one for testing the equality of covariance matrices against the proportionality of them, another for testing the proportionality against the equality of correlations between them and the other for testing the equality of correlations against the arbitrariness. This partition implies that an observation can be influential in performing some tests among the four tests but not in performing the remaining tests. Thus the partition is more informative than considering the influence measure for the test of equality alone. Each influence measure is useful for detecting outliers in performing the corresponding likelihood ratio test.     

Model-based clustering with sparse covariance matrices

Statistics and Computing - Finite Gaussian mixture models are widely used for model-based clustering of continuous data. Nevertheless, since the number of model parameters scales quadratically with...     

Hierarchic likelihood ratio tests for equality of covariance matrices

Powers and sizes are simulated for hierarchic components of an adjusted likelihood ratio test for equality of covariance matrices.     

Fast covariance estimation for sparse functional data

Smoothing of noisy sample covariances is an important component in functional data analysis. We propose a novel covariance smoothing method based on penalized splines and associated software. The proposed method is a bivariate spline smoother that is designed for covariance smoothing and can be used for sparse functional or longitudinal data. We propose a fast algorithm for covariance smoothing using leave-one-subject-out cross-validation. Our simulations show that the proposed method compares favorably against several commonly used methods. The method is applied to a study of child growth led by one of coauthors and to a public dataset of longitudinal CD4 counts.     

Testing the equality of several covariance matrices

Using a new approach based on Meijer G-functions and computer simulation, we numerically compute the exact null distribution of the modified-likelihood ratio statistic used to test the hypothesis that several covariances matrices of normal distributions are equal. Small samples of different sizes are considered, and for the case of two matrices, we introduce a new test based on determinants, with the null distribution of its criterion also fully computable. Comparisons with published results show the accuracy of our approach, which is proved to be more flexible and adaptable to different cases.     

Tests for high-dimensional covariance matrices using the theory of U-statistics

Test statistics for sphericity and identity of the covariance matrix are presented, when the data are multivariate normal and the dimension, p, can exceed the sample size, n. Under certain mild conditions mainly on the traces of the unknown covariance matrix, and using the asymptotic theory of U-statistics, the test statistics are shown to follow an approximate normal distribution for large p, also when p?n. The accuracy of the statistics is shown through simulation results, particularly emphasizing the case when p can be much larger than n. A real data set is used to illustrate the application of the proposed test statistics.