Testing a block exchangeable covariance matrix

Testing hypotheses about the structure of a covariance matrix for doubly multivariate data is often considered in the literature. In this paper the Rao's score test (RST) is derived to test the block exchangeable covariance matrix or block compound symmetry (BCS) covariance structure under the assumption of multivariate normality. It is shown that the empirical distribution of the RST statistic under the null hypothesis is independent of the true values of the mean and the matrix components of a BCS structure. A significant advantage of the RST is that it can be performed for small samples, even smaller than the dimension of the data, where the likelihood ratio test (LRT) cannot be used, and it outperforms the standard LRT in a number of contexts. Simulation studies are performed for the sample size consideration, and for the estimation of the empirical quantiles of the null distribution of the test statistic. The RST procedure is illustrated on a real data set from the medical studies.     

Posterior moments of scalar functions of a covariance matrix

Given multivariate normal data and a certain spherically invariant prior distribution on the covariance matrix, it is desired to estimate the moments of the posterior marginal distributions of some scalar functions of the covariance matrix by importance sampling. To this end a family of distributions is defined on the group of orthogonal matrices and a procedure is proposed for selecting one of these distributions for use as a weighting distribution in the importance sampling process. In an example estimates are calculated for the posterior mean and variance of each element in the covariance matrix expressed in the original coordinates, for the posterior mean of each element in the correlation matrix expressed in the original coordinates, and for the posterior mean of each element in the covariance matrix expressed in the coordinates of the principal variables.     

A permutation-based Bayesian approach for inverse covariance estimation

Abstract

Covariance estimation and selection for multivariate datasets in a high-dimensional regime is a fundamental problem in modern statistics. Gaussian graphical models are a popular class of models used for this purpose. Current Bayesian methods for inverse covariance matrix estimation under Gaussian graphical models require the underlying graph and hence the ordering of variables to be known. However, in practice, such information on the true underlying model is often unavailable. We therefore propose a novel permutation-based Bayesian approach to tackle the unknown variable ordering issue. In particular, we utilize multiple maximum a posteriori estimates under the DAG-Wishart prior for each permutation, and subsequently construct the final estimate of the inverse covariance matrix. The proposed estimator has smaller variability and yields order-invariant property. We establish posterior convergence rates under mild assumptions and illustrate that our method outperforms existing approaches in estimating the inverse covariance matrices via simulation studies.     

Robust estimation of a high-dimensional integrated covariance matrix

In this article, we consider a robust method of estimating a realized covariance matrix calculated as the sum of cross products of intraday high-frequency returns. According to recent articles in financial econometrics, the realized covariance matrix is essentially contaminated with market microstructure noise. Although techniques for removing noise from the matrix have been studied since the early 2000s, they have primarily investigated a low-dimensional covariance matrix with statistically significant sample sizes. We focus on noise-robust covariance estimation under converse circumstances, that is, a high-dimensional covariance matrix possibly with a small sample size. For the estimation, we utilize a statistical hypothesis test based on the characteristic that the largest eigenvalue of the covariance matrix asymptotically follows a Tracy–Widom distribution. The null hypothesis assumes that log returns are not pure noises. If a sample eigenvalue is larger than the relevant critical value, then we fail to reject the null hypothesis. The simulation results show that the estimator studied here performs better than others as measured by mean squared error. The empirical analysis shows that our proposed estimator can be adopted to forecast future covariance matrices using real data.     

Estimates of regression coefficients based on the sign covariance matrix

Summary. A new estimator of the regression parameters is introduced in a multivariate multiple-regression model in which both the vector of explanatory variables and the vector of response variables are assumed to be random. The affine equivariant estimate matrix is constructed using the sign covariance matrix (SCM) where the sign concept is based on Oja's criterion function. The influence function and asymptotic theory are developed to consider robustness and limiting efficiencies of the SCM regression estimate. The estimate is shown to be consistent with a limiting multinormal distribution. The influence function, as a function of the length of the contamination vector, is shown to be linear in elliptic cases; for the least squares (LS) estimate it is quadratic. The asymptotic relative efficiencies with respect to the LS estimate are given in the multivariate normal as well as the t -distribution cases. The SCM regression estimate is highly efficient in the multivariate normal case and, for heavy-tailed distributions, it performs better than the LS estimate. Simulations are used to consider finite sample efficiencies with similar results. The theory is illustrated with an example.     

Modified linear discriminant analysis using block covariance matrix in high-dimensional data

Among many classification methods, linear discriminant analysis (LDA) is a favored tool due to its simplicity, robustness, and predictive accuracy but when the number of genes is larger than the number of observations, it cannot be applied directly because the within-class covariance matrix is singular. Also, diagonal LDA (DLDA) is a simpler model compared to LDA and has better performance in some cases. However, in reality, DLDA requires a strong assumption based on mutual independence. In this article, we propose the modified LDA (MLDA). MLDA is based on independence, but uses the information that has an effect on classification performance with the dependence structure. We suggest two approaches. One is the case of using gene rank. The other involves no use of gene rank. We found that MLDA has better performance than LDA, DLDA, or K-nearest neighborhood and is comparable with support vector machines in real data analysis and the simulation study.     

A new estimator of covariance matrix

The problem of estimating a covariance matrix is considered in this paper. Using the so-called partial Iwasawa coordinates of the covariance matrix, a new improved estimator dominating the James-Stein estimator is proposed. The results of a simulation study verifies that the new estimator provides a substantial improvement in risk under Stein's loss.     

Vector random fields with compactly supported covariance matrix functions

The objective of this paper is to construct covariance matrix functions whose entries are compactly supported, and to use them as building blocks to formulate other covariance matrix functions for second-order vector stochastic processes or random fields. In terms of the scale mixture of compactly supported covariance matrix functions, we derive a class of second-order vector stochastic processes on the real line whose direct and cross covariance functions are of Pólya type. Then some second-order vector random fields in R^d${\mathbb{R}}^d$ whose direct and cross covariance functions are compactly supported are constructed by using a convolution approach and a mixture approach.     

Testing identity of high-dimensional covariance matrix

Two new statistics are proposed for testing the identity of high-dimensional covariance matrix. Applying the large dimensional random matrix theory, we study the asymptotic distributions of our proposed statistics under the situation that the dimension p and the sample size n tend to infinity proportionally. The proposed tests can accommodate the situation that the data dimension is much larger than the sample size, and the situation that the population distribution is non-Gaussian. The numerical studies demonstrate that the proposed tests have good performance on the empirical powers for a wide range of dimensions and sample sizes.     

An extension of the multivariate Chebyshev's inequality to a random vector with a singular covariance matrix

ABSTRACT

We extend Chebyshev's inequality to a random vector with a singular covariance matrix. Then we consider the case of a multivariate normal distribution for this generalization.     

A new heteroskedasticity-consistent covariance matrix estimator and inference under heteroskedasticity

To solve the heteroscedastic problem in linear regression, many different heteroskedasticity-consistent covariance matrix estimators have been proposed, including HC0 estimator and its variants, such as HC1, HC2, HC3, HC4, HC5 and HC4m. Each variant of the HC0 estimator aims at correcting the tendency of underestimating the true variances. In this paper, a new variant of HC0 estimator, HC5m, which is a combination of HC5 and HC4m, is proposed. Both the numerical analysis and the empirical analysis show that the quasi-t inference based on HC5m is typically more reliable than inferences based on other covariance matrix estimators, regardless of the existence of high leverage points.     

The covariance matrix of normal order statistics

An approximation is given to calculate V, the covariance matrix for normal order statistics. The approximation gives considerable improvement over previous approximations, and the computing algorithm is available from the authors.     

Small sample behavior of a robust heteroskedasticity consistent covariance matrix estimator

In heteroskedastic regression models, the least squares (OLS) covariance matrix estimator is inconsistent and inference is not reliable. To deal with inconsistency one can estimate the regression coefficients by OLS, and then implement a heteroskedasticity consistent covariance matrix (HCCM) estimator. Unfortunately the HCCM estimator is biased. The bias is reduced by implementing a robust regression, and by using the robust residuals to compute the HCCM estimator (RHCCM). A Monte-Carlo study analyzes the behavior of RHCCM and of other HCCM estimators, in the presence of systematic and random heteroskedasticity, and of outliers in the explanatory variables.     

Minimum density power divergence estimator for covariance matrix based on skew $$t$$ t distribution

Statistical Methods & Applications - In this paper, we study the problem of estimating the covariance matrix of stationary multivariate time series based on the minimum density power divergence...     

AFFINE‐EQUIVARIANT SPATIAL MEDIAN AND ITS USE IN THE MULTIVARIATE MULTI‐SAMPLE LOCATION PROBLEM

The classical spatial median is not affine‐equivariant, which often turns out to be an unfavourable property. In this paper, the asymptotic properties of an affine‐equivariant modification of the spatial median are investigated. It is shown that under some weak regularity conditions, the modified spatial median computed by means of the sample norming matrix is asymptotically equivalent to the one computed by means of the population norming matrix, which yields its asymptotic normality. A consistent estimate of the asymptotic covariance matrix of the modified spatial median is also presented. These results are implemented in a scheme, where the sample norm is determined by means of the sample Dümbgen scatter matrix. The results are utilized in the construction of affine‐invariant test statistics for testing the multi‐sample hypothesis of equality of location parameters. The performance of the proposed tests is demonstrated through a simulation study.     

Reference prior bayes estimator for bivariate normal covariance matrix with risk comparison

The explicit form of the reference prior bayes estimator due to Yang and Ber-ger (1994) for bivariate normal covariance matrix under entropy loss is given in terms of Legendre polynomials when degrees of freedom is even and in terms of hypergeometric functions in general case. The finite series expression of the density function of the ratio of latent roots of bivariate Wishart matrix is obtained and the exact risk is compared with those of James-Stein minimax estimator and other orthogonally equivariant estimators. It is found numerically that the reference prior bayes estimator has the smallest risk among the class of equivariant estimators compared, when the ratio of the largest to the smallest population latent roots of covariance matrix lies in the middle of the interval [1, ∞]. It has larger risk than that of James-Stein minimax estimator when the ratio is large. Moreover it has larger risk than that of MLE when, for instance, degrees of freedom is 20 and the ratio lies between 4 and 8.     

High-dimensional Markowitz portfolio optimization problem: empirical comparison of covariance matrix estimators

We compare the performance of recently developed regularized covariance matrix estimators for Markowitz's portfolio optimization and of the minimum variance portfolio (MVP) problem in particular. We focus on seven estimators that are applied to the MVP problem in the literature; three regularize the eigenvalues of the sample covariance matrix, and the other four assume the sparsity of the true covariance matrix or its inverse. Comparisons are made with two sets of long-term S&P 500 stock return data that represent two extreme scenarios of active and passive management. The results show that the MVPs with sparse covariance estimators have high Sharpe ratios but that the naive diversification (also known as the ‘uniform (on market share) portfolio’) still performs well in terms of wealth growth.     

Supervised classifiers for high-dimensional higher-order data with locally doubly exchangeable covariance structure

We explore the performance accuracy of the linear and quadratic classifiers for high-dimensional higher-order data, assuming that the class conditional distributions are multivariate normal with locally doubly exchangeable covariance structure. We derive a two-stage procedure for estimating the covariance matrix: at the first stage, the Lasso-based structure learning is applied to sparsifying the block components within the covariance matrix. At the second stage, the maximum-likelihood estimators of all block-wise parameters are derived assuming the doubly exchangeable within block covariance structure and a Kronecker product structured mean vector. We also study the effect of the block size on the classification performance in the high-dimensional setting and derive a class of asymptotically equivalent block structure approximations, in a sense that the choice of the block size is asymptotically negligible.     

A control chart based on sample ranges for monitoring the covariance matrix of the multivariate processes

For the univariate case, the R chart and the S ² chart are the most common charts used for monitoring the process dispersion. With the usual sample size of 4 and 5, the R chart is slightly inferior to the S ² chart in terms of efficiency in detecting process shifts. In this article, we show that for the multivariate case, the chart based on the standardized sample ranges, we call the RMAX chart, is substantially inferior in terms of efficiency in detecting shifts in the covariance matrix than the VMAX chart, which is based on the standardized sample variances. The user's familiarity with sample ranges is a point in favor of the RMAX chart. An example is presented to illustrate the application of the proposed chart.