Covariance structure approximation via gLasso in high-dimensional supervised classification

Recent work has shown that the Lasso-based regularization is very useful for estimating the high-dimensional inverse covariance matrix. A particularly useful scheme is based on penalizing the ?₁ norm of the off-diagonal elements to encourage sparsity. We embed this type of regularization into high-dimensional classification. A two-stage estimation procedure is proposed which first recovers structural zeros of the inverse covariance matrix and then enforces block sparsity by moving non-zeros closer to the main diagonal. We show that the block-diagonal approximation of the inverse covariance matrix leads to an additive classifier, and demonstrate that accounting for the structure can yield better performance accuracy. Effect of the block size on classification is explored, and a class of asymptotically equivalent structure approximations in a high-dimensional setting is specified. We suggest a variable selection at the block level and investigate properties of this procedure in growing dimension asymptotics. We present a consistency result on the feature selection procedure, establish asymptotic lower an upper bounds for the fraction of separative blocks and specify constraints under which the reliable classification with block-wise feature selection can be performed. The relevance and benefits of the proposed approach are illustrated on both simulated and real data.     

Covariance Matrix Estimation via Network Structure

In this article, we employ a regression formulation to estimate the high-dimensional covariance matrix for a given network structure. Using prior information contained in the network relationships, we model the covariance as a polynomial function of the symmetric adjacency matrix. Accordingly, the problem of estimating a high-dimensional covariance matrix is converted to one of estimating low dimensional coefficients of the polynomial regression function, which we can accomplish using ordinary least squares or maximum likelihood. The resulting covariance matrix estimator based on the maximum likelihood approach is guaranteed to be positive definite even in finite samples. Under mild conditions, we obtain the theoretical properties of the resulting estimators. A Bayesian information criterion is also developed to select the order of the polynomial function. Simulation studies and empirical examples illustrate the usefulness of the proposed methods.     

Covariance structure selection in general mixed models

This article describes a unified approach to variance modeling and inference in the context of a general form of the normal-theory linear mixed model. The primary variance modeling objects are parameterized covari-ance structures, examples being diagonal, compound-symmetry, unstructured, timeseries, and spatial. These structures can enter in two different places in the general mixed model, and the combination of one or both of these places with the variety of structures provides a rich class of variance models. The approach is likelihood-based, and involves the use of both maximum likelihood and restricted maximum likelihood. Two examples provide illustration.     

Covariance analysis and associated spectra for classes of nonstationary processes

In 1951, Cramér introduced a class of nonstationary processes. This broad class of processes contains the important harmonizable and stationary classes of processes. The Cramér class can have additional structure imposed upon it through Cesàro summability considerations. These refined Cramér classes, termed (c,p)-summable Cramér, have recently been considered by Swift (in: M.M. Rao (Ed.), Real and Stochastic Analysis: Recent Advances, CRC Press, Boca Raton, FL, 1997, p. 303). In this paper, the relationship between the (c,p)-summable Cramér classes and the (KF,p) classes of processes introduced by Rao in 1985 is considered. The (KF,p) classes of processes are a generalization of the class of processes considered by Kampé de Feriet and Frenkiel. A continuity theorem for the (KF,p) classes is obtained. This result yields a spectral representation for the (KF,p) classes. Some (KF,p) class processes are shown to arise as the solution to a difference equation obtained from a linear model of a noisy communication channel.     

Covariance structure of wavelet coefficients: theory and models in a Bayesian perspective

We present theoretical results on the random wavelet coefficients covariance structure. We use simple properties of the coefficients to derive a recursive way to compute the within- and across-scale covariances. We point out a useful link between the algorithm proposed and the two-dimensional discrete wavelet transform. We then focus on Bayesian wavelet shrinkage for estimating a function from noisy data. A prior distribution is imposed on the coefficients of the unknown function. We show how our findings on the covariance structure make it possible to specify priors that take into account the full correlation between coefficients through a parsimonious number of hyperparameters. We use Markov chain Monte Carlo methods to estimate the parameters and illustrate our method on bench-mark simulated signals.     

Testing Covariance Stationarity

In this paper, we show that the widely used stationarity tests such as the Kwiatkowski Phillips, Schmidt, and Shin (KPSS) test have power close to size in the presence of time-varying unconditional variance. We propose a new test as a complement of the existing tests. Monte Carlo experiments show that the proposed test possesses the following characteristics: (i) in the presence of unit root or a structural change in the mean, the proposed test is as powerful as the KPSS and other tests; (ii) in the presence of a changing variance, the traditional tests perform badly whereas the proposed test has high power comparing to the existing tests; (iii) the proposed test has the same size as traditional stationarity tests under the null hypothesis of stationarity. An application to daily observations of return on U.S. Dollar/Euro exchange rate reveals the existence of instability in the unconditional variance when the entire sample is considered, but stability is found in subsamples.     

Testing Covariance Stationarity

In this paper, we show that the widely used stationarity tests such as the Kwiatkowski Phillips, Schmidt, and Shin (KPSS) test have power close to size in the presence of time-varying unconditional variance. We propose a new test as a complement of the existing tests. Monte Carlo experiments show that the proposed test possesses the following characteristics: (i) in the presence of unit root or a structural change in the mean, the proposed test is as powerful as the KPSS and other tests; (ii) in the presence of a changing variance, the traditional tests perform badly whereas the proposed test has high power comparing to the existing tests; (iii) the proposed test has the same size as traditional stationarity tests under the null hypothesis of stationarity. An application to daily observations of return on U.S. Dollar/Euro exchange rate reveals the existence of instability in the unconditional variance when the entire sample is considered, but stability is found in subsamples.     

Power of analysis of covariance tests under conditional specification

The procedure-wise power functions of two strategies for balanced single-factor analysis of covariance in the presence of possibly unequal regression slopes are evaluated and illustrated. The strategies differ in the action to be taken following a re-^ jection by the preliminary test for equal slopes. The first strategy simply discards the covariate and respecifies the model as the one-way ANOVA model for testing factor effects. The second leaves the unequal slopes covariance model intact, but respecifies the factor effects hypothesis to address the factor level means adjusted to the sample average of the covariate. One additional strategy, that of testing factor effects only if the preliminary slopes test does not reject, is included for comparison purposes. Computation of the power functions requires extensive use of the results obtained in Hawkins and Han (1986) concerning the bivariate distributions of certain ratios of independent noncentral chi-square random variables.     

Prior specification of neighbourhood and interaction structure in binary Markov random fields

We formulate a prior distribution for the energy function of stationary binary Markov random fields (MRFs) defined on a rectangular lattice. In the prior we assign distributions to all parts of the energy function. In particular we define priors for the neighbourhood structure of the MRF, what interactions to include in the model, and for potential values. We define a reversible jump Markov chain Monte Carlo (RJMCMC) procedure to simulate from the corresponding posterior distribution when conditioned to an observed scene. Thereby we are able to learn both the neighbourhood structure and the parametric form of the MRF from the observed scene. We circumvent evaluations of the intractable normalising constant of the MRF when running the RJMCMC algorithm by adopting a previously defined approximate auxiliary variable algorithm. We demonstrate the usefulness of our prior in two simulation examples and one real data example.     

Large Dynamic Covariance Matrices

Second moments of asset returns are important for risk management and portfolio selection. The problem of estimating second moments can be approached from two angles: time series and the cross-section. In time series, the key is to account for conditional heteroscedasticity; a favored model is Dynamic Conditional Correlation (DCC), derived from the ARCH/GARCH family started by Engle (1982). In the cross-section, the key is to correct in-sample biases of sample covariance matrix eigenvalues; a favored model is nonlinear shrinkage, derived from Random Matrix Theory (RMT). The present article marries these two strands of literature to deliver improved estimation of large dynamic covariance matrices. Supplementary material for this article is available online.     

Bayesian and frequentist confidence intervals via adjusted likelihoods under prior specification on the interest parameter

Suppose a prior is specified only on the interest parameter and a posterior distribution, free from nuisance parameters, is considered on the basis of the profile likelihood or an adjusted version thereof. In this setup, we derive higher order asymptotic results on the construction of confidence intervals that have approximately correct posterior as well as frequentist coverage. Apart from meeting both Bayesian and frequentist objectives under prior specification on the interest parameter alone, these results allow a comparison with their counterpart arising when the nuisance parameters are known, and hence provide additional justification for the Cox and Reid adjustment from a Bayesian-cum-frequentist perspective, with regard to neutralization of unknown nuisance parameters.     

Graphical Comparison of Covariance Matrices

Procedures for comparing within-group covariance matrices are developed, based on separate analyses of the variances and of the correlations. The variances and the correlations are represented as two two-way tables, with the columns representing groups. Graphical procedures based on comparisons of linear regressions are presented, by considering a multiplicative columns-regression model for the interaction of groups x variances and of groups x correlations. A multivariate comparison leads to the use of a generalized eigenanalysis to display any differences in covariance structure.     

A Note on Estimating Covariance Components

A well-known formula for expressing a covariance in terms of variances is shown to hold true for estimating components of covariance.     

Copula structure analysis

Summary.  We extend the standard approach of correlation structure analysis for dimension reduction of high dimensional statistical data. The classical assumption of a linear model for the distribution of a random vector is replaced by the weaker assumption of a model for the copula. For elliptical copulas a correlation-like structure remains, but different margins and non-existence of moments are possible. After introducing the new concept and deriving some theoretical results we observe in a simulation study the performance of the estimators: the theoretical asymptotic behaviour of the statistics can be observed even for small sample sizes. Finally, we show our method at work for a financial data set and explain differences between our copula-based approach and the classical approach. Our new method yielear models also.     

Covariance changes detection in multivariate time series

This paper studies the detection of step changes in the variances and in the correlation structure of the components of a vector of time series. Two procedures based on the likelihood ratio test (LRT) statistic and on a cumulative sums (cusum) statistic are considered and compared in a simulation study. We conclude that for a single covariance change the cusum procedure is more powerful in small and medium samples, whereas the likelihood ratio test is more powerful in large samples. However, for several covariance changes the cusum procedure works clearly better. The procedures are illustrated in two real data examples.     

Covariance Analysis for Split-Plot and Split-Block Designs

Commentaries are informative essays dealing with viewpoints of statistical practice, statistical education, and other topics considered to be of general interest to the broad readership of The American Statistician. Commentaries are similar in spirit to Letters to the Editor, but they involve longer discussions of background, issues, and perspectives. All commentaries will be refereed for their merit and compatibility with these criteria.

Proper methodology for the analysis of covariance for experiments designed in a split-plot or split-block design is not found in the statistical literature. Analyses for these designs are often performed incompletely or even incorrectly. This is especially true when popular statistical computer software packages are used for the analysis of these designs. This article provides several appropriate models, ANOVA tables, and standard errors for comparisons from experiments arranged in a standard split-plot, split–split-plot, or split-block design where a covariate has been measured on the smallest size experimental unit.