Selection of Variables in Multivariate Regression Models for Large Dimensions

The Akaike information criterion, AIC, and Mallows’ C _p statistic have been proposed for selecting a smaller number of regressors in the multivariate regression models with fully unknown covariance matrix. All of these criteria are, however, based on the implicit assumption that the sample size is substantially larger than the dimension of the covariance matrix. To obtain a stable estimator of the covariance matrix, it is required that the dimension of the covariance matrix is much smaller than the sample size. When the dimension is close to the sample size, it is necessary to use ridge-type estimators for the covariance matrix. In this article, we use a ridge-type estimators for the covariance matrix and obtain the modified AIC and modified C _p statistic under the asymptotic theory that both the sample size and the dimension go to infinity. It is numerically shown that these modified procedures perform very well in the sense of selecting the true model in large dimensional cases.     

Optimal Designs for Approximating a Stochastic Process with Respect to a Minimax Criterion

We study the problem of approximating a stochastic process Y = {Y(t: t ∈ T} with known and continuous covariance function R on the basis of finitely many observations Y(t ₁,), …, Y(t _n ). Dependent on the knowledge about the mean function, we use different approximations ? and measure their performance by the corresponding maximum mean squared error sub _t∈T E(Y(t) ? ?(t))². For a compact T ? ? ^p we prove sufficient conditions for the existence of optimal designs. For the class of covariance functions on T ² = [0, 1]² which satisfy generalized Sacks/Ylvisaker regularity conditions of order zero or are of product type, we construct sequences of designs for which the proposed approximations perform asymptotically optimal.     

The formal posterior of a standard flat prior in MANOVA is incoherent

Summary A standard improper prior for the parameters of a MANOVA model is shown to yield an inference that is incoherent in the sense of Heath and Sudderth. The proof of incoherence is based on the fact that the formal Bayes estimate, sayδ ₀ , of the covariance matrix based on the improper prior and a certain bounded loss function is uniformly inadmissible in that there is another estimatorδ _l and an ɛ>0 such that the risk functions satisfyR(δ _l ,Σ)⩽R δ ₀ ,Σ)−ε for all values of the covariance matrix Σ. The estimatorδ _I is formal Bayes for an alternative improper prior which leads to a coherent inference. Research supported by National Science Foundation grants DMS-89-22607 (for Eaton) and DMS-9123358 (for Sudderth).     

Improved shrinkage estimators for the mean vector of a scale mixture of normals with unknown variance

The problem of estimating the mean θ of a not necessarily normal p-variate (p > 3) distribution with unknown covariance matrix of the form σ²A (A a known diagonal matrix) on the basis of n_i > 2 observations on each coordinate X_t (1 < i < p) is considered. It is argued that the class of scale (or variance) mixtures of normal distributions is a reasonable class to study. Assuming the loss function is quadratic, a large class of improved shrinkage estimators is developed in the case of a balanced design. We generalize results of Berger and Strawderman for one observation in the known-variance case. This methodology also permits the development of a new class of minimax shrinkage estimators of the mean of a p-variate normal distribution for an unbalanced design. Numerical calculations show that the improvements in risk can be substantial.     

On the information-based measure of covariance complexity and its application to the evaluation of multivariate linear models

This paper introduces a new information-theoretic measure of complexity called ICOMP as a decision rule for model selection and evaluation for multivariate linear models. The development of ICOMP is based on the generalization and utilization of the covariance complexity index of van Emden (1971) in estimation of the multivariate linear model. ICOMP is motivated by Akaike's (1973) Information Criterion (AIC), but it is a different procedure than AIC. In linear or nonlinear statistical models ICOMP uses an information-based characterization of: (i) the covariance matrix properties of the parameter estimates of a model starting from their finite sampling distributions, and (ii) the complexity of the inverse-Fisher information matrix (i-FIM) as a new criterion of achievable accuracy of the model As a result, it provides a trade-off between the accuracy of the parameter estimates and the interaction of the residuals of a model via the measure of complexity of their respective covariances. It controls the risks of both insufficient and overparameterized models, and incorporates the assumption of dependence and the independence of the residuals in one criterion function. A model with minimum ICOMP is chosen to be the best model among all possible competing alternative models. ICOMP relieves the researcher of any need to consider the parameter dimension of a model explicitly. A real numerical example is shown in subset selection of variables in multivariate regression analysis to demonstrate the utility and versatility of the new approach.     

Testing homogeneity of several covariance matrices and multi-sample sphericity for high-dimensional data under non-normality

A test for homogeneity of g ? 2 covariance matrices is presented when the dimension, p, may exceed the sample size, n_i, i = 1, …, g, and the populations may not be normal. Under some mild assumptions on covariance matrices, the asymptotic distribution of the test is shown to be normal when n_i, p → ∞. Under the null hypothesis, the test is extended for common covariance matrix to be of a specified structure, including sphericity. Theory of U-statistics is employed in constructing the tests and deriving their limits. Simulations are used to show the accuracy of tests.     

A new robust Kalman filter for filtering the microstructure noise

We propose a robust Kalman filter (RKF) to estimate the true but hidden return when microstructure noise is present. Following Zhou's definition, we assume the observed return Y_t is the result of adding microstructure noise to the true but hidden return X_t. Microstructure noise is assumed to be independent and identically distributed (i.i.d.); it is also independent of X_t. When X_t is sampled from a geometric Brownian motion process to yield Y_t, the Kalman filter can produce optimal estimates of X_t from Y_t. However, the covariance matrix of microstructure noise and that of X_t must be known for this claim to hold. In practice, neither covariance matrix is known so they must be estimated. Our RKF, in contrast, does not need the covariance matrices as input. Simulation results show that the RKF gives essentially identical estimates to the Kalman filter, which has access to the covariance matrices. As applications, estimated X_t can be used to estimate the volatility of X_t.     

Three estimators for the poisson regression model with measurement errors

We consider two consistent estimators for the parameters of the linear predictor in the Poisson regression model, where the covariate is measured with errors. The measurement errors are assumed to be normally distributed with known error variance σ _u ² . The SQS estimator, based on a conditional mean-variance model, takes the distribution of the latent covariate into account, and this is here assumed to be a normal distribution. The CS estimator, based on a corrected score function, does not use the distribution of the latent covariate. Nevertheless, for small σ _u ² , both estimators have identical asymptotic covariance matrices up to the order of σ _u ² . We also compare the consistent estimators to the naive estimator, which is based on replacing the latent covariate with its (erroneously) measured counterpart. The naive estimator is biased, but has a smaller covariance matrix than the consistent estimators (at least up to the order of σ _u ² ).     

Likelihood ratio test for one-sided hypothesis of covariance matrices of two normal populations

The likelihood ratio test is derived for a one-sided hypothesis about the covariance matrices from two multivariate normal populations. In the case of equal sample sizes, the limiting distribution of -21og ?_n is given, where ?_n denotes the likelihood ratio criterion. When dimension p=2, for some alternatives, the power of -21og ?_n of size 0.05 is compared with those of several well-known test statistics using Monte Carlo Methods.     

Joint Central Limit Theorem for Eigenvalue Statistics from Several Dependent Large Dimensional Sample Covariance Matrices with Application

Let X _n = (x _{i j} ) be a k ×n data matrix with complex‐valued, independent and standardized entries satisfying a Lindeberg‐type moment condition. We consider simultaneously R sample covariance matrices , where the Q _r 's are non‐random real matrices with common dimensions p ×k (k ≥p ). Assuming that both the dimension p and the sample size n grow to infinity, the limiting distributions of the eigenvalues of the matrices { B _{n r} } are identified, and as the main result of the paper, we establish a joint central limit theorem (CLT) for linear spectral statistics of the R matrices { B _{n r} }. Next, this new CLT is applied to the problem of testing a high‐dimensional white noise in time series modelling. In experiments, the derived test has a controlled size and is significantly faster than the classical permutation test, although it does have lower power. This application highlights the necessity of such joint CLT in the presence of several dependent sample covariance matrices. In contrast, all the existing works on CLT for linear spectral statistics of large sample covariance matrices deal with a single sample covariance matrix (R = 1).     

Projector operators in the multivariate Zyskind-Martin model

Summary Two quadratic formsS _H andS _E for a testable hypothesis and for an error in the multivariate Zyskind-Martin model with singular covariance matrix are expressed by means of projector operators. Thus the results for the multivariate standard model with identity covariance matrix given by Humak (1977) and Christensen (1987, 1991) are generalized for the case of Zyskind-Martin model. Special cases of our results are formulae forS _H andS _E in Aitken's (1935) model. In the case of general Gauss-Markoff modelS _H andS _E can also be expressed by means of projector operators for some subclasses of testable hypotheses. For these hypotheses, testing in Gauss-Markoff model is equivalent to testing in a Zyskind-Martin model.     

The rank transformation as a method of discrimination with some examples

The procedure of statistical discrimination Is simple in theory but so simple in practice. An observation x₀possibly uiultivariate, is to be classified into one of several populations π₁,…,π_k which have respectively, the density functions f₁(x), ? ? ? , f_k(x). The decision procedure is to evaluate each density function at X₀ to see which function gives the largest value f_i(X₀) , and then to declare that X₀ belongs to the population corresponding to the largest value. If these den-sities can be assumed to be normal with equal covariance matricesthen the decision procedure is known as Fisher’s linear discrimi-nant function (LDF) method. In the case of unequal covariance matrices the procedure is called the quadratic discriminant func-tion (QDF) method. If the densities cannot be assumed to be nor-mal then the LDF and QDF might not perform well. Several different procedures have appeared in the literature which offer discriminant procedures for nonnormal data. However, these pro-cedures are generally difficult to use and are not readily available as canned statistical programs.

Another approach to discriminant analysis is to use some sortof mathematical trans format ion on the samples so that their distribution function is approximately normal, and then use the convenient LDF and QDF methods. One transformation that:applies to all distributions equally well is the rank transformation. The result of this transformation is that a very simple and easy to use procedure is made available. This procedure is quite robust as is evidenced by comparisons of the rank transform results with several published simulation studies.     

Series Expansion for a Hypergeometric Function of Matrix Argument with Applications

A series expansion is obtained for the confluent hypergeometric function of the second kind when the argument is a 2 times 2 positive definite matrix. Applications are made to the distributions of Hotelling's generalized T₀² statistic, and the smallest latent root of the covariance matrix.     

Planning step-stress test under Type-I censoring for the exponential case

We consider in this work a k-level step-stress accelerated life-test (ALT) experiment with unequal duration steps τ=(τ₁, …, τ_k). Censoring is allowed only at the change-stress point in the final stage. An exponential failure time distribution with mean life that is a log-linear function of stress, along with a cumulative exposure model, is considered as the working model. The problem of choosing the optimal τ is addressed using the variance-optimality criterion. Under this setting, we then show that the optimal k-level step-stress ALT model with unequal duration steps reduces just to a 2-level step-stress ALT model.     

A general property among nested,pruned subtrees of a decision-support tree

Breiman, Friedman, Olshen, and Stone (1984) use a linear combination of prediction risk and tree size as a criterion in search of optimal trees. In this paper we use a linear combination of the above two components and the variable-observation cost as a criterion (C ₁) for the same purpose. This paper explicitly represents the relation among nested, pruned subtrees in terms of C ₁. Further, the theories in Breiman et al. (1984) concerning the search of optimal trees are generalized.     

Investigating the sensitivity of Gaussian processes to the choice of their correlation function and prior specifications

A Gaussian process (GP) can be thought of as an infinite collection of random variables with the property that any subset, say of dimension n, of these variables have a multivariate normal distribution of dimension n, mean vector β and covariance matrix Σ [O'Hagan, A., 1994, Kendall's Advanced Theory of Statistics, Vol. 2B, Bayesian Inference (John Wiley & Sons, Inc.)]. The elements of the covariance matrix are routinely specified through the multiplication of a common variance by a correlation function. It is important to use a correlation function that provides a valid covariance matrix (positive definite). Further, it is well known that the smoothness of a GP is directly related to the specification of its correlation function. Also, from a Bayesian point of view, a prior distribution must be assigned to the unknowns of the model. Therefore, when using a GP to model a phenomenon, the researcher faces two challenges: the need of specifying a correlation function and a prior distribution for its parameters. In the literature there are many classes of correlation functions which provide a valid covariance structure. Also, there are many suggestions of prior distributions to be used for the parameters involved in these functions. We aim to investigate how sensitive the GPs are to the (sometimes arbitrary) choices of their correlation functions. For this, we have simulated 25 sets of data each of size 64 over the square [0, 5]×[0, 5] with a specific correlation function and fixed values of the GP's parameters. We then fit different correlation structures to these data, with different prior specifications and check the performance of the adjusted models using different model comparison criteria.     

Effect of outliers on the GFI quality adjustment index in structural equation model and proposal of alternative indices

This work is intended to suggest modifications in the construction of the GFI index using robust methods for estimating the unrestricted sample covariance matrix, leading to new indices called GFI_(MCD) and GFI_(MVE). The validation of this proposal was made using Monte Carlo simulation methods, considering differences between the unrestricted sample covariance matrix and those imposed by the structural model, and different numbers of outliers generated by distributions with deviations from symmetry and excess kurtosis. It was concluded that for larger samples size (n ? 100), given that the outliers are from distributions that are symmetrical, the GFI_(MCD) and GFI_(MVE) present similar results, including samples with high percentages of outliers.