Vec and vech operators for matrices,with some uses in jacobians and multivariate statistics

The vec of a matrix X stacks columns of X one under another in a single column; the vech of a square matrix X does the same thing but starting each column at its diagonal element. The Jacobian of a one-to-one transformation X → Y is then ∣∣?(vecX)/?(vecY) ∣∣ when X and Y each have functionally independent elements; it is ∣∣ ?(vechX)/?(vechY) ∣∣ when X and Y are symmetric; and there is a general form for when X and Y are other patterned matrices. Kronecker product properties of vec(ABC) permit easy evaluation of this determinant in many cases. The vec and vech operators are also very convenient in developing results in multivariate statistics.     

Further results on the vecd operator and its applications

Abstract

In this paper, we consider the matrix vectorization operator termed the vecd operator, which has recently been introduced in the literature. This operator stacks up distinct elements of a symmetric matrix in a way that differs from that of the well-known vech operator; it stacks up not columns, but diagonals. We give further consideration to the vecd operator and related matrices, and derive their various useful properties. We provide some statistical applications of the vecd operator to illustrate its usefulness.     

On the relationship between the matrix operators,vech and vecd

We introduce a matrix operator, which we call “vecd” operator. This operator stacks up “diagonals” of a symmetric matrix. This operator is more convenient for some statistical analyses than the commonly used “vech” operator. We show an explicit relationship between the vecd and vech operators. Using this relationship, various properties of the vecd operator are derived. As applications of the vecd operator, we derive concise and explicit expressions of the Wald and score tests for equal variances of a multivariate normal distribution and for the diagonality of variance coefficient matrices in a multivariate generalized autoregressive conditional heteroscedastic (GARCH) model, respectively.     

Some results on commutation matrices,with statistical applications

The commutation matrix P _mn changes the order of multiplication of a Kronecker matrix product. The vec operator stacks columns of a matrix one under another in a single column. It is possible to express the vec of a Kronecker matrix product in terms of a Kronecker product of vecs of matrices. The commutation matrix plays an important role here. “Super-vec-operators” like vec A ? vec A vec ( A ? A ), and vec{( A ? A ) P _nn} are very convenient. Several of their properties are being studied. Both the traditional commutation matrix and vec operator and the newer concepts developed from these are applied to multivariate statistical and related problems.     

Patterned matrix derivatives

A general procedure for obtaining matrix derivatives of functions of nonlinear patterned matrices is proposed. The method is extended to obtain the Jacobians of patterned matrix transformations. Nel (1980) and Wiens (1985) consider the linear patterned cases. The procedure proposed here takes care of these cases as well.     

Dispersion-diminishing transformations

Let D(σ) consist of matrices congruent to and dominated by a given matrix σ , and let T(σ) be the corresponding congruent transformations. These classes are characterized and their properties studied when σ is positive definite. Dispersion orderings are considered, including dispersion-diminishing linear transformations, concentration properties of which are shown. Arbitrary linear transformations are decomposed into contractions, isometries and dilations on subspaces relative to Mahalanobis norms. Applications are noted in statistical process control and linear inference     

Biomedical applications for a generalized linear functional Poisson model

Many medical and biological studies involve response in the form of Poisson counts which can bemodelled using explanatory variables which also arise from count data. If the explanatory variables are observable without error (also as Poisson counts) we have a generalized linear model with a logarithmic link function and Poisson error structure. If,however, some of the explanatory variables are not directly observable, but arise with superimposed errors (again of Poisson form), the model is of a new type:a generalised linear functional Poisson model. In this paper,maximum likelihood estimates of the parameters of this model are determined along with the information matrix which (on noting its particular patterned form) is amenable to inversion in explicit form. Methods are proposed of an iterative type for computing estimates of the parameters and of their variational properties (e.g. standard errors) for this model, which also has application in other fields such as road traffic studies.     

On the maximal invariance of manova step down procedure statistics

Subbaiah and Mudhol kar (1978) remark the general mu1tivariate linear hypothesis testing step down procedure statistics do not appear to be maximal invariants under nonsingular lower triangular matrix transformations of the original variates. This paper proves the maximal invariance of these statistics. The invariance results are essential to study the power functions of the step down procedures for MANOVA problems. An example is given to show that such power function studies are very involved.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

Linear Transformations of Linear Mixed-Effects Models

A number of articles have discussed the way lower order polynomial and interaction terms should be handled in linear regression models. Only if all lower order terms are included in the model will the regression model be invariant with respect to coding transformations of the variables. If lower order terms are omitted, the regression model will not be well formulated. In this paper, we extend this work to examine the implications of the ordering of variables in the linear mixed-effects model. We demonstrate how linear transformations of the variables affect the model and tests of significance of fixed effects in the model. We show how the transformations modify the random effects in the model, as well as their covariance matrix and the value of the restricted log-likelihood. We suggest a variable selection strategy for the linear mixed-effects model.     

Estimation of patterned covariance in the multivariate linear models: an outer product least-squares approach

In this article, we present a framework of estimating patterned covariance of interest in the multivariate linear models. The main idea in it is to estimate a patterned covariance by minimizing a trace distance function between outer product of residuals and its expected value. The proposed framework can provide us explicit estimators, called outer product least-squares estimators, for parameters in the patterned covariance of the multivariate linear model without or with restrictions on regression coefficients. The outer product least-squares estimators enjoy the desired properties in finite and large samples, including unbiasedness, invariance, consistency and asymptotic normality. We still apply the framework to three special situations where their patterned covariances are the uniform correlation, a generalized uniform correlation and a general q-dependence structure, respectively. Simulation studies for three special cases illustrate that the proposed method is a competent alternative of the maximum likelihood method in finite size samples.     

Information attainable in some randomly incomplete multivariate response models

In a general parametric setup, a multivariate regression model is considered when responses may be missing at random while the explanatory variables and covariates are completely observed. Asymptotic optimality properties of maximum likelihood estimators for such models are linked to the Fisher information matrix for the parameters. It is shown that the information matrix is well defined for the missing-at-random model and that it plays the same role as in the complete-data linear models. Applications of the methodologic developments in hypothesis-testing problems, without any imputation of missing data, are illustrated. Some simulation results comparing the proposed method with Rubin's multiple imputation method are presented.     

Linear signed rank tests for hultivariate location

Previously proposed linear signed rank tests for multivariate location are not invariant under linear transformations of the observations, The asymptotic relative efficiencies of the tests 2 with respect to Hotelling's T²test depend on the direction of shift and the covariance matrix of the alternative distributions. For distributions with highly correlated components, the efficiencies of some of these tests can be arbitrarily low; they approach zero for certain multivariate normal alternatives, This article proposes a transformation of the data to be performed prior to standard linear signed rank tests, The resulting procedures have attractive power and efficiency properties compared to the original tests, In particular, for elliptically symmetric contiguous alternafives, the efficiencies of the new tests equal those of corresponding univariate linear signed rank tests with respect to the t test.     

Linear Transformations and the k-Means Clustering Algorithm: Applications to Clustering Curves

Functional data can be clustered by plugging estimated regression coefficients from individual curves into the k-means algorithm. Clustering results can differ depending on how the curves are fit to the data. Estimating curves using different sets of basis functions corresponds to different linear transformations of the data. k-means clustering is not invariant to linear transformations of the data. The optimal linear transformation for clustering will stretch the distribution so that the primary direction of variability aligns with actual differences in the clusters. It is shown that clustering the raw data will often give results similar to clustering regression coefficients obtained using an orthogonal design matrix. Clustering functional data using an L(2) metric on function space can be achieved by clustering a suitable linear transformation of the regression coefficients. An example where depressed individuals are treated with an antidepressant is used for illustration.     

A simple derivation of the distribution of the multiple correlation coefficient

It is shown that the non-null distribution of the multiple correlation coefficient may be derived rather easily if the correlated normal variables are defined in a convenient vay. The invariance of the correlation distribution to linear transformations of the variables makes the results generally applicable. The distribution is derived as the well-known mixture of null distributions, and some generalizations when the variables are not normally distributed are indicated.     

Some properties of,and relationships among,several uncorrelated and homoscedastic residual vectors

In this paper we examine the properties of four types of residual vectors, arising from fitting a linear regression model to a set of data by least squares. The four types of residuals are (i) the Stepwise residuals (Hedayat and Robson, 1970), (ii) the Recursive residuals (Brown, Durbin, and Evans, 1975), (iii) the Sequentially Adjusted residuals (to be defined herein), and (iv) the BLUS residuals (Theil, 1965, 1971). We also study the relationships among the four residual vectors. It is found that, for any given sequence of observations, (i) the first three sets of residuals are identical, (ii) each of the first three sets, being identical, is a member of Thei’rs (1965, 1971) family of residuals; specifically, they are Linear Unbiased with a Scalar covariance matrix (LUS) but not Best Linear Unbiased with a Scalar covariance matrix (BLUS). We find the explicit form of the transformation matrix and show that the first three sets of residual vectors can be written as an orthogonal transformation of the BLUS residual vector. These and other properties may prove to be useful in the statistical analysis of residuals.     

Best quadratic and positive semidefinite unbiased estimation of the variance matrix of the multivariate normal distribution

A BQPUE (best quadratic and positive semidefinite unbiased estimator) of the matrix V for the distribution vec X^′∽N_np(vec M^′, U?V) is being given. It is unique, although still depending on U and M. When U = I and M = (μ,…,μ), we get a well-known (unique) result not depending on M.     

Simple principal components

We introduce an algorithm for producing simple approximate principal components directly from a variance–covariance matrix. At the heart of the algorithm is a series of 'simplicity preserving' linear transformations. Each transformation seeks a direction within a two-dimensional subspace that has maximum variance. However, the choice of directions is limited so that the direction can be represented by a vector of integers whenever the subspace can also be represented by vector if integers. The resulting approximate components can therefore always be represented by integers. Furthermore the elements of these integer vectors are often small, particularly for the first few components. We demonstrate the performance of this algorithm on two data sets and show that good approximations to the principal components that are also clearly simple and interpretable can result.     

Joint response graphs and separation induced by triangular systems

Summary.  We consider joint probability distributions generated recursively in terms of univariate conditional distributions satisfying conditional independence restrictions. The independences are captured by missing edges in a directed graph. A matrix form of such a graph, called the generating edge matrix, is triangular so the distributions that are generated over such graphs are called triangular systems. We study consequences of triangular systems after grouping or reordering of the variables for analyses as chain graph models, i.e. for alternative recursive factorizations of the given density using joint conditional distributions. For this we introduce families of linear triangular equations which do not require assumptions of distributional form. The strength of the associations that are implied by such linear families for chain graph models is derived. The edge matrices of chain graphs that are implied by any triangular system are obtained by appropriately transforming the generating edge matrix. It is shown how induced independences and dependences can be studied by graphs, by edge matrix calculations and via the properties of densities. Some ways of using the results are illustrated.     

Asymptotic expansions of moments and cumulants

This paper shows how procedures for computing moments and cumulants may themselves be computed from a few elementary identities.Many parameters, such as variance, may be expressed or approximated as linear combinations of products of expectations. The estimates of such parameters may be expressed as the same linear combinations of products of averages. The moments and cumulants of such estimates may be computed in a straightforward way if the terms of the estimates, moments and cumulants are represented as lists and the expectation operation defined as a transformation of lists. Vector space considerations lead to a unique representation of terms and hence to a simplification of results. Basic identities relating variables and their expectations induce transformations of lists, which transformations may be computed from the identities. In this way procedures for complex calculations are computed from basic identities.The procedures permit the calculation of results which would otherwise involve complementary set partitions, k-statistics, and pattern functions. The examples include the calculation of unbiased estimates of cumulants, of cumulants of these, and of moments of bootstrap estimates.