Some results on commutation matrices,with statistical applications

The commutation matrix P _mn changes the order of multiplication of a Kronecker matrix product. The vec operator stacks columns of a matrix one under another in a single column. It is possible to express the vec of a Kronecker matrix product in terms of a Kronecker product of vecs of matrices. The commutation matrix plays an important role here. “Super-vec-operators” like vec A ? vec A vec ( A ? A ), and vec{( A ? A ) P _nn} are very convenient. Several of their properties are being studied. Both the traditional commutation matrix and vec operator and the newer concepts developed from these are applied to multivariate statistical and related problems.     

PROPERTIES OF THE PATTERNED VECH OPERATOR

The vec operator arranges the columns of a matrix one below the other. When the matrix is symmetric such elements are not distinct but an extraction of only the distinct elements on or below the diagonal forms the operation denoted by vech. For other types of patterned matrices a ‘patterned vech’ operator is defined. The transformations from vech to vec are not uniquely defined. Here we examine properties of linear transformations which overcome the lack of uniqueness and develop properties of such linear transformations.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

On the relationship between the matrix operators,vech and vecd

We introduce a matrix operator, which we call “vecd” operator. This operator stacks up “diagonals” of a symmetric matrix. This operator is more convenient for some statistical analyses than the commonly used “vech” operator. We show an explicit relationship between the vecd and vech operators. Using this relationship, various properties of the vecd operator are derived. As applications of the vecd operator, we derive concise and explicit expressions of the Wald and score tests for equal variances of a multivariate normal distribution and for the diagonality of variance coefficient matrices in a multivariate generalized autoregressive conditional heteroscedastic (GARCH) model, respectively.     

Estimation of parameters in the singular gauss-markoff model

Consider the Gauss-Markoff model (Y, Xβ, σ² V) in the usual notation (Rao, 1973a, p. 294). If V is singular, there exists a matrix N such that N'Y has zero covariance. The minimum variance unbiased estimator of an estimable parametric function p'β is obtained in the wider class of (non-linear) unbiased estimators of the form f(N'Y) + Y'g(N'Y) where f is a scalar and g is a vector function.     

The exact noncentral distribution of the generalized multiple correlation matrix

Given p×n X N(βY, ∑?I), β, ∑ unknown, the noncentral multivariate beta density of the matrix L = [(YY′)^-1/2Y X′ (XX′)^-1XY′ (YY′)^-1/2] is desired. Khatri (1964) finds this density when β is of rank unity. The present paper derives the noncentral density of L and the density of the roots matrix of L for full rank β. The dual case density of L is also obtained. The derivations are based on generalized Sverdrup's lemma, Kabe (1965), and the relationship between primal and dual density of L is explicitly established.     

Secondary design considerations for minimum bias estimation

Several authors have suggested the method of minimum bias estimation for estimating response surfaces. The minimum bias estimation procedure achieves minimum average squared bias of the fitted model without depending on the values of the unknown parameters of the true surface. The only requirement is that the design satisfies a simple estimability condition. Subject to providing minimum average squared bias, the minimum bias estimator also provides minimum average variance of ?(x) where ?(x) is the estimate of the response at the point x.

To support the estimation of the parameters in the fitted model, very little has been suggested in the way of experimental designs except to say that a full rank matrix X of independent variables should be used. This paper presents a closer look at the estimability conditions that are required for minimum bias estimation, and from the form of the matrix X, a formula is derived which measures the amount of design flexibility available. The design flexibility is termed “the degrees of freedom” of the X matrix and it is shown how the degrees of freedom can be used to decide if other design optimality criteria might be considered along with minimum bias estimation. Several examples are provided.     

Least squares theory for possibly singular models

In a recent paper, Scobey (1975) observed that the usual least squares theory can be applied even when the covariance matrix σ²V of Y in the linear model Y = Xβ + e is singular by choosing the Moore-Penrose inverse (V+XX′)⁺ instead of V^-1 when V is nonsingular. This result appears to be wrong. The appropriate treatment of the problem in the singular case is described.     

Best quadratic and positive semidefinite unbiased estimation of the variance matrix of the multivariate normal distribution

A BQPUE (best quadratic and positive semidefinite unbiased estimator) of the matrix V for the distribution vec X^′∽N_np(vec M^′, U?V) is being given. It is unique, although still depending on U and M. When U = I and M = (μ,…,μ), we get a well-known (unique) result not depending on M.     

Some matrix results related to a partitioned singular linear model

Consider the linear model (y, Xβ V), where the model matrix X may not have a full column rank and V might be singular. In this paper we introduce a formula for the difference between the BLUES of Xβ under the full model and the model where one observation has been deleted. We also consider the partitioned linear regression model where the model matrix is (X₁: X₂) the corresponding vector of unknown parameters being (β′₁ : β′₂)′. We show that the BLUE of X₁ β₁ under a specific reduced model equals the corresponding BLUE under the original full model and consider some interesting consequences of this result.     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

A theorem on invariance of estimability of linear parametric functions in linear models

Consider the general linear model Y = Xβ + ? , where E[??'] = σ²I and rank of X is less than or equal to the number of columns of X. It is well known that the linear parametric function λ'β is estimable if and only if λ' is in the row space of X. This paper characterizes all orthogonal matrices P such that the row space of XP is equal to the row space of X, i.e. the estimability of λ'β is invariant under P. An additional property of these matrices is the invariance of the spectrum of the information matrix X'X. An application of the results is also given.     

Further results on the vecd operator and its applications

Abstract

In this paper, we consider the matrix vectorization operator termed the vecd operator, which has recently been introduced in the literature. This operator stacks up distinct elements of a symmetric matrix in a way that differs from that of the well-known vech operator; it stacks up not columns, but diagonals. We give further consideration to the vecd operator and related matrices, and derive their various useful properties. We provide some statistical applications of the vecd operator to illustrate its usefulness.     

A property of the yule distribution and its applications

The relationship Y = RX between two random variables X and Y, where R is distributed independently of X in (0, l), is known to have important consequences in different fields such as income distribution analysis, Inventory decision models, etc.

In this paper it is shown that when X and Y are discrete random variables, relationships of similar nature lead to Yule-type distributions. The implications of the results are studied in connection with problems of income underreporting and inventory decision making.     

Testing Hypotheses in the Functional Linear Model

The functional linear model with scalar response is a regression model where the predictor is a random function defined on some compact set of R and the response is scalar. The response is modelled as Y =Ψ( X )+ ɛ , where Ψ is some linear continuous operator defined on the space of square integrable functions and valued in R . The random input X is independent from the noise ɛ . In this paper, we are interested in testing the null hypothesis of no effect, that is, the nullity of Ψ restricted to the Hilbert space generated by the random variable X . We introduce two test statistics based on the norm of the empirical cross-covariance operator of ( X , Y ). The first test statistic relies on a χ ² approximation and we show the asymptotic normality of the second one under appropriate conditions on the covariance operator of X . The test procedures can be applied to check a given relationship between X and Y . The method is illustrated through a simulation study.     

Linear Regression Models under Conditional Independence Restrictions

Maximum likelihood estimation is investigated in the context of linear regression models under partial independence restrictions. These restrictions aim to assume a kind of completeness of a set of predictors Z in the sense that they are sufficient to explain the dependencies between an outcome Y and predictors X: ?(Y|Z, X) = ?(Y|Z), where ?(·|·) stands for the conditional distribution. From a practical point of view, the former model is particularly interesting in a double sampling scheme where Y and Z are measured together on a first sample and Z and X on a second separate sample. In that case, estimation procedures are close to those developed in the study of double‐regression by Engel & Walstra (1991) and Causeur & Dhorne (1998) . Properties of the estimators are derived in a small sample framework and in an asymptotic one, and the procedure is illustrated by an example from the food industry context.     

Optimality of blue's in a general linear model with incorrect design matrix

We consider the Gauss-Markoff model (Y,X₀β,σ²V) and provide solutions to the following problem: What is the class of all models (Y,Xβ,σ²V) such that a specific linear representation/some linear representation/every linear representation of the BLUE of every estimable parametric functional p'β under (Y,X₀β,σ²V) is (a) an unbiased estimator, (b) a BLUE, (c) a linear minimum bias estimator and (d) best linear minimum bias estimator of p'β under (Y,Xβ,σ²V)? We also analyse the above problems, when attention is restricted to a subclass of estimable parametric functionals.     

Singular gauss-markov models

For the general linear model Y = X$sZ + e in which e has a singular dispersion matrix $sG²A, $sG > 0, where A is n x n and singular, Mitra [2] considers the problem of testing F$sZ, where F is a known q x q matrix and claims that the sum of squares (SS) due to hypothesis is not distributed (as a x² variate with degrees of freedom (d. f.) equal to the rank of F) independent of the SS due to error, when a generalized inverse of A is chosen as (A + X'X)^–. This claim does not hold if a pseudo-inverse of A is taken to be (A + X'X)⁺ where A⁺ denotes the unique Moore-Penrose inverse (MPI) of A.     

Linearly admissible estimators on linear functions of regression coefficient under balanced loss function

Linearly admissible estimators on linear functions of regression coefficient are studied in a singular linear model and balanced loss when the design matrix has not full column rank. The sufficient and necessary conditions for linear estimators to be admissible are obtained respectively in homogeneous and inhomogeneous classes.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.