On O.L.S. estimation in singular linear models with general positive definite dispersion matrix

A condition in Graybill [1976] for the O.L.S.estimator to be B.L.U.E. in a linear model with positive definite dispersion not necessarily proportional to the identity matrix is extended to cover the case of a singular linear model.     

Some matrix results related to a partitioned singular linear model

Consider the linear model (y, Xβ V), where the model matrix X may not have a full column rank and V might be singular. In this paper we introduce a formula for the difference between the BLUES of Xβ under the full model and the model where one observation has been deleted. We also consider the partitioned linear regression model where the model matrix is (X₁: X₂) the corresponding vector of unknown parameters being (β′₁ : β′₂)′. We show that the BLUE of X₁ β₁ under a specific reduced model equals the corresponding BLUE under the original full model and consider some interesting consequences of this result.     

Optimality of blue's in a general linear model with incorrect design matrix

We consider the Gauss-Markoff model (Y,X₀β,σ²V) and provide solutions to the following problem: What is the class of all models (Y,Xβ,σ²V) such that a specific linear representation/some linear representation/every linear representation of the BLUE of every estimable parametric functional p'β under (Y,X₀β,σ²V) is (a) an unbiased estimator, (b) a BLUE, (c) a linear minimum bias estimator and (d) best linear minimum bias estimator of p'β under (Y,Xβ,σ²V)? We also analyse the above problems, when attention is restricted to a subclass of estimable parametric functionals.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.     

Projectors and linear estimation in general linear models

The paper gives a self-contained account of minimum disper­sion linear unbiased estimation of the expectation vector in a linear model with the dispersion matrix belonging to some, rather arbitrary, set of nonnegative definite matrices. The approach to linear estimation in general linear models recommended here is a direct generalization of some ideas and results presented by Rao (1973, 19 74) for the case of a general Gauss-Markov model

A new insight into the nature of some estimation problems originaly arising in the context of a general Gauss-Markov model as well as the correspondence of results known in the literature to those obtained in the present paper for general linear models are also given. As preliminary results the theory of projectors defined by Rao (1973) is extended.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

Equality of two blues and ridge-type estimates

Equality is shown of the g-inverse and Moore-Penrose inverse representation of the BLUE in the general linear model. The proof is based on a matrix identity which allows also to establish a functional relationship between the BLUE and Ridge-type estimates.     

On the inadmssibelity of ridge estimator in a linear model

The necessary and sufficient conditions for the inadmissibility of the ridge regression is discussed under two different criteria, namely, average loss and Pitman nearness. Although the two criteria are very different, same conclusions are obtained. The loss functions considered in this article are th likelihood loss function and the Mahalanobis loss function. The two loss functions are motivated from the point of view of classification of two normal populations. Under the Mahalanobis loss it is demonstrated that the ridge regression is always inadmissible as long as the errors are assumed to be symmetrically distributed about the origin.     

Best minimum bias linear estimators in gauss-markoff model

In this note we present a criterion for linear estimation which is similar to MV-MB-LE of Rao (1978) in Gauss-Markoff model (Y, XB, α²G). We call this criterion MMS-MB-LE (Minimum Mean Square Error-Minimum Bias-Linear Estimation)> Representations of solutions to such estimators similar to those of Rao (1978) are provided.     

An estimation in a linear model with response-related error

A maximum likelihood solution is presented for analyzing data which arise from a linear model whose error term is assumed to have variance proportional to some unknown power of the response. An efficient iterative method for solving the likelihood equations is obtained which incoporates use of a transfomation to orthogonalize the two variance paramaters. Assessments of the method are made through simulations study and the results are compared with those of the ordinary least squares. Examples from the literature are included to illustrate the method and also to compare the results with a weighted least squares estimates.     

Representations of the covariance matrix for robustness in the gauss-markov model

This paper considers some extensions of the results of Rao and Rao and Mitra. They gave a table of general representations of the covariance matrix in terms of the given design matrix, under which various statistical procedures in the least squares theory based on the simple Gauss-Markov model with the spherical covariance matrix are also valid under the general Gauss-Markov model. We shall give extended tables adding some more results relating to robustness, especially in connection with the estimation and testing of hypotheses on linear parametric functions     

Residuals for the linear model with general covariance structure

A general theory is presented for residuals from the general linear model with correlated errors. It is demonstrated that there are two fundamental types of residual associated with this model, referred to here as the marginal and the conditional residual. These measure respectively the distance to the global aspects of the model as represented by the expected value and the local aspects as represented by the conditional expected value. These residuals may be multivariate. Some important dualities are developed which have simple implications for diagnostics. The results are illustrated by reference to model diagnostics in time series and in classical multivariate analysis with independent cases.     

A non-iterative approach to estimating parameters in a linear structural equation model

The research described herein was motivated by a study of the relationship between the performance of students in senior high schools and at universities in China. A special linear structural equation model is established, in which some parameters are known and both the responses and the covariables are measured with errors. To explore the relationship between the true responses and latent covariables and to estimate the parameters, we suggest a non-iterative estimation approach that can account for the external dependence between the true responses and latent covariables. This approach can also deal with the collinearity problem because the use of dimension-reduction techniques can remove redundant variables. Combining further with the information that some of parameters are given, we can perform estimation for the other unknown parameters. An easily implemented algorithm is provided. A simulation is carried out to provide evidence of the performance of the approach and to compare it with existing methods. The approach is applied to the education example for illustration, and it can be readily extended to more general models.     

Quadratic estimators of covariance components in a multivariate mixed linear model

It is known that the Henderson Method III (Biometrics 9:226–252, 1953) is of special interest for the mixed linear models because the estimators of the variance components are unaffected by the parameters of the fixed factor (or factors). This article deals with generalizations and minor extensions of the results obtained for the univariate linear models. A MANOVA mixed model is presented in a convenient form and the covariance components estimators are given on finite dimensional linear spaces. The results use both the usual parametric representations and the coordinate-free approach of Kruskal (Ann Math Statist 39:70–75, 1968) and Eaton (Ann Math Statist 41:528–538, 1970). The normal equations are generalized and it is given a necessary and sufficient condition for the existence of quadratic unbiased estimators for covariance components in the considered model.     

Mean square error matrix comparisons of estimators in linear regression

The aim of this paper is to provide criteria which allow to compare two estimators of the parameter vector in the linear regression model with respect to their mean square error matrices, where the main interest is focussed on the case when the difference of the covariance matrices is singular. The results obtained are applied to equality restricted and pretest estimators.     

Linear trimmed means for the linear regression with AR(1) errors model

For the linear regression with AR(1) errors model, the robust generalized and feasible generalized estimators of Lai et al. (2003) of regression parameters are shown to have the desired property of a robust Gauss Markov theorem. This is done by showing that these two estimators are the best among classes of linear trimmed means. Monte Carlo and data analysis for this technique have been performed.     

An error-free generalized matrix inversion and linear least squares method based on bordering

The paper develops a method from which algorithms can be constructed to numerically compute error-free (free from computer roundoff error) generalized inverses and solutions to linear least squares problems having rational entries. A multiple modulus system is used to avoid error accumulation that is inherent in the floating-point number system. Some properties of finite fields of characteristic p, GF(p), are used in conjunction with a bordering method for matrix inversion to find nonsingular minors of a matrix over the field of rational numbers.     

Multiple linear regression model with stochastic design variables

In a simple multiple linear regression model, the design variables have traditionally been assumed to be non-stochastic. In numerous real-life situations, however, they are stochastic and non-normal. Estimators of parameters applicable to such situations are developed. It is shown that these estimators are efficient and robust. A real-life example is given.     

Estimation in the simple linear regression model when there is heteroscedasticity of unknown form

A well-known problem is that ordinary least squares estimation of the parameters in the usual linear model can be highly ineficient when the error term has a heavy-tailed distribution. Inefficiency is also associated with situations where the error term is heteroscedastic, and standard confidence intervals can have probability coverage substantially different from the nominal level. This paper compares the small-sample efficiency of six methods that address this problem, three of which model the variance heterogeneity nonparametrically. Three methods were found to be relatively ineffective, but the other three perform relatively well. One of the six (M-regression with a Huber φ function and Schweppe weights) was found to have the highest efficiency for most of the situations considered in the simulations, but there might be situations where one of two other methods gives better results. One of these is a new method that uses a running interval smoother to estimate the optimal weights in weighted least squares, and the other is a method recently proposed by Cohen, Dalal, and Tukey. Computing a confidence interval for the slope using a bootstrap technique is also considered.