Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

Comparison of estimators for heteroskedastic models

Ordinary least squares (OLS) yield inefficient parameter estimates and inconsistent estimates of the covariance matrix in case of heteroskedastic errors. Robinson's adaptive estimator and the Cragg estimator avoid any explicit parameterization of heteroskedasticity, and reduce the danger of misspecification. A small Monte Carlo experiment is performed to compare the behavior of the adaptive estimator with the performance of the Cragg estimator. The Monte Carlo experiment includes simulations of the Generalized Least Squares (GLS) estimator. Indeed, an interesting question is how more sophisticated techniques, like the adaptive estimator, compare with GLS when the latter relies on an incorrect specification of the heteroskedastic process. It turns out that the regression parameters, when estimated adaptively, display small mean squared errors and great efficiency in case of medium or high heteroskedasticity. The covariance matrix, instead, is better estimated by the Cragg estimator or by GLS based on a misspecified error term, since the adaptive estimator overpredicts the standard errors of the regression parameters.     

Comparisons of the <Emphasis Type="Italic">r</Emphasis> − <Emphasis Type="Italic">k</Emphasis> class estimator to the ordinary least squares estimator under the Pitman’s closeness criterion

In the presence of multicollinearity, the r − k class estimator is proposed as an alternative to the ordinary least squares (OLS) estimator which is a general estimator including the ordinary ridge regression (ORR), the principal components regression (PCR) and the OLS estimators. Comparison of competing estimators of a parameter in the sense of mean square error (MSE) criterion is of central interest. An alternative criterion to the MSE criterion is the Pitman’s (1937) closeness (PC) criterion. In this paper, we compare the r − k class estimator to the OLS estimator in terms of PC criterion so that we can get the comparison of the ORR estimator to the OLS estimator under the PC criterion which was done by Mason et al. (1990) and also the comparison of the PCR estimator to the OLS estimator by means of the PC criterion which was done by Lin and Wei (2002).     

Estimating partially linear panel data models with one-way error components

We consider the problem of estimating a partially linear panel data model whenthe error follows an one-way error components structure. We propose a feasiblesemiparametric generalized least squares (GLS) type estimator for estimating the coefficient of the linear component and show that it is asymptotically more efficient than a semiparametric ordinary least squares (OLS) type estimator. We also discussed the case when the regressor of the parametric component is correlated with the error, and propose an instrumental variable GLS-type semiparametric estimator.     

Improving survey-weighted least squares regression

The weighted least squares (WLS) estimator is often employed in linear regression using complex survey data to deal with the bias in ordinary least squares (OLS) arising from informative sampling. In this paper a 'quasi-Aitken WLS' (QWLS) estimator is proposed. QWLS modifies WLS in the same way that Cragg's quasi-Aitken estimator modifies OLS. It weights by the usual inverse sample inclusion probability weights multiplied by a parameterized function of covariates, where the parameters are chosen to minimize a variance criterion. The resulting estimator is consistent for the superpopulation regression coefficient under fairly mild conditions and has a smaller asymptotic variance than WLS.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.     

Form and selection of covariance adjusted estimators in repeated measures models

The GMANOVA model is considered when one characteristic is measured at q time points. The covariance adjusted estimator is the OLS estimator adjusted using analysis of covariance. The covariates are obtained from the space vertical to that derived from the design matrix. Its form is obtained as a GLS estimator using a weight matrix of reduced rank unless all available covariates are used. The choice of the appropriate covariate combination is made by introducing a method based on cross validation (CV). A comparison is made with two other methods that appeared in statistical literature using simulation.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

The Joint Treatment of Exact and Stochastic Restrictions

In regression analysis both exact and stochastic extraneous information may be represented via restrictions on the parameters of a linear model which then may be estimated by applying constrained generalized least squares. It is shown that this estimator can be recast as a computationally simpler estimator that is a combination of the ordinary least squares estimator and the discrepancy between the OLS estimator and both types of restrictions. The variance of the restricted parameters is explicitly shown to depend on the variance of the extraneous information.     

Asymptotic normal tests for integration in panels with cross-dependent units

The asymptotically normal, regression-based LM integration test is adapted for panels with correlated units. The N different units may be integrated of different (fractional) orders under the null hypothesis. The paper first reviews conditions under which the test statistic is asymptotically (as T→∞) normal in a single unit. Then we adopt the framework of seemingly unrelated regression [SUR] for cross-correlated panels, and discuss a panel test statistic based on the feasible generalized least squares [GLS] estimator, which follows a χ ²(N) distribution. Third, a more powerful statistic is obtained by working under the assumption of equal deviations from the respective null in all units. Fourth, feasible GLS requires inversion of sample covariance matrices typically imposing T>N; in addition we discuss alternative covariance matrix estimators for T<N. The usefulness of our results is assessed in Monte Carlo experimentation.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

Three estimators for the poisson regression model with measurement errors

We consider two consistent estimators for the parameters of the linear predictor in the Poisson regression model, where the covariate is measured with errors. The measurement errors are assumed to be normally distributed with known error variance σ _u ² . The SQS estimator, based on a conditional mean-variance model, takes the distribution of the latent covariate into account, and this is here assumed to be a normal distribution. The CS estimator, based on a corrected score function, does not use the distribution of the latent covariate. Nevertheless, for small σ _u ² , both estimators have identical asymptotic covariance matrices up to the order of σ _u ² . We also compare the consistent estimators to the naive estimator, which is based on replacing the latent covariate with its (erroneously) measured counterpart. The naive estimator is biased, but has a smaller covariance matrix than the consistent estimators (at least up to the order of σ _u ² ).     

Conditional inference in finite population sampling under a calibration model

Several estimators, including the classical and the regression estimators of finite population mean, are compared, both theoretically and empirically, under a calibration model, where the dependent variable(y), and not the independent variable(x), can be observed for all units of the finite population. It is shown asymptotically that when conditioned on x, the bias of the classical estimator may be much smaller than that of the regression estimators; whereas when conditioned on y, the regression estimator may have much smaller conditional bias than the classical estimator. Since all the y's(not x's) can be observed, it seems appropriate to make comparison under the conditional distribution of each estimator with y fixed. In this case, the regression estimator has smaller variance, smaller conditional bias, and the conditional coverage probability closer to its nominal level     

Combining the unrestricted estimators into a single estimator and a simulation study on the unrestricted estimators

The purpose of this paper is to combine several regression estimators (ordinary least squares (OLS), ridge, contraction, principal components regression (PCR), Liu, r?k and r?d class estimators) into a single estimator. The conditions for the superiority of this new estimator over the PCR, the r?k class, the r?d class, β?(k, d), OLS, ridge, Liu and contraction estimators are derived by the scalar mean square error criterion and the estimators of the biasing parameters for this new estimator are examined. Also, a numerical example based on Hald data and a simulation study are used to illustrate the results.     

Adaptive estimators for parameters of the autoregression function of a Markov chain

Suppose we observe an ergodic Markov chain on the real line, with a parametric model for the autoregression function, i.e. the conditional mean of the transition distribution. If one specifies, in addition, a parametric model for the conditional variance, one can define a simple estimator for the parameter, the maximum quasi-likelihood estimator. It is robust against misspecification of the conditional variance, but not efficient. We construct an estimator which is adaptive in the sense that it is efficient if the conditional variance is misspecified, and asymptotically as good as the maximum quasi-likelihood estimator if the conditional variance is correctly specified. The adaptive estimator is a weighted nonlinear least-squares estimator, with weights given by predictors for the conditional variance.     

Consistency,asymptotic unbiasedness and bounds on the bias of s2 in the linear regression model with error component disturbances

The OLS estimator of the disturbance variance in the linear regression model with error component disturbances is shown to be weakly consistent and asymptotically unbiased without any restrictions on the regressor matrix. Also, simple exact bounds on the expected value of s² are given for both the one-way and two-way error component models.