Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

Functional analysis of variance for Hilbert-valued multivariate fixed effect models

This paper presents new results on functional analysis of variance for fixed effect models with correlated Hilbert-valued Gaussian error components. The geometry of the reproducing kernel Hilbert space of the error term is considered in the computation of the total sum of squares, the residual sum of squares, and the sum of squares due to the regression. Under suitable linear transformation of the correlated functional data, the distributional characteristics of these statistics, their moment generating and characteristic functions, are derived. Fixed effect linear hypothesis testing is finally formulated in the Hilbert-valued multivariate Gaussian context considered.     

General linear estimators under the prediction error sum of squares criterion in a linear regression model

In this paper, the notion of the general linear estimator and its modified version are introduced using the singular value decomposition theorem in the linear regression model y=X β+e to improve some classical linear estimators. The optimal selections of the biasing parameters involved are theoretically given under the prediction error sum of squares criterion. A numerical example and a simulation study are finally conducted to illustrate the superiority of the proposed estimators.     

两个部分线性模型的比较

内容提要:对于两个部分线性模型参数部分中模型系数是否相等的检验问题,本文基于比较原假设与备择假设下模型拟合的残差平方和的思想构造了检验统计量,并给出了计算检验_^p* 值的F分布逼近法。     

Testing Covariate Effects in Aalen's Linear Hazard Model

The performance of tests in Aalen's linear regression model is studied using asymptotic power calculations and stochastic simulation. Aalen's original least squares test is compared to two modifications: a weighted least squares test with correct weights and a test where the variance is re-estimated under the null hypothesis. The test with re-estimated variance provides the highest power of the tests for the setting of this paper, and the gain is substantial for covariates following a skewed distribution like the exponential. It is further shown that Aalen's choice for weight function with re-estimated variance is optimal in the one-parameter case against proportional alternatives.     

Asymptotic distribution of least square estimators for linear models with dependent errors

In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan (Central limit theorems for time series regression. Probab Theory Relat Fields. 1973;26(2):157–170), who proved a Central Limit Theorem for the usual least squares estimator under general conditions on the design and on the error process. Whatever the design satisfying Hannan's conditions, we define an estimator of the covariance matrix and we prove its consistency under very mild conditions. As an application, we show how to modify the usual tests on the linear model in this dependent context, in such a way that the type-I error rate remains asymptotically correct, and we illustrate the performance of this procedure through different sets of simulations.     

Testing for the parametric component of partially linear EV models under random censorship

This paper investigates the hypothesis test of the parametric component in partially linear errors-in-variables (EV) model with random censorship. We construct two test statistics based on the difference of the corrected residual sum of squares and empirical likelihood ratio under the null and alternative hypotheses. It is shown that the limiting distributions of the proposed test statistics are both weighted sum of independent standard chi-squared distribution with one degree of freedom under the null hypothesis. Based on the adjusted test statistics, we further develop two new types of test procedures. Finite sample performance of the proposed test procedures is evaluated by extensive simulation studies.     

Fitting Time Series Models by Minimizing Multistep-ahead Errors: a Frequency Domain Approach

This paper brings together two topics in the estimation of time series forecasting models: the use of the multistep-ahead error sum of squares as a criterion to be minimized and frequency domain methods for carrying out this minimization. The methods are developed for the wide class of time series models having a spectrum which is linear in unknown coefficients. This includes the IMA(1, 1) model for which the common exponentially weigh-ted moving average predictor is optimal, besides more general structural models for series exhibiting trends and seasonality. The method is extended to include the Box–Jenkins `air line' model. The value of the multistep criterion is that it provides protection against using an incorrectly specified model. The value of frequency domain estimation is that the iteratively reweighted least squares scheme for fitting generalized linear models is readily extended to construct the parameter estimates and their standard errors. It also yields insight into the loss of efficiency when the model is correct and the robustness of the criterion against an incorrect model. A simple example is used to illustrate the method, and a real example demonstrates the extension to seasonal models. The discussion considers a diagnostic test statistic for indicating an incorrect model.     

Statistical inference on partial linear additive models with distortion measurement errors

We consider statistical inference for partial linear additive models (PLAMs) when the linear covariates are measured with errors and distorted by unknown functions of commonly observable confounding variables. A semiparametric profile least squares estimation procedure is proposed to estimate unknown parameter under unrestricted and restricted conditions. Asymptotic properties for the estimators are established. To test a hypothesis on the parametric components, a test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we further show that its limiting distribution is a weighted sum of independent standard chi-squared distributions. A bootstrap procedure is further proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure and a real example is analyzed for an illustration.     

A Generalized Diagonal Ridge-type Estimator in Linear Regression

This article introduces a general class of biased estimator, namely a generalized diagonal ridge-type (GDR) estimator, for the linear regression model when multicollinearity occurs. The estimator represents different kinds of biased estimators when different parameters are obtained. Some properties of this estimator are discussed and an iterative procedure is provided for selecting the parameters. A Monte Carlo simulation study and an application show that the GDR estimator performs much better than the ordinary least squares (OLS) estimator under the mean square error (MSE) criterion when severe multicollinearity is present.     

A new biased estimator based on ridge estimation

In this paper we introduce a new biased estimator for the vector of parameters in a linear regression model and discuss its properties. We show that our new biased estimator is superior, in the mean square error(mse) sense, to the ordinary least squares (OLS) estimator, the ordinary ridge regression (ORR) estimator and the Liu estimator. We also compare the performance of our new biased estimator with two other special Liu-type estimators proposed in Liu (2003). We illustrate our findings with a numerical example based on the widely analysed dataset on Portland cement.     

A CONSISTENT MODEL SPECIFICATION TEST BASED ON THE KERNEL SUM OF SQUARES OF RESIDUALS

This paper constructs a consistent model specification test based on the difference between the nonparametric kernel sum of squares of residuals and the sum of squares of residuals from a parametric null model. We establish the asymptotic normality of the proposed test statistic under the null hypothesis of correct parametric specification and show that the wild bootstrap method can be used to approximate the null distribution of the test statistic. Results from a small simulation study are reported to examine the finite sample performance of the proposed tests.     

Statistical inference for restricted partially linear varying coefficient errors-in-variables models

As a useful extension of partially linear models and varying coefficient models, the partially linear varying coefficient model is useful in statistical modelling. This paper considers statistical inference for the semiparametric model when the covariates in the linear part are measured with additive error and some additional linear restrictions on the parametric component are available. We propose a restricted modified profile least-squares estimator for the parametric component, and prove the asymptotic normality of the proposed estimator. To test hypotheses on the parametric component, we propose a test statistic based on the difference between the corrected residual sums of squares under the null and alterative hypotheses, and show that its limiting distribution is a weighted sum of independent chi-square distributions. We also develop an adjusted test statistic, which has an asymptotically standard chi-squared distribution. Some simulation studies are conducted to illustrate our approaches.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Stability of linear programming solutions using regression coefficients

In at least one important application of stochastic linear programming (Lavaca-Tres Palacios Estuary:A Study of the Influence of Freshwater Inflows, 1980)constraint parameters are simultaneously estimated using multiple regression with historic data for the values of the decision variables and the right hand side of the constraint function. In this circumstance, the question immediately arises "How stable is the linear programming (LP) solution with regard to regression issues such as sample size, magnitude of the error variance, centroids of the decision variables, apd collinearity?" This paper reports a simulation designed to assess the stability of the LP solution and to compare the effectiveness of ridge as an alternative to ordinary least squares (OLS) regression. For the given scenario, the LP solution is consistently "biased." The amount of bias is exacerbated by small samples, large error variances, and collinearity among observations of the decision variables. The best regression criterion is a function not only of collinearity, but also of the magnitude of the error variance and the sum of the means of the decision variables relative to the right hand side of the stochastic constraint

In the application that motivated this research, the LP solutions were recommended fresh water inflows from Lake Texana into the estuaries of the Gulf of Mexico. The stochastic constraint estimates commercial fish harvest as a function of seasonal fresh water inflow. The historic data set used to estimate parameters of the constraint comprised rainfall data and fish harvest data prior to the construction of the Lake Texana dam, of necessity a small sample with collinear seasonal rainfall. It is not the authors' intent to solve this application, but rather to investigate through a simpler simulated systemwhether or not regression estimates in similar circumstances might introduce a systematic and predictable bias. The answer to this latter question is a qualified Yes!.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

Bias and mean squared error of the slope estimator in a regression with not necessarily normal errors in both variables

The least squares estimation of the slope parameter of a simple linear regression is biased if the regressor variable is measured with random errors. This bias as well as the mean squared error is computed up to the order of 1/T without assuming normality for the error variable. They depend on the fourth moment of the error variable.     

Improved ridge estimators in a linear regression model

In this paper, the notion of the improved ridge estimator (IRE) is put forward in the linear regression model y=X β+e. The problem arises if augmenting the equation 0=c′α+ε instead of 0=C α+? to the model. Three special IREs are considered and studied under the mean-squared error criterion and the prediction error sum of squares criterion. The simulations demonstrate that the proposed estimators are effective and recommendable, especially when multicollinearity is severe.     

Estimating partially linear panel data models with one-way error components

We consider the problem of estimating a partially linear panel data model whenthe error follows an one-way error components structure. We propose a feasiblesemiparametric generalized least squares (GLS) type estimator for estimating the coefficient of the linear component and show that it is asymptotically more efficient than a semiparametric ordinary least squares (OLS) type estimator. We also discussed the case when the regressor of the parametric component is correlated with the error, and propose an instrumental variable GLS-type semiparametric estimator.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.