Distribution of the biased hypothesis sum of squares in linear models with missing observations

In a linear model with missing observations, one can substitute algebraic quantities and then minimize the error sum of squares for the augmented model. This gives the correct error sum of squares. But this method does not produce the correct hypothesis sum of squares for testing a linear hypothesis about the parameters. The sum of squares obtained is biased but practitioners still use it. The distribution of this biased sum of squares is derived in this paper and the consequences of using this biased sum of squares on the type I and II errors is examined.     

An algorithm for weighted bilinear regression

Bilinear models in which the expectation of a two-way array is the sum of products of parameters are widely used in spectroscopy. In this paper we present an algorithm called combined-vector successive overrelaxation (COV-SOR) for bilinear models, and compare it with methods like alternating least squares, singular value decomposition, and the Marquardt procedure. Comparisons are done for missing data also.     

Least squates estimation of missing values in time series

The minimum mean square error linear interpolator for missing values in time series is extended to handle any pattern of nonconsecutive observations. The paper then develops evidence with simple ARMA models that the usefulness of either the"nonparametric"or the parametric form of the least squares interpolator depends on the time series model, the arrangement of the missing data and the objective for completing the series.     

Classification trees aided mixed regression model

This paper introduces a novel hybrid regression method (MixReg) combining two linear regression methods, ordinary least square (OLS) and least squares ratio (LSR) regression. LSR regression is a method to find the regression coefficients minimizing the sum of squared error rate while OLS minimizes the sum of squared error itself. The goal of this study is to combine two methods in a way that the proposed method superior both OLS and LSR regression methods in terms of R² statistics and relative error rate. Applications of MixReg, on both simulated and real data, show that MixReg method outperforms both OLS and LSR regression.     

Estimation of missing data in analysis of covariance: A least-squares approach

Abstract

A method is proposed for the estimation of missing data in analysis of covariance models. This is based on obtaining an estimate of the missing observation that minimizes the error sum of squares. Specific derivation of this estimate is carried out for the one-factor analysis of covariance, and numerical examples are given to show the nature of the estimates produced. Parameter estimates of the imputed data are then compared with those of the incomplete data.     

Statistical inference on partial linear additive models with distortion measurement errors

We consider statistical inference for partial linear additive models (PLAMs) when the linear covariates are measured with errors and distorted by unknown functions of commonly observable confounding variables. A semiparametric profile least squares estimation procedure is proposed to estimate unknown parameter under unrestricted and restricted conditions. Asymptotic properties for the estimators are established. To test a hypothesis on the parametric components, a test statistic based on the difference between the residual sums of squares under the null and alternative hypotheses is proposed, and we further show that its limiting distribution is a weighted sum of independent standard chi-squared distributions. A bootstrap procedure is further proposed to calculate critical values. Simulation studies are conducted to demonstrate the performance of the proposed procedure and a real example is analyzed for an illustration.     

Stability of linear programming solutions using regression coefficients

In at least one important application of stochastic linear programming (Lavaca-Tres Palacios Estuary:A Study of the Influence of Freshwater Inflows, 1980)constraint parameters are simultaneously estimated using multiple regression with historic data for the values of the decision variables and the right hand side of the constraint function. In this circumstance, the question immediately arises "How stable is the linear programming (LP) solution with regard to regression issues such as sample size, magnitude of the error variance, centroids of the decision variables, apd collinearity?" This paper reports a simulation designed to assess the stability of the LP solution and to compare the effectiveness of ridge as an alternative to ordinary least squares (OLS) regression. For the given scenario, the LP solution is consistently "biased." The amount of bias is exacerbated by small samples, large error variances, and collinearity among observations of the decision variables. The best regression criterion is a function not only of collinearity, but also of the magnitude of the error variance and the sum of the means of the decision variables relative to the right hand side of the stochastic constraint

In the application that motivated this research, the LP solutions were recommended fresh water inflows from Lake Texana into the estuaries of the Gulf of Mexico. The stochastic constraint estimates commercial fish harvest as a function of seasonal fresh water inflow. The historic data set used to estimate parameters of the constraint comprised rainfall data and fish harvest data prior to the construction of the Lake Texana dam, of necessity a small sample with collinear seasonal rainfall. It is not the authors' intent to solve this application, but rather to investigate through a simpler simulated systemwhether or not regression estimates in similar circumstances might introduce a systematic and predictable bias. The answer to this latter question is a qualified Yes!.     

A study of simple unbalanced factorial designs that use type ii and type iii sums of squares

Methods for analysing unbalanced factorial designs can be traced back to the work of Frank Yates in the 1930s . Yet, still today the question on how his methods of fitting constants (Type II) and weighted squares of means (Type III) behave when negligible or insignificant interactions exist, is still unanswered. In this paper, by means of a simulation study, Type II and Type III ANOVA results are examined for all unbalanced structures originating from a 2x3 balanced factorial design within homogeneous groups (design types) accounting for structure, number of observations lost and which cells contained the missing observations. The two level factor is further analysed to test the null hypothesis, for both Type II and Type III analyses, that the unbalanced structures within each design type provide comparable F values. These results are summarised and the conclusion shows that this work agrees with statements made by Yates Burdick and Herr and Shaw and Mitchell-Olds, but there are some results which require further investigation.     

Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

Estimation and Prediction in the Presence of Spatial Confounding for Spatial Linear Models

In studies that produce data with spatial structure, it is common that covariates of interest vary spatially in addition to the error. Because of this, the error and covariate are often correlated. When this occurs, it is difficult to distinguish the covariate effect from residual spatial variation. In an i.i.d. normal error setting, it is well known that this type of correlation produces biased coefficient estimates, but predictions remain unbiased. In a spatial setting, recent studies have shown that coefficient estimates remain biased, but spatial prediction has not been addressed. The purpose of this paper is to provide a more detailed study of coefficient estimation from spatial models when covariate and error are correlated and then begin a formal study regarding spatial prediction. This is carried out by investigating properties of the generalized least squares estimator and the best linear unbiased predictor when a spatial random effect and a covariate are jointly modelled. Under this setup, we demonstrate that the mean squared prediction error is possibly reduced when covariate and error are correlated.     

Series Estimation in Partially Linear In‐Slide Regression Models

Abstract. The partially linear in‐slide model (PLIM) is a useful tool to make econometric analyses and to normalize microarray data. In this article, by using series approximations and a least squares procedure, we propose a semiparametric least squares estimator (SLSE) for the parametric component and a series estimator for the non‐parametric component. Under weaker conditions than those imposed in the literature, we show that the SLSE is asymptotically normal and that the series estimator attains the optimal convergence rate of non‐parametric regression. We also investigate the estimating problem of the error variance. In addition, we propose a wild block bootstrap‐based test for the form of the non‐parametric component. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on a set of economical data is also illustrated.     

Ridge and related estimation procedures: theory and practice

For the classical linear regression problem, a number of estimators alternative to least squares have been proposed for situations in which multicollinearity is a problem. There is, however, relatively little known about how these estimators behave in practice. This paper investigates mean square error properties for a number of biased regression estimators, and discusses some practical implications of the use of such estimators, A conclusion is that certain types of ridge estimatorsappear to have good mean square error properties, and this may be useful in situations in which mean square error is important     

Estimating expected sojourn times for a markov renewal process

We consider the estimation of the expected sojourn time in a Markov renewal process under the data condition that only the counts of the exits from the states are available for fixed intervals of time. For analytical and illustrative purposes we concentrate on the two-state process case. We present least squares and method of moments estimators and compare their statistical properties both analytically and empirically. We also present modified estimators with improved properties based upon an overlapping interval sampling strategy. The major results indicate that the least squares estimator is biased in general with the bias depending on the size of the sampling interval and the first two moments of the sojourn time distribution function. The bias becomes negligible as the size of the sampling interval increases. Analytical and empirical results indicate that the method of moments estimator is less sensitive to the size of the sampling interval and has slightly better mean squared error properties than the least squares estimator.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

Parameter estimation for a new distribution for the strength of brittle fibers: a simulation study

Parameter estimates of a new distribution for the strength of brittle fibers and composite materials are considered. An algorithm for generating random numbers from the distribution is suggested. Two parameter estimation methods, one based on a simple least squares procedure and the other based on the maximum likelihood principle, are studied using Monte Carlo simulation. In most cases, the maximum likelihood estimators were found to have somewhat smaller root mean squared error and bias than the least squares estimators. However, the least squares estimates are generally good and provide useful initial values for the numerical iteration used to find the maximum likelihood estimates.     

Missing plot technique in linear models

This paper extends the missing plot substitution technique to the case where the missing observations-cause some previously estimable functions to become non-estimable. It is shown that with appropriate modifications, the usual methods of analysis remain valid. We also obtain necessary and sufficient conditions under which the sum of squares due to a hypothesis can be calculated without “re-estimating” the missing observations     

The asymptotic behaviour of the residual sum of squares in models with multiple break points

ABSTRACT

Models with multiple discrete breaks in parameters are usually estimated via least squares. This paper, first, derives the asymptotic expectation of the residual sum of squares and shows that the number of estimated break points and the number of regression parameters affect the expectation differently. Second, we propose a statistic for testing the joint hypothesis that the breaks occur at specified points in the sample. Our analytical results cover models estimated by the ordinary, nonlinear, and two-stage least squares. An application to U.S. monetary policy rejects the assumption that breaks are associated with changes in the chair of the Fed.     

Estimation of a Change Point in a Hazard Function Based on Censored Data

The hazard function plays an important role in reliability or survival studies since it describes the instantaneous risk of failure of items at a time point, given that they have not failed before. In some real life applications, abrupt changes in the hazard function are observed due to overhauls, major operations or specific maintenance activities. In such situations it is of interest to detect the location where such a change occurs and estimate the size of the change. In this paper we consider the problem of estimating a single change point in a piecewise constant hazard function when the observed variables are subject to random censoring. We suggest an estimation procedure that is based on certain structural properties and on least squares ideas. A simulation study is carried out to compare the performance of this estimator with two estimators available in the literature: an estimator based on a functional of the Nelson-Aalen estimator and a maximum likelihood estimator. The proposed least squares estimator tums out to be less biased than the other two estimators, but has a larger variance. We illustrate the estimation method on some real data sets.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Estimating parameters in a simple errors-in-variables model: a new approach base on finite sample distribution theory

This article presents a first direct application of finite sample distribution theory. The relevance of analytical finite sample research is exemplified in the framework of a simple linear errors-in-variables model (EV Model) with known or approximately known measurement error variance. Analytical results derived byRichardson/Wu (1970) are applied for constructing new approximately unbiased estimators for the slope coefficient in the EV model. The new estimators are compared with the biased least squares estimator and with asymptotic theory based corrected least squares estimators. Retaining responsibility for remaining errors the author is indebted to Prof. H. Schneewei\ and Prof. J. Gruber for helpful comments and discussions. Mrs. A. Brandtstater deserves special mention and thanks for performing the computations reported in section 4.