Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

Bootstrapping a time series model: some empirical results

The bootstrap is a methodology for estimating standard errors. The idea is to use a Monte Carlo simulation experiment based on a nonparametric estimate of the error distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order autoregressive model fitted by least squares and maximum likelihood estimation. Additionally, a comparison of the bootstrap and the conventional methodology is made. As it turns out, the conventional asymptotic formulae (both the least squares and maximum likelihood estimates) for estimating standard errors appear to overestimate the true standard errors. But there are two problems:i. The first two observations y₁ and y₂ have been fixed, and ii. The residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional maximum likelihood and bootstrap estimates of the standard errors appear to be performing quite well.     

Dealing with emrqes in variables in response surface exploration

This paper is concerned with the analysis of data obtained from a designed experiment where the experimental design cannot be implemented exactly as planned, because errors in the levels of the variables cannot be avoided or measured. When the primary interest of the investigator lies In obtaining a satisfactory response surface model for the investigated relationship, the precision of the model estimates is essential for successful model building and accurate prediction of the response. An iterative procedure is proposed which estimates the effect of the variable in errors and obtains efficient weighted least squares estimates of the parameters of Interest.     

Testing Covariate Effects in Aalen's Linear Hazard Model

The performance of tests in Aalen's linear regression model is studied using asymptotic power calculations and stochastic simulation. Aalen's original least squares test is compared to two modifications: a weighted least squares test with correct weights and a test where the variance is re-estimated under the null hypothesis. The test with re-estimated variance provides the highest power of the tests for the setting of this paper, and the gain is substantial for covariates following a skewed distribution like the exponential. It is further shown that Aalen's choice for weight function with re-estimated variance is optimal in the one-parameter case against proportional alternatives.     

Bootstrap methods for heteroskedastic regression models: evidence on estimation and testing

This paper uses Monte Carlo simulation analysis to study the finite-sample behavior of bootstrap estimators and tests in the linear heteroskedastic model. We consider four different bootstrapping schemes, three of them specifically tailored to handle heteroskedasticity. Our results show that weighted bootstrap methods can be successfully used to estimate the variances of the least squares estimators of the linear parameters both under normality and under nonnormality. Simulation results are also given comparing the size and power of the bootstrapped Breusch-Pagan test with that of the original test and of Bartlett and Edgeworth-corrected tests. The bootstrap test was found to be robust against unfavorable regression designs.     

Bootstrapping an Econometric Model: Some Empirical Results

The bootstrap, like the jackknife, is a technique for estimating standard errors. The idea is to use Monte Carlo simulation, based on a nonparametric estimate of the underlying error distribution. The bootstrap will be applied to an econometric model describing the demand for capital, labor, energy, and materials. The model is fitted by three-stage least squares. In sharp contrast with previous results, the coefficient estimates and the estimated standard errors perform very well. However, the model's forecasts show serious bias and large random errors, significantly understated by the conventional standard error of forecast.     

Parameter estimation of three-parameter Weibull distribution based on progressively Type-II censored samples

In this paper, the estimation of parameters for a three-parameter Weibull distribution based on progressively Type-II right censored sample is studied. Different estimation procedures for complete sample are generalized to the case with progressively censored data. These methods include the maximum likelihood estimators (MLEs), corrected MLEs, weighted MLEs, maximum product spacing estimators and least squares estimators. We also proposed the use of a censored estimation method with one-step bias-correction to obtain reliable initial estimates for iterative procedures. These methods are compared via a Monte Carlo simulation study in terms of their biases, root mean squared errors and their rates of obtaining reliable estimates. Recommendations are made from the simulation results and a numerical example is presented to illustrate all of the methods of inference developed here.     

Confirmatory Factor Analysis of Ordinal Data Using Full‐Information Adaptive Quadrature

We conducted confirmatory factor analysis (CFA) of responses (N=803) to a self‐reported measure of optimism, using full‐information estimation via adaptive quadrature (AQ), an alternative estimation method for ordinal data. We evaluated AQ results in terms of the number of iterations required to achieve convergence, model fit, parameter estimates, standard errors (SE), and statistical significance, across four link‐functions (logit, probit, log‐log, complimentary log‐log) using 3–10 and 20 quadrature points. We compared AQ results with those obtained using maximum likelihood, robust maximum likelihood, and robust diagonally weighted least‐squares estimation. Compared to the other two link‐functions, logit and probit not only produced fit statistics, parameters estimates, SEs, and levels of significance that varied less across numbers of quadrature points, but also fitted the data better and provided larger completely standardised loadings than did maximum likelihood and diagonally weighted least‐squares. Our findings demonstrate the viability of using full‐information AQ to estimate CFA models with real‐world ordinal data.     

Fourth Quarter Survey of the Economic Outlook

Written mainly for its pedagogical interest, this note deals with the computational formulas for the recursive updating of weighted least squares parameter estimates and the residual sum of squares in the general linear model under the assumption that the errors have a multivariate normal distribution. This approach simplifies considerably the derivations of Haslett (1985).     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Computational and statistical efficiency of semiparametric gls estimators

In this note, we report a dramatic improvement in the computational efficiency of semiparametric generalized least squares(SGLS) estimation. Computation of SGLS estimates no longer presents serious problems with data sets of moderate size. We also correct a numerical error in the standard errors of the SGLS estimates reported in our recent paper in this journal (Horowitz and Neumann, 1987). The corrected standard errors of SGLS are comparable to those we reported for quantile estimates.     

Computational and statistical efficiency of semiparametric gls estimators

In this note, we report a dramatic improvement in the computational efficiency of semiparametric generalized least squares(SGLS) estimation. Computation of SGLS estimates no longer presents serious problems with data sets of moderate size. We also correct a numerical error in the standard errors of the SGLS estimates reported in our recent paper in this journal (Horowitz and Neumann, 1987). The corrected standard errors of SGLS are comparable to those we reported for quantile estimates.     

Variable selection in spatial regression via penalized least squares

We consider variable selection in linear regression of geostatistical data that arise often in environmental and ecological studies. A penalized least squares procedure is studied for simultaneous variable selection and parameter estimation. Various penalty functions are considered including smoothly clipped absolute deviation. Asymptotic properties of penalized least squares estimates, particularly the oracle properties, are established, under suitable regularity conditions imposed on a random field model for the error process. Moreover, computationally feasible algorithms are proposed for estimating regression coefficients and their standard errors. Finite‐sample properties of the proposed methods are investigated in a simulation study and comparison is made among different penalty functions. The methods are illustrated by an ecological dataset of landcover in Wisconsin. The Canadian Journal of Statistics 37: 607–624; 2009 © 2009 Statistical Society of Canada     

Parametric Proportional Odds Frailty Models

We define a parametric proportional odds frailty model to describe lifetime data incorporating heterogeneity between individuals. An unobserved individual random effect, called frailty, acts multiplicatively on the odds of failure by time t. We investigate fitting by maximum likelihood and by least squares. For the latter, the parametric survivor function is fitted to the nonparametric Kaplan–Meier estimate at the observed failure times. Bootstrap standard errors and confidence intervals are obtained for the least squares estimates. The models are applied successfully to simulated data and to two real data sets. Least squares estimates appear to have smaller bias than maximum likelihood.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

An estimation in a linear model with response-related error

A maximum likelihood solution is presented for analyzing data which arise from a linear model whose error term is assumed to have variance proportional to some unknown power of the response. An efficient iterative method for solving the likelihood equations is obtained which incoporates use of a transfomation to orthogonalize the two variance paramaters. Assessments of the method are made through simulations study and the results are compared with those of the ordinary least squares. Examples from the literature are included to illustrate the method and also to compare the results with a weighted least squares estimates.     

A simple least squares method for estimating a change in mean

A simple least squares method for estimating a change in mean of a sequence of independent random variables is studied. The method first tests for a change in mean based on the regression principle of constrained and unconstrained sums of squares. Conditionally on a decision by this test that a change has occurred, least squares estimates are used to estimate the change point, the initial mean level (prior to the change point) and the change itself. The estimates of the initial level and change are functions of the change point estimate. All estimates are shown to be consistent, and those for the initial level and change are shown to be asymptotically jointly normal. The method performs well for moderately large shifts (one standard deviation or more), but the estimates of the initial level and change are biased in a predictable way for small shifts. The large sample theory is helpful in understanding this problem. The asymptotic distribution of the change point estimator is obtained for local shifts in mean, but the case of non-local shifts appears analytically intractable.     

A Multivariate Unit Root Test Based on the Modified Weighted Symmetric Estimator for VAR(p)

Multivariate unit root tests for the VAR model have been commonly used in time series analysis. Several unit root tests were developed. Most of the estimators of coefficient matrices developed in the VAR model are obtained using ordinary least squares estimators. In this paper, we suggest a multivariate unit root test based on a modified weighted symmetric estimator. Using a limited Monte Carlo simulation, we compare the powers of the new test statistic and the test statistic suggested in Fuller (1996).     

Effects of non-normality and mild heteroscedasticity on estimators in regression

Several estimators are examined for the simple linear regression model under a controlled, experimental situation with multiple observations at each design point. The model is examined under normal and non-normal error distributions and mild heterogeneity of variances across the chosen design points. We consider the ordinary, generalized, and estimated generalized least squares estimators and several examples of M estimators. The asymptotic properties of the M estimator using the Huber ψ are presented under these conditions for the multiple regression model. A simulation study is also presented which indicates that the M estimator possesses strong robustness properties under the presence of both non-normality and mild heteroscedasticity o£ errors. Finally, the M estimates are compared to the least squares estimates in two examples.