Least Trimmed Squares Estimator in the Errors-in-Variables Model

We propose a robust estimator in the errors-in-variables model using the least trimmed squares estimator. We call this estimator the orthogonal least trimmed squares (OLTS) estimator. We show that the OLTS estimator has the high breakdown point and appropriate equivariance properties. We develop an algorithm for the OLTS estimate. Simulations are performed to compare the efficiencies of the OLTS estimates with the total least squares (TLS) estimates and a numerical example is given to illustrate the effectiveness of the estimate.     

Estimation Bias in the First-Order Autoregressive Model and Its Impact on Predictions and Prediction Intervals

The least squares estimate of the autoregressive coefficient in the AR(1) model is known to be biased towards zero, especially for parameters close to the stationarity boundary. Several methods for correcting the autoregressive parameter estimate for the bias have been suggested. Using simulations, we study the bias and the mean square error of the least squares estimate and the bias-corrections proposed by Kendall and Quenouille.

We also study the mean square forecast error and the coverage of the 95% prediction interval when using the biased least squares estimate or one of its bias-corrected versions. We find that the estimation bias matters little for point forecasts, but that it affects the coverage of the prediction intervals. Prediction intervals for forecasts more than one step ahead, when calculated with the biased least squares estimate, are too narrow.     

Linear least squares estimates and nonlinear means

The consistency and asymptotic normality of a linear least squares estimate of the form (X'X)^-X'Y when the mean is not Xβ is investigated in this paper. The least squares estimate is a consistent estimate of the best linear approximation of the true mean function for the design chosen. The asymptotic normality of the least squares estimate depends on the design and the asymptotic mean may not be the best linear approximation of the true mean function. Choices of designs which allow large sample inferences to be made about the best linear approximation of the true mean function are discussed.     

Consistent parametric estimation of the intensity of a spatial–temporal point process

We consider conditions under which parametric estimates of the intensity of a spatial–temporal point process are consistent. Although the actual point process being estimated may not be Poisson, an estimate involving maximizing a function that corresponds exactly to the log-likelihood if the process is Poisson is consistent under certain simple conditions. A second estimate based on weighted least squares is also shown to be consistent under quite similar assumptions. The conditions for consistency are simple and easily verified, and examples are provided to illustrate the extent to which consistent estimation may be achieved. An important special case is when the point processes being estimated are in fact Poisson, though other important examples are explored as well.     

Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

Parameter estimation for a new distribution for the strength of brittle fibers: a simulation study

Parameter estimates of a new distribution for the strength of brittle fibers and composite materials are considered. An algorithm for generating random numbers from the distribution is suggested. Two parameter estimation methods, one based on a simple least squares procedure and the other based on the maximum likelihood principle, are studied using Monte Carlo simulation. In most cases, the maximum likelihood estimators were found to have somewhat smaller root mean squared error and bias than the least squares estimators. However, the least squares estimates are generally good and provide useful initial values for the numerical iteration used to find the maximum likelihood estimates.     

On Sensitivity of Inverse Response Plot Estimation and the Benefits of a Robust Estimation Approach

Abstract. Inverse response plots are a useful tool in determining a response transformation function for response linearization in regression. Under some mild conditions it is possible to seek such transformations by plotting ordinary least squares fits versus the responses. A common approach is then to use nonlinear least squares to estimate a transformation by modelling the fits on the transformed response where the transformation function depends on an unknown parameter to be estimated. We provide insight into this approach by considering sensitivity of the estimation via the influence function. For example, estimation is insensitive to the method chosen to estimate the fits in the initial step. Additionally, the inverse response plot does not provide direct information on how well the transformation parameter is being estimated and poor inverse response plots may still result in good estimates. We also introduce a simple robustified process that can vastly improve estimation.     

Pricing of American options in discrete time using least squares estimates with complexity penalties

Pricing of American options in discrete time is considered, where the option is allowed to be based on several underlying stocks. It is assumed that the price processes of the underlying stocks are given by Markov processes. We use the Monte Carlo approach to generate artificial sample paths of these price processes, and then we use nonparametric regression estimates to estimate from this data so-called continuation values, which are defined as mean values of the American option for given values of the underlying stocks at time t subject to the constraint that the option is not exercised at time t. As nonparametric regression estimates we use least squares estimates with complexity penalties, which include as special cases least squares spline estimates, least squares neural networks, smoothing splines and orthogonal series estimates. General results concerning rate of convergence are presented and applied to derive results for the special cases mentioned above. Furthermore the pricing of American options is illustrated by simulated data.     

AN ESTIMATE OF THE ASYMPTOTIC STANDARD ERROR OF THE SAMPLE MEDIAN

A method for estimating the asymptotic standard error of the sample median based on generalized least squares is outlined. The practical problems of implementing this new estimate along with those associated with two existing estimates are discussed. Finally a simulation study is presented to compare the three estimates.     

A statistical uncertainty principle for estimating the time of a discrete shift in the mean of a continuous time random process

The purpose of this article is to present a statistical uncertainty principle that can be used when localizing a single change in the mean of a band-limited stationary random process. The statistical model investigated is a continuous time process that experiences a shift in its mean. This continuous time process is presumed to be sampled using an ideal low-pass filter. The least squares estimate of the location of the change in mean is asymptotically Gaussian. The standard deviation of the least squares estimate of the location of the change-point provides a physical limit to the accuracy of the estimate of the time of the mean shift which cannot be bettered.     

Estimating a Convex Function in Nonparametric Regression

Abstract.  A new nonparametric estimate of a convex regression function is proposed and its stochastic properties are studied. The method starts with an unconstrained estimate of the derivative of the regression function, which is firstly isotonized and then integrated. We prove asymptotic normality of the new estimate and show that it is first order asymptotically equivalent to the initial unconstrained estimate if the regression function is in fact convex. If convexity is not present, the method estimates a convex function whose derivative has the same L ^p -norm as the derivative of the (non-convex) underlying regression function. The finite sample properties of the new estimate are investigated by means of a simulation study and it is compared with a least squares approach of convex estimation. The application of the new method is demonstrated in two data examples.     

Estimation of a Change Point in a Hazard Function Based on Censored Data

The hazard function plays an important role in reliability or survival studies since it describes the instantaneous risk of failure of items at a time point, given that they have not failed before. In some real life applications, abrupt changes in the hazard function are observed due to overhauls, major operations or specific maintenance activities. In such situations it is of interest to detect the location where such a change occurs and estimate the size of the change. In this paper we consider the problem of estimating a single change point in a piecewise constant hazard function when the observed variables are subject to random censoring. We suggest an estimation procedure that is based on certain structural properties and on least squares ideas. A simulation study is carried out to compare the performance of this estimator with two estimators available in the literature: an estimator based on a functional of the Nelson-Aalen estimator and a maximum likelihood estimator. The proposed least squares estimator tums out to be less biased than the other two estimators, but has a larger variance. We illustrate the estimation method on some real data sets.     

EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

Efficiency of an adaptive estimator of a log-zero-poisson parameter

An estimator, λ is proposed for the parameter λ of the log-zero-Poisson distribution. While it is not a consistent estimator of λ in the usual statistical sense, it is shown to be quite close to the maximum likelihood estimates for many of the 35 sets of data on which it is tried. Since obtaining maximum likelihood estimates is extremely difficult for this and other contagious distributions, this estimate will act at least as an initial estimate in solving the likelihood equations iteratively. A lesson learned from this experience is that in the area of contagious distributions, variability is so large that attention should be focused directly on the mean squared error and not on consistency or unbiasedness, whether for small samples or for the asymptotic case. Sample sizes for some of the data considered in the paper are in hundreds. The fact that the estimator which is not a consistent estimator of λ is closer to the maximum likeli-hood estimator than the consistent moment estimator shows that the variability is large enough to not permit consistency to materialize even for such large sample sizes usually available in actual practice.     

One-step GM-estimators for orthogonal refression

This paper presents a one–step robust generalised M-estimation for orthogonal regression. The GM-estimator uses Schweppe weights which are based on high breakdown initial and scale estimates to downweight outliers and high leverage points. The one-step iteratively reweighted least squares procedure was used to compute the GM estimates. The robustness of the GM-estimator was shown from the results illustrated on measurements of concrete compressive strengths data.     

Robustness of least distances estimate in ultivariate linear models

In this paper, the robustness of the least distances (LD) estimate in multivariate linear models, as defined by Bai, Chen, Miao and Rao (1990), is discussed in terms of the influence function as well as the breakdown point. The LD estimate is shown to be more robust than the least squares (LS) estimate. The robustness of the LD is similar to that of the least absolute deviations (LAD) estimate, a well studied robust estimate in the univariate case. In particular, if there are no outliers in the design matrices, the breakdown point of the LD estimate reaches the highest value, 1/2.     

A Balanced System of U.S. Industry Accounts and Distribution of the Aggregate Statistical Discrepancy by Industry

This article describes and illustrates a generalized least squares (GLS) method that systematically incorporates all available information on the reliability of initial data in the reconciliation of a large disaggregated system of national accounts. The GLS method is applied to reconciling the 1997 U.S. Input-Output and Gross Domestic Product (GDP)-by-industry accounts with benchmarked GDP estimated from expenditures. The GLS procedure produced a balanced system of industry accounts and distributed the aggregate statistical discrepancy by industry according to the estimated relative reliabilities of initial estimates. The study demonstrates the empirical feasibility and computational efficiency of the GLS method for large accounts reconciliation.     

Estimation of a linear model under microaggregation by individual ranking

Summary Microaggregation by individual ranking is one of themost commonly applied disclosure control techniques for continuous microdata. The paper studies the effect of microaggregation by individual ranking on the least squares estimation of a multiple linear regression model. It is shown that the traditional least squares estimates are asymptotically unbiased. Moreover, the least squares estimates asymptotically have the same variances as the least squares estimates based on the original (non-aggregated) data. Thus, asymptotically, microaggregation by individual ranking does not result in a loss of efficiency in the least squares estimation of a multiple linear regression model. I thank Hans Schneeweiss for very helpful discussions and comments. Financial support from the Deutsche Forschungsgemeinschaft (German Science Foundation) is gratefully acknowledged.     

A comparison of robust estimators based on two types of trimming

The least trimmed squares (LTS) estimator and the trimmed mean (TM) are two well-known trimming-based estimators of the location parameter. Both estimates are used in practice, and they are implemented in standard statistical software (e.g., S-PLUS, R, Matlab, SAS). The breakdown point of each of these estimators increases as the trimming proportion increases, while the efficiency decreases. Here we have shown that for a wide range of distributions with exponential and polynomial tails, TM is asymptotically more efficient than LTS as an estimator of the location parameter, when they have equal breakdown points.     

Applying Least Absolute Deviation Regression to Regression-type Estimation of the Index of a Stable Distribution Using the Characteristic Function

Least absolute deviation regression is applied using a fixed number of points for all values of the index to estimate the index and scale parameter of the stable distribution using regression methods based on the empirical characteristic function. The recognized fixed number of points estimation procedure uses ten points in the interval zero to one, and least squares estimation. It is shown that using the more robust least absolute regression based on iteratively re-weighted least squares outperforms the least squares procedure with respect to bias and also mean square error in smaller samples.