Statistical Analysis of Non Linear Least Squares Estimation for Harmonic Signals in Multiplicative and Additive Noise

Abstract

This article concerns the stochastically constrained linear model under a biased assumption. We propose a quasi-stochastically constrained least squares estimator. Furthermore, we provide the expectation of this estimator, demonstrate its consistency and asymptotic normality. In the end of the article, the simulation study of the new estimator shows that it is superior to the least squares estimator, ridge estimator, and the linear constrained estimators under certain conditions by comparing the mean squared errors of these estimators.     

Regression with an infinite number of observations applied to estimating the parameters of the stable distribution using the empirical characteristic function

ABSTRACT

The parameters of stable law parameters can be estimated using a regression based approach involving the empirical characteristic function. One approach is to use a fixed number of points for all parameters of the distribution to estimate the characteristic function. In this work the results are derived where all points in an interval is used to estimate the empirical characteristic function, thus least squares estimators of a linear function of the parameters, using an infinite number of observations. It was found that the procedure performs very good in small samples.     

Properties of Designs in Experiments on Spatial Lattices

ABSTRACT

When spatial variation is present in experiments, it is clearly sensible to use designs with favorable properties under both generalized and ordinary least squares. This will make the statistical analysis more robust to misspecification of the spatial model than would be the case if designs were based solely on generalized least squares. In this article, treatment information is introduced as a way of studying the ordinary least squares properties of designs. The treatment information is separated into orthogonal frequency or polynomial components which are assumed to be independent under the spatial model. The well-known trend-resistant designs are those with no treatment information at the very low order frequency or polynomial components which tend to have the higher variances under the spatial model. Ideally, designs would be chosen with all the treatment information distributed at the higher-order components. However, the results in this article show that there are limits on how much trend resistance can be achieved as there are many constraints on the treatment information. In addition, appropriately chosen Williams squares designs are shown to have favorable properties under both ordinary and generalized least squares. At all times, the ordinary least squares properties of the designs are balanced against the generalized least squares objectives of optimizing neighbor balance.     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.

     

An algorithm for weighted bilinear regression

Bilinear models in which the expectation of a two-way array is the sum of products of parameters are widely used in spectroscopy. In this paper we present an algorithm called combined-vector successive overrelaxation (COV-SOR) for bilinear models, and compare it with methods like alternating least squares, singular value decomposition, and the Marquardt procedure. Comparisons are done for missing data also.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

The partial least squares-fix point method of estimating interdependent systems with latent variables

This paper describes a method for estimating the unknown parameters of an interdependent simultaneous equations model with latent variables. For each latent variable there may be single or multiple indicators. Estimation proceeds in three stages: first, estimates of the latent variables are constructed from the associated manifest indicators; second, treating the estimates as directly observed, fix-point estimates of the structural form parameters are obtained; third, the location parameters are estimated. The method involves only repeated application of ordinary least squares and no distributional assumptions are needed. The paper concludes with an empirical application of the method.     

Multiyear,through the season crop acreage estimation using estimated acreage in sample segments

Large scale crop surveys can be made frequently and inex¬pensively during a crop growing season using Landsat data. A crop's at-harvest acreage in a stratum can be estimated from the crop's estimated at-harvest acreage in a small sample of the stratum's segments. The stratum estimate can utilize Landsat imagery obtained during the current crop grow¬ing season and in previous years. A mixed effects analysis of variance model is used to generate a weighted least squares es¬timate of the stratum at-harvest acreage proportion for the cur¬rent year. Similar Landsat based stratum crop proportion esti¬mates can be combined with historical information on non-sampled (or unsuccessfully sampled) strata to provide crop acreage estimates for large regions. These regional estimates of the at-harvest acreage can be determined early in the crop growing season, at different intermediate points, and at har¬vest time     

Smoothing Time Series with Local Polynomial Regression on Time

Time series smoothers estimate the level of a time series at time t as its conditional expectation given present, past and future observations, with the smoothed value depending on the estimated time series model. Alternatively, local polynomial regressions on time can be used to estimate the level, with the implied smoothed value depending on the weight function and the bandwidth in the local linear least squares fit. In this article we compare the two smoothing approaches and describe their similarities. Through simulations, we assess the increase in the mean square error that results when approximating the estimated optimal time series smoother with the local regression estimate of the level.     

AMEMIYA'S FORM OF THE WEIGHTED LEAST SQUARES ESTIMATOR

Amemiya's estimator is a weighted least squares estimator of the regression coefficients in a linear model with heteroscedastic errors. It is attractive because the heteroscedasticity is not parametrized and the weights (which depend on the error covariance matrix) are estimated nonparametrically. This paper derives an asymptotic expansion for Amemiya's form of the weighted least squares estimator, and uses it to discuss the effects of estimating the weights, of the number of iterations, and of the choice of the initial estimate. The paper also discusses the special case of normally distributed errors and clarifies the particular consequences of assuming normality.     

空间回归模型估计中的最小二乘法

空间回归模型由于引入了空间地理信息而使得其参数估计变得复杂,因为主要采用最大似然法,致使一般人认为在空间回归模型参数估计中不存在最小二乘法。通过分析空间回归模型的参数估计技术,研究发现,最小二乘法和最大似然法分别用于估计空间回归模型的不同的参数,只有将两者结合起来才能快速有效地完成全部的参数估计。数理论证结果表明,空间回归模型参数最小二乘估计量是最佳线性无偏估计量。空间回归模型的回归参数可以在估计量为正态性的条件下而实施显著性检验,而空间效应参数则不可以用此方法进行检验。     

The Equality of the Ordinary Least Squares Estimator and the Best Linear Unbiased Estimator

It is well known that the ordinary least squares estimator of Xβ in the general linear model E _y = Xβ, cov y = σ² V, can be the best linear unbiased estimator even if V is not a multiple of the identity matrix. This article presents, in a historical perspective, the development of the several conditions for the ordinary least squares estimator to be best linear unbiased. Various characterizations of these conditions, using generalized inverses and orthogonal projectors, along with several examples, are also given. In addition, a complete set of references is provided.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Testing inference in heteroskedastic linear regressions: a comparison of two alternative approaches

We consider the issue of performing testing inferences on the parameters that index the linear regression model under heteroskedasticity of unknown form. Quasi-t test statistics use asymptotically correct standard errors obtained from heteroskedasticity-consistent covariance matrix estimators. An alternative approach involves making an assumption about the functional form of the response variances and jointly modelling mean and dispersion effects. In this paper we compare the accuracy of testing inferences made using the two approaches. We consider several different quasi-t tests and also z tests performed after estimated generalized least squares estimation which was carried out using three different estimation strategies. The numerical evidence shows that some quasi-t tests are typically considerably less size distorted in small samples than the tests carried out after the jointly modelling of mean and dispersion effects. Finally, we present and discuss two empirical applications.     

Bootstrapping an Econometric Model: Some Empirical Results

The bootstrap, like the jackknife, is a technique for estimating standard errors. The idea is to use Monte Carlo simulation, based on a nonparametric estimate of the underlying error distribution. The bootstrap will be applied to an econometric model describing the demand for capital, labor, energy, and materials. The model is fitted by three-stage least squares. In sharp contrast with previous results, the coefficient estimates and the estimated standard errors perform very well. However, the model's forecasts show serious bias and large random errors, significantly understated by the conventional standard error of forecast.     

Efficiency of least squares estimators in the presence of spatial autocorrelation

The effect of spatial autocorrelation on inferences made using ordinary least squares estimation is considered. It is found, in some cases, that ordinary least squares estimators provide a reasonable alternative to the estimated generalized least squares estimators recommended in the spatial statistics literature. One of the most serious problems in using ordinary least squares is that the usual variance estimators are severely biased when the errors are correlated. An alternative variance estimator that adjusts for any observed correlation is proposed. The need to take autocorrelation into account in variance estimation negates much of the advantage that ordinary least squares estimation has in terms of computational simplicity     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Performance of adaptive estimators in slowly varying parameter models

This paper analyzes the MSE of the exponentially weighted least squares (EWLS) estimator in dynamic regression models with time-varying parameters. Under the assumption of differentiable parameter functions, it is derived an asymptotic expression which is the sum of a stationary and of an evolutionary component. The validity of the analytical expression is illustrated with simulation experiments, and its usefulness in designing the exponential discounting factor is illustrated on a real case-study. The practical finding is similar to the plug-in bandwidth selection in nonparametric smoothers.     

Nonparametric bayesian inference for dichotomous response models

A Bayesian least squares approach is taken here to estimate certain parameters in generalized linear models for dichotomous response data. The method requires that only first and second moments of the probability distribution representing prior information be specified* Examples are presented to illustrate situations having direct estimates as well as those which require approximate or iterative solutions.     

On approximate least squares estimators of parameters of one-dimensional chirp signal

Chirp signals are quite common in many natural and man-made systems such as audio signals, sonar, and radar. Estimation of the unknown parameters of a signal is a fundamental problem in statistical signal processing. Recently, Kundu and Nandi [Parameter estimation of chirp signals in presence of stationary noise. Stat Sin. 2008;75:187–201] studied the asymptotic properties of least squares estimators (LSEs) of the unknown parameters of a simple chirp signal model under the assumption of stationary noise. In this paper, we propose periodogram-type estimators called the approximate least squares estimators (ALSEs) to estimate the unknown parameters and study the asymptotic properties of these estimators under the same error assumptions. It is observed that the ALSEs are strongly consistent and asymptotically equivalent to the LSEs. Similar to the periodogram estimators, these estimators can also be used as initial guesses to find the LSEs of the unknown parameters. We perform some numerical simulations to see the performance of the proposed estimators and compare them with the LSEs and the estimators proposed by Lahiri et al. [Efficient algorithm for estimating the parameters of two dimensional chirp signal. Sankhya B. 2013;75(1):65–89]. We have analysed two real data sets for illustrative purposes.