On approximate least squares estimators of parameters of one-dimensional chirp signal

Chirp signals are quite common in many natural and man-made systems such as audio signals, sonar, and radar. Estimation of the unknown parameters of a signal is a fundamental problem in statistical signal processing. Recently, Kundu and Nandi [Parameter estimation of chirp signals in presence of stationary noise. Stat Sin. 2008;75:187–201] studied the asymptotic properties of least squares estimators (LSEs) of the unknown parameters of a simple chirp signal model under the assumption of stationary noise. In this paper, we propose periodogram-type estimators called the approximate least squares estimators (ALSEs) to estimate the unknown parameters and study the asymptotic properties of these estimators under the same error assumptions. It is observed that the ALSEs are strongly consistent and asymptotically equivalent to the LSEs. Similar to the periodogram estimators, these estimators can also be used as initial guesses to find the LSEs of the unknown parameters. We perform some numerical simulations to see the performance of the proposed estimators and compare them with the LSEs and the estimators proposed by Lahiri et al. [Efficient algorithm for estimating the parameters of two dimensional chirp signal. Sankhya B. 2013;75(1):65–89]. We have analysed two real data sets for illustrative purposes.     

Efficiency of least squares estimators in the presence of spatial autocorrelation

The effect of spatial autocorrelation on inferences made using ordinary least squares estimation is considered. It is found, in some cases, that ordinary least squares estimators provide a reasonable alternative to the estimated generalized least squares estimators recommended in the spatial statistics literature. One of the most serious problems in using ordinary least squares is that the usual variance estimators are severely biased when the errors are correlated. An alternative variance estimator that adjusts for any observed correlation is proposed. The need to take autocorrelation into account in variance estimation negates much of the advantage that ordinary least squares estimation has in terms of computational simplicity     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

When there are outliers in the carriers:the univariate case

Robust regression estimators studied to date are robust against non-normal distributions of the errors only If the carriers ‘Independent variables’ do not also contain outliers. Several alternative estimators that are robust even 1f there are outliers in the carriers are studied. Two estimators seem to be preferable, but even these can be very Inefficient ‘relative to least squares’ If the errors are normally distributed.     

A Comparison of Partially Adaptive and Reweighted Least Squares Estimation

The small sample performance of least median of squares, reweighted least squares, least squares, least absolute deviations, and three partially adaptive estimators are compared using Monte Carlo simulations. Two data problems are addressed in the paper: (1) data generated from non-normal error distributions and (2) contaminated data. Breakdown plots are used to investigate the sensitivity of partially adaptive estimators to data contamination relative to RLS. One partially adaptive estimator performs especially well when the errors are skewed, while another partially adaptive estimator and RLS perform particularly well when the errors are extremely leptokur-totic. In comparison with RLS, partially adaptive estimators are only moderately effective in resisting data contamination; however, they outperform least squares and least absolute deviation estimators.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Limiting admissible estimators for variance components

Simultaneous estimation of the vector of the variance components for mixed and random models under the quadratic loss function is considered. For a large class of such models there are identified classes of admissible biased invariant quadratic estimators that are better than some admissible unbiased estimators. Numerous numerical results presented in the paper show that for many of the commonly used balanced models the improvements in the quadratic risk may be considerable over a large set of the parameter space.     

A note on minimum average risk estimators for coefficients in linear models

In this note, we have derived a set of necessary and sufficient conditions for the biased estimators analyzed by Swamy and Mehta (1976) to be better than the generalized least squares estimator of the coefficient vector in a standard linear regression model.     

Super efficient frequency estimation

In this paper we propose a modified Newton-Raphson method to obtain super efficient estimators of the frequencies of a sinusoidal signal in presence of stationary noise. It is observed that if we start from an initial estimator with convergence rate O_p(n⁻¹) and use Newton-Raphson algorithm with proper step factor modification, then it produces super efficient frequency estimator in the sense that its asymptotic variance is lower than the asymptotic variance of the corresponding least squares estimator. The proposed frequency estimator is consistent and it has the same rate of convergence, namely O_p(n^−3/2), as the least squares estimator. Monte Carlo simulations are performed to observe the performance of the proposed estimator for different sample sizes and for different models. The results are quite satisfactory. One real data set has been analyzed for illustrative purpose.     

Estimating parameters in a simple errors-in-variables model: a new approach base on finite sample distribution theory

This article presents a first direct application of finite sample distribution theory. The relevance of analytical finite sample research is exemplified in the framework of a simple linear errors-in-variables model (EV Model) with known or approximately known measurement error variance. Analytical results derived byRichardson/Wu (1970) are applied for constructing new approximately unbiased estimators for the slope coefficient in the EV model. The new estimators are compared with the biased least squares estimator and with asymptotic theory based corrected least squares estimators. Retaining responsibility for remaining errors the author is indebted to Prof. H. Schneewei\ and Prof. J. Gruber for helpful comments and discussions. Mrs. A. Brandtstater deserves special mention and thanks for performing the computations reported in section 4.     

On the equality of usual and amemiya's partially generalized least squares estimator

Equivalent conditions are derived for the equality of GLSE (generalized least squares estimator) and partially GLSE (PGLSE), the latter introduced by Amemiya (1983). By adopting a more general approach the ordinary least squares estimator (OLSE) can shown to be a special PGLSE. Furthcrmore, linearly restricted estimators proposed by Balestra (1983) are investigated in this context. To facilitate the comparison of estimators extensive use of oblique and orthogonal projectors is made.     

Shrinkage Structure of Partial Least Squares

Partial least squares regression (PLS) is one method to estimate parameters in a linear model when predictor variables are nearly collinear. One way to characterize PLS is in terms of the scaling (shrinkage or expansion) along each eigenvector of the predictor correlation matrix. This characterization is useful in providing a link between PLS and other shrinkage estimators, such as principal components regression (PCR) and ridge regression (RR), thus facilitating a direct comparison of PLS with these methods. This paper gives a detailed analysis of the shrinkage structure of PLS, and several new results are presented regarding the nature and extent of shrinkage.     

Further results on the two-stage hierarchial information (2shi) estimators in the linear regression models

The paper considers a class of 2SHI estimators for the linear regression models and provides some results regarding the dominance in quadratic loss of this class over the OLS and usual Stein-rule estimators.     

Variance least squares estimators for multivariate linear mixed model

Non-iterative, distribution-free, and unbiased estimators of variance components by least squares method are derived for multivariate linear mixed model. A general inter-cluster variance matrix, a same-member only general inter-response variance matrix, and an uncorrelated intra-cluster error structure for each response are assumed. Projection method is suggested when unbiased estimators of variance components are not nonnegative definite matrices. A simulation study is conducted to investigate the properties of the proposed estimators in terms of bias and mean square error with comparison to the Gaussian (restricted) maximum likelihood estimators. The proposed estimators are illustrated by an application of gene expression familial study.     

An efficient and fast algorithm for estimating the parameters of two-dimensional sinusoidal signals

In this paper we propose a computationally efficient algorithm to estimate the parameters of a 2-D sinusoidal model in the presence of stationary noise. The estimators obtained by the proposed algorithm are consistent and asymptotically equivalent to the least squares estimators. Monte Carlo simulations are performed for different sample sizes and it is observed that the performances of the proposed method are quite satisfactory and they are equivalent to the least squares estimators. The main advantage of the proposed method is that the estimators can be obtained using only finite number of iterations. In fact it is shown that starting from the average of periodogram estimators, the proposed algorithm converges in three steps only. One synthesized texture data and one original texture data have been analyzed using the proposed algorithm for illustrative purpose.     

Estimation of parameters of two-dimensional sinusoidal signal in heavy-tailed errors

In this paper, we consider a two-dimensional sinusoidal model observed in an additive random field. The proposed model has wide applications in statistical signal processing. The additive noise has mean zero but the variance may not be finite. We propose the least squares estimators to estimate the unknown parameters. It is observed that the least squares estimators are strongly consistent. We obtain the asymptotic distribution of the least squares estimators under the assumption that the additive errors are from a symmetric stable distribution. Some numerical experiments are performed to see how the results work for finite samples.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

The unconditional distributions of the OLS,TSLS and LIML estimators in a simple structural equations model

The exact distributions of the standard estimators of the structural coe?cients in a linear structural equations model conditional on the exogenous variables have been shown to have some unexpected and quirky features. Since the argument for conditioning on exogenous (ancillary) variables has been weakened over the past 20 years by the discovery of an “ancillarity paradox,” it is natural to wonder whether such finite sample properties are in fact due to conditioning on the exogenous variables. This article studies the exact distributions of the ordinary least squares (OLS), two-stage least squares (TSLS), and limited information maximum likelihood (LIML) estimators of the structural coe?cients in a linear structural equation without conditioning on the exogenous variables.