A note on minimum average risk estimators for coefficients in linear models

In this note, we have derived a set of necessary and sufficient conditions for the biased estimators analyzed by Swamy and Mehta (1976) to be better than the generalized least squares estimator of the coefficient vector in a standard linear regression model.     

Ordinary and weighted least-squares estimators

We propose a method of estimating the asymptotic relative efficiency (ARE) of the weighted least-squares estimator (WLSE) with respect to the ordinary least-squares estimator (OLSE) in a heteroscedastic linear regression model with a large number of observations but a small number of replicates at each value of the regressors. The weights used in the WLSE are the reciprocals of the (within-group) average of squared residuals. It is shown that the OLSE is more efficient than the WLSE if the maximum number of replicates is not larger than two. The proposed estimator of the ARE is consistent as the number of observations tends to infinity. Finite-sample performance of this estimator is examined in a simulation study. An adaptive estimator, which is asymptotically more efficient than the OLSE and the WLSE, is proposed.     

Comparison of some linear calibration estimators

In this article, we consider a family of linear calibration estimators arising from inverse estimator and analyze its properties employing the small disturbance asymptotic theory. The asymptotic approximations for bias and mean squared error of this family are compared with the corresponding results for classical and inverse estimators, whose properties are also compared.     

On approximate least squares estimators of parameters of one-dimensional chirp signal

Chirp signals are quite common in many natural and man-made systems such as audio signals, sonar, and radar. Estimation of the unknown parameters of a signal is a fundamental problem in statistical signal processing. Recently, Kundu and Nandi [Parameter estimation of chirp signals in presence of stationary noise. Stat Sin. 2008;75:187–201] studied the asymptotic properties of least squares estimators (LSEs) of the unknown parameters of a simple chirp signal model under the assumption of stationary noise. In this paper, we propose periodogram-type estimators called the approximate least squares estimators (ALSEs) to estimate the unknown parameters and study the asymptotic properties of these estimators under the same error assumptions. It is observed that the ALSEs are strongly consistent and asymptotically equivalent to the LSEs. Similar to the periodogram estimators, these estimators can also be used as initial guesses to find the LSEs of the unknown parameters. We perform some numerical simulations to see the performance of the proposed estimators and compare them with the LSEs and the estimators proposed by Lahiri et al. [Efficient algorithm for estimating the parameters of two dimensional chirp signal. Sankhya B. 2013;75(1):65–89]. We have analysed two real data sets for illustrative purposes.     

Comparison of methods of estimation of parameters of the weibull distribution

The generalised least squares, maximum likelihood, Bain-Antle 1 and 2, and two mixed methods of estimating the parameters of the two-parameter Weibull distribution are compared. The comparison is made using (a) the observed relative efficiency of parameter estimates and (b) themean squared relative error in estimated quantiles, to summarize the results of 1000 simulated samples of sizes 10 and 25. The results are that: generalised least squares is the best method of estimating the shape parameter ß the best method of estimating the scale parameter a depends onthe size of ß for quantile estimation maximum likelihood is best Bain-Antle 2 is uniformly the worst of the methods.     

Superiority comparisons of homogeneous linear estimators

In the paper homogeneous linear estimators of the parameter vector of the general linear model are compared in terms of their MSE matrices. A necessary and sufficient condition for the difference of two MSE matrices to be positive definite is obtained and its practical existence discussed. The non-negative definiteness of the difference also receives attention, and conditions for this case are discussed. The absence of any conditions of the above type is taken into consideration as well.     

On nonparametric regression estimators based on regression quantiles

In the ciassical regression model Y_i=h(x_i) + ? _i, i=1,…,n, Cheng (1984) introduced linear combinations of regression quantiles as a new class of estimators for the unknown regression function h(x). The asymptotic properties studied in Cheng (1984) are reconsidered. We obtain a sharper scrong consistency rate and we improve on the conditions for asymptotic normality by proving a new result on the remainder term in the Bahadur representation for regression quantiles.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

Further results on the two-stage hierarchial information (2shi) estimators in the linear regression models

The paper considers a class of 2SHI estimators for the linear regression models and provides some results regarding the dominance in quadratic loss of this class over the OLS and usual Stein-rule estimators.     

On the discretization of nonparametric isotropic covariogram estimators

In this article, we describe the discretization of nonparametric covariogram estimators for isotropic stationary stochastic processes. The use of nonparametric estimators is important to avoid the difficulties in selecting a parametric model. The key property the isotropic covariogram must satisfy is to be positive definite and thus have the form characterized by Yaglom's representation of Bochner's theorem. We present an optimal discretization of the latter in the sense that the resulting nonparametric covariogram estimators are guaranteed to be smooth and positive definite in the continuum. This provides an answer to an issue raised by Hall, Fisher and Hoffmann (1994). Furthermore, from a practical viewpoint, our result is important because a nonlinear constrained algorithm can sometimes be avoided and the solution can be found by least squares. Some numerical results are presented for illustration.     

Efficiency of least squares estimators in the presence of spatial autocorrelation

The effect of spatial autocorrelation on inferences made using ordinary least squares estimation is considered. It is found, in some cases, that ordinary least squares estimators provide a reasonable alternative to the estimated generalized least squares estimators recommended in the spatial statistics literature. One of the most serious problems in using ordinary least squares is that the usual variance estimators are severely biased when the errors are correlated. An alternative variance estimator that adjusts for any observed correlation is proposed. The need to take autocorrelation into account in variance estimation negates much of the advantage that ordinary least squares estimation has in terms of computational simplicity     

Minimum average risk estimators for coefficients in linear models

We consider the problem of estimating the coefficient vector β of a linear regression model with quadratic loss function. Some biased estimators which utilize the prior information about β are considered. Also studied is the problem of estimating the parameters of an over-identified structural equation from undersized samples.     

Mean square error matrix comparisons of estimators in linear regression

The aim of this paper is to provide criteria which allow to compare two estimators of the parameter vector in the linear regression model with respect to their mean square error matrices, where the main interest is focussed on the case when the difference of the covariance matrices is singular. The results obtained are applied to equality restricted and pretest estimators.     

Superiority comparisons of heterogeneous linear estimators

In this paper conditions for strong and weak superiority of a heterogeneous linear estimator over another are derived. The general results are applied to some special cases: in particular, two restricted least squares estimators are compared using the superiority conditions obtained. The weak superiority criterion is used as a basis in forming a generalization of an optimal se-quence of tests (Anderson, 1962) for searching for the best estimator when the alternative linear restrictions form a nested se-quence of hypotheses. An application of this is the determination of the correct length of lag and appropriate degree of polynomial in the estimation of polynomial distributed lag models.     

On the bias and variance of some proportion estimators

Providing certain parameters are known, almost any linear map from R^P to R¹ can be adjusted to yield a consistent and unbiased estimator in the context of estimating the mixing proportion θ on the basis of an unclassified sample of observations taken from a mixture of two p-dimensional distributions in proportions θ and 1-θ. Attention is focused on an estimator proposed recently, θ, which has minimum variance over all such linear maps. Unfortunately, the form of θ depends on the means of the component distributions and the covariance matrix of the mixture distribution. The effect of using appropriate sample estimates for these unknown parameters in forming θ is investigated by deriving the asymptotic mean and variance of the resulting estimator. The relative efficiency of this estimator under normality is derived. Also, a study is undertaken of the performance of a similar type of estimator appropriate in the context where an observed data vector is not an observation from either one or the other onent distributions, but is recorded as an integrated measurement over a surface area which is a mixture of two categories whose characteristics have different statistical distributions.The asymptotic bias in this case is compared with some available practical results.     

Calibration of the estimators of variance

This investigation suggests new techniques to calibrate estimators of variance. Estimators of the variance of simple mean, ratio and regression estimators under different sampling schemes are shown to be special cases of the proposed calibration techniques. The approach has more practical use due to recent advances in programming techniques and computational speed. An empirical study has been carried out to address the properties of these proposed strategies.     

On generalized least squares estimation of the weibull distribution

A simple estimation procedure, based on the generalized least squares method, for the parameters of the Weibull distribution is described and investigated. Through a simulation study, this estimation technique is compared with maximum likelihood estimation, ordinary least squares estimation, and Menon's estimation procedure; this comparison is based on observed relative efficiencies (that is, the ratio of the Cramer-Rao lower bound to the observed mean squared error). Simulation results are presented for samples of size 25. Among the estimators considered in this simulation study, the generalized least squares estimator was found to be the "best" estimator for the shape parameter and a close competitor to the maximum likelihood estimator of the scale parameter.     

Statistical inference on asymptotic properties of two estimators for the partially linear single-index models

The outer product of gradients (OPG) estimation procedure based on least squares (LS) approach has been presented by Xia et al. [An adaptive estimation of dimension reduction space. J Roy Statist Soc Ser B. 2002;64:363–410] to estimate the single-index parameter in partially linear single-index models (PLSIM). However, its asymptotic property has not been established yet and the efficiency of LS-based method can be significantly affected by outliers and heavy-tailed distributions. In this paper, we firstly derive the asymptotic property of OPG estimator developed by Xia et al. [An adaptive estimation of dimension reduction space. J Roy Statist Soc Ser B. 2002;64:363–410] in theory, and a novel robust estimation procedure combining the ideas of OPG and local rank (LR) inference is further developed for PLSIM along with its theoretical property. Then, we theoretically derive the asymptotic relative efficiency (ARE) of the proposed LR-based procedure with respect to LS-based method, which is shown to possess an expression that is closely related to that of the signed-rank Wilcoxon test in comparison with the t-test. Moreover, we demonstrate that the new proposed estimator has a great efficiency gain across a wide spectrum of non-normal error distributions and almost not lose any efficiency for the normal error. Even in the worst case scenarios, the ARE owns a lower bound equalling to 0.864 for estimating the single-index parameter and a lower bound being 0.8896 for estimating the nonparametric function respectively, versus the LS-based estimators. Finally, some Monte Carlo simulations and a real data analysis are conducted to illustrate the finite sample performance of the estimators.     

On the unbiasedness of iterated gls estimators

We Formulate sufficienct conditions for the existonce of the expectation of iterated generalized expectation of the iterated generalized least squares estimator, which consequently guarantee its unbiasedness, The analysis is applied to the maximum likelihood estimator in the general linear model with normal disturbances, where a set of assumptions ensures convergence of the iteration as well as unbiasedness.