Properties of the mixed regression estimator when disturbances are not necessarily normal

Small-disturbance approximations for the bias vector and mean squared error matrix of the mixed regression estimator for the coefficients in a linear regression model are derived and efficiency with respect to least squares estimator is examined.     

On the minimum mean squared error estimators in a regression model

In this paper we analyze the properties of two estimators oroposed by Farebrother (1975) for linear regression models.     

Generalized mean squared error properties of regression estimators

Theobald (1974) compares Ordinary Least Squares and Ridge Regression estimators of regression parameters using a generalized mean squared error criterion. This paper presents the generalized mean squared error of a Principal Components Regression estimator and comparisons are made with each of the above estimators. In general the choice of which estimator to use depends on the magnitude and the orientation of the unknown parameter vector.     

On the pointwise mean squared error of a multidimensional term-by-term thresholding wavelet estimator

In this paper we provide a theoretical contribution to the pointwise mean squared error of an adaptive multidimensional term-by-term thresholding wavelet estimator. A general result exhibiting fast rates of convergence under mild assumptions on the model is proved. It can be applied for a wide range of non parametric models including possible dependent observations. We give applications of this result for the non parametric regression function estimation problem (with random design) and the conditional density estimation problem.     

Bounds on minimum mean squared error in ridge regression

The minimum MSE (mean squared error) of ridge regression coefficient estimates (for a given set of eigenvalues and variance) is a function of the transformed coefficient vector. In this paper, the authors prove that the minimum MSE is bounded, for a given coefficient vector length, by the two cases corresponding to the signal completely contained in the component associated with the smallest or largest eigenvalue. The implication for evaluating proposed estimators of the ridge regression biasing parameter is discussed.     

A small sample estimator for a polynomial regression with errors in the variables

An adjusted least squares estimator, introduced by Cheng and Schneeweiss for consistently estimating a polynomial regression of any degree with errors in the variables, is modified such that it shows good results in small samples without losing its asymptotic properties for large samples. Simulation studies corroborate the theoretical findings.     

Minimum mean squared error estimation of each individual coefficient in a linear regression model

In this paper, we derive the exact mean squared error (MSE) of the minimum MSE estimator for each individual coefficient in a linear regression model, and show a sufficient condition for the minimum MSE estimator for each individual coefficient to dominate the OLS estimator. Numerical results show that when the number of independent variables is 2 and 3, the minimum MSE estimator for each individual coefficient can be a good alternative to the OLS and Stein-rule estimators.     

Optimal minimax squared error risk estimation of the mean of a multivariate normal distribution

Minimax squared error risk estimators of the mean of a multivariate normal distribution are characterized which have smallest Bayes risk with respect to a spherically symmetric prior distribution for (i) squared error loss, and (ii) zero-one loss depending on whether or not estimates are consistent with the hypothesis that the mean is null. In (i), the optimal estimators are the usual Bayes estimators for prior distributions with special structure. In (ii), preliminary test estimators are optimal. The results are obtained by applying the theory of minimax-Bayes-compromise decision problems.     

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

An exact formula for the mean squared error of the inverse estimator in the linear calibration problem

The linear calibration problem is considered. An exact formula for the mean squared error of the inverse estimator, involving expectations of functions of a Poisson random variable, is derived. The formula may be expressed in closed form if the number of observations in the calibration experiment is odd; for an even number of observations, the numerical evaluation of a simple integral or the use of a standard table of the confluent hypergeometric function is required. Previous expressions for the mean squared error have either been asymptotic expansions or estimates obtained by simulation.     

Bootstrap mean squared error of a small-area EBLUP

Concerning the estimation of linear parameters in small areas, a nested-error regression model is assumed for the values of the target variable in the units of a finite population. Then, a bootstrap procedure is proposed for estimating the mean squared error (MSE) of the EBLUP under the finite population setup. The consistency of the bootstrap procedure is studied, and a simulation experiment is carried out in order to compare the performance of two different bootstrap estimators with the approximation given by Prasad and Rao [Prasad, N.G.N. and Rao, J.N.K., 1990, The estimation of the mean squared error of small-area estimators. Journal of the American Statistical Association, 85, 163–171.]. In the numerical results, one of the bootstrap estimators shows a better bias behavior than the Prasad–Rao approximation for some of the small areas and not much worse in any case. Further, it shows less MSE in situations of moderate heteroscedasticity and under mispecification of the error distribution as normal when the true distribution is logistic or Gumbel. The proposed bootstrap method can be applied to more general types of parameters (linear of not) and predictors.     

The exact mean squared error risks of preliminary test type estimators for the multivariate normal mean

The exact mean squared error risks of the preliminary test estimtor and the Sclove modified Stein rule estimator (Sclove, Morris and Radhakrishnan, 1972) for the multivariate normal mean are computed and their risks are compared with the risks of Stein estimators.     

Polynomial regression with errors in the variables

A polynomial functional relationship with errors in both variables can be consistently estimated by constructing an ordinary least squares estimator for the regression coefficients, assuming hypothetically the latent true regressor variable to be known, and then adjusting for the errors. If normality of the error variables can be assumed, the estimator can be simplified considerably. Only the variance of the errors in the regressor variable and its covariance with the errors of the response variable need to be known. If the variance of the errors in the dependent variable is also known, another estimator can be constructed.     

Reducing the mean squared error in kernel density estimation

In this article, we propose a version of a kernel density estimator which reduces the mean squared error of the existing kernel density estimator by combining bias reduction and variance reduction techniques. Its theoretical properties are investigated, and a Monte Carlo simulation study supporting theoretical results on the proposed estimator is given.     

On the bias and mean square error of the least square estimator in a regression model with two inequality constraints and multivariate t error terms

We derive and numerically evaluate the bias and mean square error of the inequality constrained least squares estimator in a model with two inequality constraints and multivariate terror terms. Our results suggest that qualitatively, the estimator properties found for models with normal errors carry over to the case of multivariate terrors.     

The exact distribution and density functions of a pre-test estimator of the error variance in a linear regression model with proxy variables

We consider a linear regression model when some independent variables are unobservable, but proxy variables are available instead of them. We derive the distribution and density functions of a pre-test estimator of the error variance after a pre-test for the null hypothesis that the coefficients for the unobservable variables are zeros. Based on the density function, we show that when the critical value of the pre-test is unity, the coverage probability in the interval estimation of the error variance is maximum.     

Relative squared error prediction in the generalized linear regression model

In linear regression models, predictors based on least squares or on generalized least squares estimators are usually applied which, however, fail in case of multicollinearity. As an alternative biased estimators like ridge estimators, Kuks-Olman estimators, Bayes or minimax estimators are sometimes suggested. In our analysis the relative instead of the generally used absolute squared error enters the objective function. An explicit minimax solution is derived which, in an important special case, can be viewed as a predictor based on a Kuks-Olman estimator.     

Liu estimator in partly linear regression models with correlated errors

This article is concerned with the parameter estimation in partly linear regression models when the errors are dependent. To overcome the multicollinearity problem, a generalized Liu estimator is proposed. The theoretical properties of the proposed estimator and its relationship with some existing methods designed for partly linear models are investigated. Finally, a hypothetical data is conducted to illustrate some of the theoretical results.     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.