Estimation in a linear regression model with stochastic linear restrictions: a new two-parameter-weighted mixed estimator

The present paper considers the weighted mixed regression estimation of the coefficient vector in a linear regression model with stochastic linear restrictions binding the regression coefficients. We introduce a new two-parameter-weighted mixed estimator (TPWME) by unifying the weighted mixed estimator of Schaffrin and Toutenburg [1] and the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [2]. This new estimator is a general estimator which includes the weighted mixed estimator, the TPE and the restricted two-parameter estimator (RTPE) proposed by Özkale and Kaç?ranlar [2] as special cases. Furthermore, we compare the TPWME with the weighted mixed estimator and the TPE with respect to the matrix mean square error criterion. A numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters to illustrate some of the theoretical results.     

Modified Restricted Almost Unbiased Liu Estimator in Linear Regression Model

In this article we introduce a modified restricted almost unbiased Liu estimator in linear regression model which satisfies the linear restrictions. The mean squared error matrix (MSEM) of the proposed estimator is derived and compared with the corresponding competitors in literature. Finally, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

On the Stochastic Restricted Almost Unbiased Estimators in Linear Regression Model

In this article, the stochastic restricted almost unbiased ridge regression estimator and stochastic restricted almost unbiased Liu estimator are proposed to overcome the well-known multicollinearity problem in linear regression model. The quadratic bias and mean square error matrix of the proposed estimators are derived and compared. Furthermore, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

Two kinds of restricted modified estimators in linear regression model

In this paper, we introduce two kinds of new restricted estimators called restricted modified Liu estimator and restricted modified ridge estimator based on prior information for the vector of parameters in a linear regression model with linear restrictions. Furthermore, the performance of the proposed estimators in mean squares error matrix sense is derived and compared. Finally, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

On the restricted almost unbiased estimators in linear regression

In this paper, the restricted almost unbiased ridge regression estimator and restricted almost unbiased Liu estimator are introduced for the vector of parameters in a multiple linear regression model with linear restrictions. The bias, variance matrices and mean square error (MSE) of the proposed estimators are derived and compared. It is shown that the proposed estimators will have smaller quadratic bias but larger variance than the corresponding competitors in literatures. However, they will respectively outperform the latter according to the MSE criterion under certain conditions. Finally, a simulation study and a numerical example are given to illustrate some of the theoretical results.     

An alternative stochastic restricted Liu estimator in linear regression

In this paper, we introduce an alternative stochastic restricted Liu estimator for the vector of parameters in a linear regression model when additional stochastic linear restrictions on the parameter vector are assumed to hold. The new estimator is a generalization of the ordinary mixed estimator (OME) (Durbin in J Am Stat Assoc 48:799–808, 1953; Theil and Goldberger in Int Econ Rev 2:65–78, 1961; Theil in J Am Stat Assoc 58:401–414, 1963) and Liu estimator proposed by Liu (Commun Stat Theory Methods 22:393–402, 1993). Necessary and sufficient conditions for the superiority of the new stochastic restricted Liu estimator over the OME, the Liu estimator and the estimator proposed by Hubert and Wijekoon (Stat Pap 47:471–479, 2006) in the mean squared error matrix (MSEM) sense are derived. Furthermore, a numerical example based on the widely analysed dataset on Portland cement (Woods et al. in Ind Eng Chem 24:1207–1241, 1932) and a Monte Carlo evaluation of the estimators are also given to illustrate some of the theoretical results.     

On weak consistency in linear models with equi-correlated random errors

A new estimator in linear models with equi-correlated random errors is postulated. Consistency properties of the proposed estimator and the ordinary least squares estimator are studied. It is shown that the new estimator has smaller variance than the usual least squares estimator under some mild conditions. In addition, it is observed that the new estimator tends to be weakly consistent in many cases where the usual least squares estimator is not.     

Performances of the Positive-Rule Stein-Type Ridge Estimator in a Linear Regression Model with Spherically Symmetric Error Distributions

In this article, the positive-rule Stein-type ridge estimator (PSRE) is introduced for the parameters in a multiple linear regression model with spherically symmetric error distributions when it is suspected that the parameter vector may be restricted to a linear manifold. The bias and quadratic risk functions of the PSRE are derived and compared with some related competing estimators in literatures. Particularly, some sufficient conditions are derived for superiority of the PSRE over the ordinary ridge estimator, the restricted ridge estimator and the preliminary test ridge estimator, respectively. Furthermore, some graphical results are provided to illustrate some of the theoretical results.     

Performance of the stein-type two-parameter estimator in multiple linear regression model

In this article, we consider the Stein-type approach to the estimation of the regression parameter in a multiple regression model under a multicollinearity situation. The Stein-type two-parameter estimator is proposed when it is suspected that the regression parameter may be restricted to a subspace. The bias and the quadratic risk of the proposed estimator are derived and compared with the two-parameter estimator (TPE), the restricted TPE and the preliminary test TPE. The conditions of superiority of the proposed estimator are obtained. Finally, a real data example is provided to illustrate some of the theoretical results.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

A revised Cholesky decomposition to combat multicollinearity in multiple regression models

As known, the ordinary least-squares estimator (OLSE) is unbiased and also, has the minimum variance among all the linear unbiased estimators. However, under multicollinearity the estimator is generally unstable and poor in the sense that variance of the regression coefficients may be inflated and absolute values of the estimates may be too large. There are several classes of biased estimators in statistical literature to decrease the effect of multicollinearity in the design matrix. Here, based on the Cholesky decomposition, we propose such an estimator which makes the data to be slightly distorted. The exact risk expressions as well as the biases are derived for the proposed estimator. Also, some results demonstrating superiority of the suggested estimator over OLSE are obtained. Finally, a Monté-Carlo simulation study and a real data application related to acetylene data are presented to support our theoretical discussions.     

Estimation in Singular Linear Models with Stochastic Linear Restrictions

This article generalizes the ordinary mixed estimator (OME) in theory, and obtains the estimator of the unknown regression parameters in singular linear models with stochastic linear restrictions: singular mixed estimator (SME). We also give some properties of SME obtained in this article, and prove that it is superior to unrestricted least squared estimator (LSE) in singular linear models in the sense of the covariance matrix and generalized mean square error (GMSE). After that, we also have a discussion about the two-stage estimator of SME. The result we give in this article could be regarded as generalizations of both OME and unrestricted LSE at the same time.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

Two Kinds of Restricted Estimators in Singular Linear Regression

In this article, the parameter estimators in singular linear model with linear equality restrictions are considered. The restricted root estimator and the generalized restricted root estimator are proposed and some properties of the estimators are also studied. Furthermore, we compare them with the restricted unified least squares estimator and show their sufficient conditions under which their superior over the restricted unified least squares estimator in terms of mean squares error, and discuss the choice of the unknown parameters of the generalized restricted root estimator.     

Prediction Error Property of the Lasso Estimator and its Generalization

The lasso procedure is an estimator‐shrinkage and variable selection method. This paper shows that there always exists an interval of tuning parameter values such that the corresponding mean squared prediction error for the lasso estimator is smaller than for the ordinary least squares estimator. For an estimator satisfying some condition such as unbiasedness, the paper defines the corresponding generalized lasso estimator. Its mean squared prediction error is shown to be smaller than that of the estimator for values of the tuning parameter in some interval. This implies that all unbiased estimators are not admissible. Simulation results for five models support the theoretical results.     

On preliminary test almost unbiased two-parameter estimator in linear regression model with student's t errors

In this paper, the preliminary test approach to the estimation of the linear regression model with student's t errors is considered. The preliminary test almost unbiased two-parameter estimator is proposed, when it is suspected that the regression parameter may be restricted to a constraint. The quadratic biases and quadratic risks of the proposed estimators are derived and compared under both null and alternative hypotheses. The conditions of superiority of the proposed estimators for departure parameter and biasing parameters k and d are derived, respectively. Furthermore, a real data example and a Monte Carlo simulation study are provided to illustrate some of the theoretical results.     

A modified ridge m-estimator for linear regression model with multicollinearity and outliers

The ordinary least-square estimators for linear regression analysis with multicollinearity and outliers lead to unfavorable results. In this article, we propose a new robust modified ridge M-estimator (MRME) based on M-estimator (ME) to deal with the combined problem resulting from multicollinearity and outliers in the y-direction. MRME outperforms modified ridge estimator, robust ridge estimator and ME, according to mean squares error criterion. Furthermore, a numerical example and a Monte Carlo simulation experiment are given to illustrate some of the theoretical results.     

On the Restricted Liu Estimator in the Gauss–Markov Model

In this paper, we compare two estimators, the RLE (restricted Liu estimator) and the RLSE (restricted least squares estimator) of parameters in linear models under Gauss–Markov models. Using generalized inverse of matrices, we found some equivalency conditions for the superiority of the RLE with respect to the MSE criterion.     

A Weighted Linear Estimator of Multivariate ARCH Parameters

A weighted linear estimator (WLE) of the parameters of multivariate ARCH models is proposed. The accuracy of WLE in estimating the parameters of multivariate ARCH models is compared with the widely used quasi-maximum likelihood estimator (QMLE) through simulations. Application to real data sets are also presented and forecasts of variance-covariance matrix and value-at-risk (VaR) are obtained. The weighted resampling methods are used to approximate the sampling distribution of the proposed estimator. Our study indicates that the forecasting performance of WLE is not inferior and one-day ahead risk estimates are also found better than the QMLE.     

A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing

This paper adopts a Bayesian strategy for generalized ridge estimation for high-dimensional regression. We also consider significance testing based on the proposed estimator, which is useful for selecting regressors. Both theoretical and simulation studies show that the proposed estimator can simultaneously outperform the ordinary ridge estimator and the LSE in terms of the mean square error (MSE) criterion. The simulation study also demonstrates the competitive MSE performance of our proposal with the Lasso under sparse models. We demonstrate the method using the lung cancer data involving high-dimensional microarrays.