A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

A Generalized Stochastic Restricted Ridge Regression Estimator

In this article, we introduce a new stochastic restricted estimator for the unknown vector parameter in the linear regression model when stochastic linear restrictions on the parameters hold. We show that the new estimator is a generalization of the ordinary mixed estimator (OME), Liu estimator (LE), ordinary ridge estimator (ORR), (k-d) class estimator, stochastic restricted Liu estimator (SRLE), and stochastic restricted ridge estimator (SRRE). Performance of the new estimator in comparison to other estimators in terms of the mean squares error matrix (MMSE) is examined. Numerical example from literature have been given to illustrate the results.     

A New Two-Parameter Estimator in Linear Regression

This article is concerned with the parameter estimation in linear regression model. To overcome the multicollinearity problem, a new two-parameter estimator is proposed. This new estimator is a general estimator which includes the ordinary least squares (OLS) estimator, the ridge regression (RR) estimator, and the Liu estimator as special cases. Necessary and sufficient conditions for the superiority of the new estimator over the OLS, RR, Liu estimators, and the two-parameter estimator proposed by Ozkale and Kaciranlar (2007 Ozkale , M. R. , Kaciranlar , S. ( 2007 ). The restricted and unrestricted two-parameter estimators . Commun. Statist. Theor. Meth. 36 : 2707 – 2725 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) in the mean squared error matrix (MSEM) sense are derived. Furthermore, we obtain the estimators of the biasing parameters and give a numerical example to illustrate some of the theoretical results.     

MSE Performance of a Heterogeneous Pre-Test Ridge Regression Estimator

This article describes how diagnostic procedures were derived for symmetrical nonlinear regression models, continuing the work carried out by Cysneiros and Vanegas (2008 Cysneiros , F. J. A. , Vanegas , L. H. ( 2008 ). Residuals and their statistical properties in symmetrical nonlinear models . Statist. Probab. Lett. 78 : 3269 – 3273 .[Crossref], [Web of Science ®] , [Google Scholar]) and Vanegas and Cysneiros (2010 Vanegas , L. H. , Cysneiros , F. J. A. ( 2010 ). Assesment of diagnostic procedures in symmetrical nonlinear regression models . Computat. Statist. Data Anal. 54 : 1002 – 1016 .[Crossref], [Web of Science ®] , [Google Scholar]), who showed that the parameters estimates in nonlinear models are more robust with heavy-tailed than with normal errors. In this article, we focus on assessing if the robustness of this kind of models is also observed in the inference process (i.e., partial F-test). Symmetrical nonlinear regression models includes all symmetric continuous distributions for errors covering both light- and heavy-tailed distributions such as Student-t, logistic-I and -II, power exponential, generalized Student-t, generalized logistic, and contaminated normal. Firstly, a statistical test is shown to evaluating the assumption that the error terms all have equal variance. The results of simulation studies which describe the behavior of the test for heteroscedasticity proposed in the presence of outliers are then given. To assess the robustness of inference process, we present the results of a simulation study which described the behavior of partial F-test in the presence of outliers. Also, some diagnostic procedures are derived to identify influential observations on the partial F-test. As ilustration, a dataset described in Venables and Ripley (2002 Venables , W. N. , Ripley , B. D. ( 2002 ). Modern Applied with S. , 4th ed. New York : Springer .[Crossref] , [Google Scholar]), is also analyzed.     

New Ridge Regression Estimator in Semiparametric Regression Models

In the context of ridge regression, the estimation of shrinkage parameter plays an important role in analyzing data. Many efforts have been put to develop the computation of risk function in different full-parametric ridge regression approaches using eigenvalues and then bringing an efficient estimator of shrinkage parameter based on them. In this respect, the estimation of shrinkage parameter is neglected for semiparametric regression model. Not restricted, but the main focus of this approach is to develop necessary tools for computing the risk function of regression coefficient based on the eigenvalues of design matrix in semiparametric regression. For this purpose the differencing methodology is applied. We also propose a new estimator for shrinkage parameter which is of harmonic type mean of ridge estimators. It is shown that this estimator performs better than all the existing ones for the regression coefficient. For our proposal, a Monte Carlo simulation study and a real dataset analysis related to housing attributes are conducted to illustrate the efficiency of shrinkage estimators based on the minimum risk and mean squared error criteria.     

Tuning Parameter Selection and Various Good Fitting Characteristics for the Liu-Type Estimator in Linear Regression

Liu (2003 Liu , K. ( 2003 ). Using Liu-Type estimator to combat collinearity . Commun. Statist. Theor. Meth. 32 ( 5 ): 1009 – 1020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) proposed the Liu-Type estimator (LTE) to combat the well-known multicollinearity problem in linear regression. In this article, various better fitting characteristics of the LTE than those of the ordinary ridge regression estimator (Hoerl and Kennard, 1970 Hoerl , A. E. , Kennard , R. W. ( 1970 ). Ridge regression: Biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) are considered. In particular, we derived two methods to determine the parameter d for the LTE and find that the ridge parameter k could serve for regularization of an ill-conditioned design matrix, while the other parameter d could be used for tuning the fit quality. In addition, the coefficients of regression, coefficient of multiple determination, residual error variance, and generalized cross validation (GCV) of the prediction quality are very stable, and as the ridge parameter increases they eventually reach asymptotic levels, which produces robust regression models. Furthermore, a Monte Carlo evaluation of these features is also given to illustrate some of the theoretical results.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.     

Choosing Ridge Parameter for Regression Problems

ABSTRACT

Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) introduced the ridge regression estimator as an alternative to the ordinary least squares estimator in the presence of multicollinearity. In this article, a new approach for choosing the ridge parameter (K), when multicollinearity among the columns of the design matrix exists, is suggested and evaluated by simulation techniques, in terms of mean squared errors (MSE). A number of factors that may affect the properties of these methods have been varied. The MSE from this approach has shown to be smaller than using Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) in almost all situations.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.     

Linearized Ridge Regression Estimator Under the Mean Squared Error Criterion in a Linear Regression Model

This article mainly aims to study the superiority of the notion of linearized ridge regression estimator (LRRE) under the mean squared error criterion in a linear regression model. Firstly, we derive uniform lower bound of MSE for the class of the generalized shrinkage estimator (GSE), based on which it is shown that the optimal LRRE is the best estimator in the class of GSE's. Secondly, we propose the notion of the almost unbiased completeness and show that LRRE possesses such a property. Thirdly, the simulation study is given, from which it indicates that the LRRE performs desirably. Finally, the main results are applied to the well known Hald data.     

A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions

Based on the work of Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameters for regression problems . Communications in Statistics – Theory and Methods 34 : 1177 – 1182 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi , M. , Khalaf , G. , Shukur , G. ( 2006 ). Some modifications for choosing ridge parameters . Communications in Statistics – Theory and Methods 35 : 2005 – 2020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2010 Muniz , G. , Kibria , B. M. G. , Shukur , G. ( 2010 ). On developing ridge regression parameters: a graphical Investigation. Submitted for Publication . [Google Scholar]), this article considers several estimators for estimating the ridge parameter k. This article differs from aforementioned articles in three ways: (1) Data are generated from Normal, Student's t, and F distributions with appropriate degrees of freedom; (2) The number of regressors considered are from 4–12 instead of 2–4, which are the usual practice; (3) Both mean square error (MSE) and prediction sum of square (PRESS) are considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has negative effect on the MSE and PRESS. However, increasing the number of regressors has positive effect on MSE and PRESS. When the sample size increases the MSE decreases even when the correlation between the independent variables is large. It is interesting to note that the dominance pictures of the estimators are remained the same under both the MSE and PRESS criterion. However, the performance of the estimators depends on the choice of the assumption of the error distribution of the regression model.     

A Monte Carlo Study of Recent Ridge Parameters

A number of procedures have been developed for finding biased estimators of regression parameters. One of these procedures is the ridge regression. In this article, a new approach to obtain the ridge parameter K is suggested and then evaluated by Monte Carlo simulations. A number of different models are investigated for different number of observations, the strength of correlation between the explanatory variables, and distribution of the error terms. The mean squared error (MSE) criterion is used to examine the performance of the proposed estimators when compared with other well-known estimators. Under certain conditions, it is shown that at least one of the proposed estimators have a smaller MSE than the ordinary least squared estimator (OLS) and Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) estimator (HK).     

A Note on a Commonly Used Ridge Regression Monte Carlo Design

Ridge estimators are usually examined through Monte Carlo simulations since their properties are difficult to obtain analytically. In this paper we argue that a simulation design commonly used in the literature will give biased results of Monte Carlo simulations in favor of ridge regression over ordinary least square estimators. Specifically, it is argued that the properties of ridge estimators that are functions of p distinct regressor eigenvalues should not be evaluated through Monte Carlo designs using only two distinct eigenvalues.     

An Application of the Local Influence Diagnostics to Ridge Regression Under Elliptical Model

In this article, we study the effect of a minor perturbation on the ridge estimator considering the elliptical distribution for the errors. The necessary matrices for assessing the local influence under the perturbation of the explanatory variables and the scale matrix are derived. The Longley data is analyzed for illustration.     

Performance of Some Ridge Parameters for Probit Regression: With Application to Swedish Job Search Data

In ridge regression, the estimation of the ridge parameter is an important issue. This article generalizes some methods for estimating the ridge parameter for probit ridge regression (PRR) model based on the work of Kibria et al. (2011 Kibria, B. M. G., Månsson, K. and Shukur, G. 2011. Performance of some logistic ridge regression parameters. Computational Economics, DOI: 10.1007/s10614-011-9275-x [Google Scholar]). The performance of these new estimators is judged by calculating the mean squared error (MSE) using Monte Carlo simulations. In the design of the experiment, we chose to vary the sample size and the number of regressors. Furthermore, we generate explanatory variables that are linear combinations of other regressors, which is a common situation in economics. In an empirical application regarding Swedish job search data, we also illustrate the benefits of the new method.     

Liu-Type Logistic Estimator

It is known that multicollinearity inflates the variance of the maximum likelihood estimator in logistic regression. Especially, if the primary interest is in the coefficients, the impact of collinearity can be very serious. To deal with collinearity, a ridge estimator was proposed by Schaefer et al. The primary interest of this article is to introduce a Liu-type estimator that had a smaller total mean squared error (MSE) than the Schaefer's ridge estimator under certain conditions. Simulation studies were conducted that evaluated the performance of this estimator. Furthermore, the proposed estimator was applied to a real-life dataset.     

Linearized Ridge Regression Estimator in Linear Regression

In this article, we aim to study the linearized ridge regression (LRR) estimator in a linear regression model motivated by the work of Liu (1993). The LRR estimator and the two types of generalized Liu estimators are investigated under the PRESS criterion. The method of obtaining the optimal generalized ridge regression (GRR) estimator is derived from the optimal LRR estimator. We apply the Hald data as a numerical example and then make a simulation study to show the main results. It is concluded that the idea of transforming the GRR estimator as a complicated function of the biasing parameters to a linearized version should be paid more attention in the future.     

Some Modifications for Choosing Ridge Parameters

Standard least square regression can produce estimates having a large mean squares error (MSE) when predictor variables are highly correlated or multicollinear. In this article, we propose four modifications to choose the ridge parameter (K) when multicollinearity exists among the columns of the design matrix. The proposed new estimators are extended versions of that suggested by Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameter for regression problems . Commun. Statist. A 34 : 1177 – 1182 . [CSA] [Taylor & Francis Online] , [Google Scholar]). The properties of these estimators are compared with those of Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. . 12 : 55 – 67 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the OLS using the MSE criterion. All estimators under consideration are evaluated using simulation techniques under certain conditions where a number of factors that may affect their properties have been varied. In addition, it is shown that at least one of the proposed estimators either has a smaller MSE than the others or is the next best otherwise.     

A New Ridge Regression Causality Test in the Presence of Multicollinearity

The VAR lag structure applied for the traditional Granger causality (GC) test is always severely affected by multicollinearity due to autocorrelation among the lags. Therefore, as a remedy to this problem we introduce a new Ridge Regression Granger Causality (RRGC) test, which is compared to the GC test by means of Monte Carlo simulations. Based on the simulation study we conclude that the traditional OLS version of the GC test over-rejects the true null hypothesis when there are relatively high (but empirically normal) levels of multicollinearity, while the new RRGC test will remedy or substantially decrease this problem.