On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.     

New Shrinkage Parameters for the Liu-type Logistic Estimators

The binary logistic regression is a widely used statistical method when the dependent variable has two categories. In most of the situations of logistic regression, independent variables are collinear which is called the multicollinearity problem. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Therefore, this article introduces new shrinkage parameters for the Liu-type estimators in the Liu (2003) in the logistic regression model defined by Huang (2012) in order to decrease the variance and overcome the problem of multicollinearity. A Monte Carlo study is designed to show the goodness of the proposed estimators over MLE in the sense of mean squared error (MSE) and mean absolute error (MAE). Moreover, a real data case is given to demonstrate the advantages of the new shrinkage parameters.     

Restricted Ridge Estimators of the Parameters in Semiparametric Regression Model

In this article, we introduce a ridge estimator for the vector of parameters β in a semiparametric model when additional linear restrictions on the parameter vector are assumed to hold. We also obtain the semiparametric restricted ridge estimator for the parametric component in the semiparametric regression model. The ideas in this article are illustrated with a data set consisting of housing prices and through a comparison of the performances of the proposed and related estimators via a Monte Carlo simulation.     

A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

Some Modifications for Choosing Ridge Parameters

Standard least square regression can produce estimates having a large mean squares error (MSE) when predictor variables are highly correlated or multicollinear. In this article, we propose four modifications to choose the ridge parameter (K) when multicollinearity exists among the columns of the design matrix. The proposed new estimators are extended versions of that suggested by Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameter for regression problems . Commun. Statist. A 34 : 1177 – 1182 . [CSA] [Taylor & Francis Online] , [Google Scholar]). The properties of these estimators are compared with those of Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. . 12 : 55 – 67 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the OLS using the MSE criterion. All estimators under consideration are evaluated using simulation techniques under certain conditions where a number of factors that may affect their properties have been varied. In addition, it is shown that at least one of the proposed estimators either has a smaller MSE than the others or is the next best otherwise.     

Ridge Regression Estimation for Survey Samples

This paper describes procedure for constructing a vector of regression weights. Under the regression superpopulation model, the ridge regression estimator that has minimum model mean squared error is derived. Through a simulation study, we compare the ridge regression weights, regression weights, quadratic programming weights, and raking ratio weights. The ridge regression procedure with weights bounded by zero performed very well.     

Performance of Some Ridge Parameters for Probit Regression: With Application to Swedish Job Search Data

In ridge regression, the estimation of the ridge parameter is an important issue. This article generalizes some methods for estimating the ridge parameter for probit ridge regression (PRR) model based on the work of Kibria et al. (2011 Kibria, B. M. G., Månsson, K. and Shukur, G. 2011. Performance of some logistic ridge regression parameters. Computational Economics, DOI: 10.1007/s10614-011-9275-x [Google Scholar]). The performance of these new estimators is judged by calculating the mean squared error (MSE) using Monte Carlo simulations. In the design of the experiment, we chose to vary the sample size and the number of regressors. Furthermore, we generate explanatory variables that are linear combinations of other regressors, which is a common situation in economics. In an empirical application regarding Swedish job search data, we also illustrate the benefits of the new method.     

Some Asymptotic Properties of Ridge Regression in a System of Seemingly Unrelated Regression Equations

A discussion is made of asymptotic properties of an Operational Ordinary Ridge Regression estimator and comparison is made with the Operational Generalized Least Squares estimator. Also, some simulation experiments are carried showing efficiency gains can be made through the use of de Ridge estimator.     

MSE Performance of a Heterogeneous Pre-Test Ridge Regression Estimator

This article describes how diagnostic procedures were derived for symmetrical nonlinear regression models, continuing the work carried out by Cysneiros and Vanegas (2008 Cysneiros , F. J. A. , Vanegas , L. H. ( 2008 ). Residuals and their statistical properties in symmetrical nonlinear models . Statist. Probab. Lett. 78 : 3269 – 3273 .[Crossref], [Web of Science ®] , [Google Scholar]) and Vanegas and Cysneiros (2010 Vanegas , L. H. , Cysneiros , F. J. A. ( 2010 ). Assesment of diagnostic procedures in symmetrical nonlinear regression models . Computat. Statist. Data Anal. 54 : 1002 – 1016 .[Crossref], [Web of Science ®] , [Google Scholar]), who showed that the parameters estimates in nonlinear models are more robust with heavy-tailed than with normal errors. In this article, we focus on assessing if the robustness of this kind of models is also observed in the inference process (i.e., partial F-test). Symmetrical nonlinear regression models includes all symmetric continuous distributions for errors covering both light- and heavy-tailed distributions such as Student-t, logistic-I and -II, power exponential, generalized Student-t, generalized logistic, and contaminated normal. Firstly, a statistical test is shown to evaluating the assumption that the error terms all have equal variance. The results of simulation studies which describe the behavior of the test for heteroscedasticity proposed in the presence of outliers are then given. To assess the robustness of inference process, we present the results of a simulation study which described the behavior of partial F-test in the presence of outliers. Also, some diagnostic procedures are derived to identify influential observations on the partial F-test. As ilustration, a dataset described in Venables and Ripley (2002 Venables , W. N. , Ripley , B. D. ( 2002 ). Modern Applied with S. , 4th ed. New York : Springer .[Crossref] , [Google Scholar]), is also analyzed.     

A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions

Based on the work of Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameters for regression problems . Communications in Statistics – Theory and Methods 34 : 1177 – 1182 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi , M. , Khalaf , G. , Shukur , G. ( 2006 ). Some modifications for choosing ridge parameters . Communications in Statistics – Theory and Methods 35 : 2005 – 2020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2010 Muniz , G. , Kibria , B. M. G. , Shukur , G. ( 2010 ). On developing ridge regression parameters: a graphical Investigation. Submitted for Publication . [Google Scholar]), this article considers several estimators for estimating the ridge parameter k. This article differs from aforementioned articles in three ways: (1) Data are generated from Normal, Student's t, and F distributions with appropriate degrees of freedom; (2) The number of regressors considered are from 4–12 instead of 2–4, which are the usual practice; (3) Both mean square error (MSE) and prediction sum of square (PRESS) are considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has negative effect on the MSE and PRESS. However, increasing the number of regressors has positive effect on MSE and PRESS. When the sample size increases the MSE decreases even when the correlation between the independent variables is large. It is interesting to note that the dominance pictures of the estimators are remained the same under both the MSE and PRESS criterion. However, the performance of the estimators depends on the choice of the assumption of the error distribution of the regression model.     

Jackknifing the Ridge Regression Estimator: A Revisit

Singh et al. (1986 Singh, B., Chaubey, Y.P., Dwivedi, T.D. (1986). An almost unbiased ridge estimator. Sankhya B48: 34236. [Google Scholar]) proposed an almost unbiased ridge estimator using Jackknife method that required transformation of the regression parameters. This article shows that the same method can be used to derive the Jackknifed ridge estimator of the original (untransformed) parameter without transformation. This method also leads in deriving easily the second-order Jackknifed ridge that may reduce the bias further. We further investigate the performance of these estimators along with a recent method by Batah et al. (2008 Batah, F. S.M., Ramanathan, T.V., Gore, S.D. (2008). The efficiency of modified Jack-knife and ridge type regression estimators: a comparison. Surv. Math. Applic. 3:111122. [Google Scholar]) called modified Jackknifed ridge theoretically as well as numerically.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

On the Choice of the Ridge Parameter: A Maximum Entropy Approach

In this article, a new method to estimate the ridge parameter, based on the ridge trace and an analytical method borrowed from maximum entropy, is presented. The performance of the new estimator is illustrated through a Monte Carlo simulation study and an empirical application to the well-known Portland cement data set.     

A Monte Carlo Study of Recent Ridge Parameters

A number of procedures have been developed for finding biased estimators of regression parameters. One of these procedures is the ridge regression. In this article, a new approach to obtain the ridge parameter K is suggested and then evaluated by Monte Carlo simulations. A number of different models are investigated for different number of observations, the strength of correlation between the explanatory variables, and distribution of the error terms. The mean squared error (MSE) criterion is used to examine the performance of the proposed estimators when compared with other well-known estimators. Under certain conditions, it is shown that at least one of the proposed estimators have a smaller MSE than the ordinary least squared estimator (OLS) and Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) estimator (HK).     

Choosing Ridge Parameter for Regression Problems

ABSTRACT

Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) introduced the ridge regression estimator as an alternative to the ordinary least squares estimator in the presence of multicollinearity. In this article, a new approach for choosing the ridge parameter (K), when multicollinearity among the columns of the design matrix exists, is suggested and evaluated by simulation techniques, in terms of mean squared errors (MSE). A number of factors that may affect the properties of these methods have been varied. The MSE from this approach has shown to be smaller than using Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) in almost all situations.     

Modified Ridge Parameters for Seemingly Unrelated Regression Model

In this article, we modify a number of new biased estimators of seemingly unrelated regression (SUR) parameters which are developed by Alkhamisi and Shukur (2008 Alkhamisi , M. A. , Shukur , G. ( 2008 ). Developing ridge parameters for SUR model . Commun. Statist. Theor. Meth. 37 ( 4 ): 544 – 564 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), AS, when the explanatory variables are affected by multicollinearity. Nine estimators of the ridge parameters have been modified and compared in terms of the trace mean squared error (TMSE) and (PR) criterion. The results from this extended study are the also compared with those founded by AS. A simulation study has been conducted to compare the performance of the modified estimators of the ridge parameters. The results showed that under certain conditions the performance of the multivariate ridge regression estimators based on SUR ridge R _MSmax is superior to other estimators in terms of TMSE and PR criterion. In large samples and when the collinearity between the explanatory variables is not high, the unbiased SUR, estimator produces a smaller TMSEs.     

Consistency of Ridge Function Fields for Varying Nonparametric Regression

A nonparametric regression model proposed in Pelletier and Frouin (2006 Pelletier , B. , Frouin , R. ( 2006 ). Remote sensing of phytoplankton chlorophyll-a concentration by ridge function fields . App. Optics 45 : 784 – 798 . [Google Scholar]) as a solution to the geophysical problem of ocean color remote sensing is studied. The model, called ridge function field, combines a regression estimate in the form of a superposition of ridge functions, or which is equivalent to a neural network, with the idea pertaining to varying-coefficients models, where the parameters of a parametric family are allowed to vary with other variables. Under mild assumptions on the underlying distribution of the data, the strong universal consistency of the least-squares ridge function fields estimate is established.     

Penalized Maximum Likelihood Principle for Choosing Ridge Parameter

We consider the problem of choosing the ridge parameter. Two penalized maximum likelihood (PML) criteria based on a distribution-free and a data-dependent penalty function are proposed. These PML criteria can be considered as “continuous” versions of AIC. A systematic simulation is conducted to compare the suggested criteria to several existing methods. The simulation results strongly support the use of our method. The method is also applied to two real data sets.