Performance of ridge estimator in inverse Gaussian regression model

The presence of multicollinearity among the explanatory variables has undesirable effects on the maximum likelihood estimator (MLE). Ridge estimator (RE) is a widely used estimator in overcoming this issue. The RE enjoys the advantage that its mean squared error (MSE) is less than that of MLE. The inverse Gaussian regression (IGR) model is a well-known model in the application when the response variable positively skewed. The purpose of this paper is to derive the RE of the IGR under multicollinearity problem. In addition, the performance of this estimator is investigated under numerous methods for estimating the ridge parameter. Monte Carlo simulation results indicate that the suggested estimator performs better than the MLE estimator in terms of MSE. Furthermore, a real chemometrics dataset application is utilized and the results demonstrate the excellent performance of the suggested estimator when the multicollinearity is present in IGR model.     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

Some new methods to solve multicollinearity in logistic regression

The binary logistic regression is a widely used statistical method when the dependent variable is binary or dichotomous. In some of the situations of logistic regression, independent variables are collinear which leads to the problem of multicollinearity. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Thus, this article introduces new methods to estimate the shrinkage parameters of Liu-type logistic estimator proposed by Inan and Erdogan (2013 Inan, D., Erdogan, B. E. (2013). Liu-type logistic estimator. Communications in Statistics-Simulation and Computation 42(7):1578–1586. [Google Scholar]) which is a generalization of the Liu-type estimator defined by Liu (2003 Liu, K. (2003). Using Liu-type estimator to combat collinearity. Communications in Statistics: Theory and Methods 32(5):1009–1020. [Google Scholar]) for the linear model. A Monte Carlo study is used to show the effectiveness of the proposed methods over MLE using the mean squared error (MSE) and mean absolute error (MAE) criteria. A real data application is illustrated to show the benefits of new methods. According to the results of the simulation and application proposed methods have better performance than MLE.     

A non stochastic ridge regression estimator and comparison with the James-Stein estimator

Abstract

This article presents a non-stochastic version of the Generalized Ridge Regression estimator that arises from a discussion of the properties of a Generalized Ridge Regression estimator whose shrinkage parameters are found to be close to their upper bounds. The resulting estimator takes the form of a shrinkage estimator that is superior to both the Ordinary Least Squares estimator and the James-Stein estimator under certain conditions. A numerical study is provided to investigate the range of signal to noise ratio under which the new estimator dominates the James-Stein estimator with respect to the prediction mean square error.     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.     

Minimum Φ-divergence estimator in logistic regression models

A general class of minimum distance estimators for logistic regression models based on the ϕ-divergence measures is introduced: The minimum ϕ-divergence estimator, which is seen to be a generalization of the maximum likelihood estimator. Its asymptotic properties are studied as well as its behaviour in small samples throught a simulation study. This work was supported partially by Grant DGI (BMF2003-00892).     

Ridge estimator with correlated errors and two-stage ridge estimator under inequality restrictions

Liew (1976a Liew, C.K. (1976a). A two-stage least-squares estimation with inequality restrictions on parameters. Rev. Econ. Stat. LVIII(2):234–238.[Crossref], [Web of Science ®] , [Google Scholar]) introduced generalized inequality constrained least squares (GICLS) estimator and inequality constrained two-stage and three-stage least squares estimators by reducing primal–dual relation to problem of Dantzig and Cottle (1967 Dantzig, G.B., Cottle, R.W. (1967). Positive (semi-) definite matrices and mathematical programming. In: Abadie, J., ed. Nonlinear Programming (pp. 55–73). Amsterdam: North Holland Publishing Co. [Google Scholar]), Cottle and Dantzig (1974 Cottle, R.W., Dantzig, G.B. (1974). Complementary pivot of mathematical programming. In: Dantzig, G.B., Eaves, B.C., eds. Studies in OptimizationVol. 10. Washington: Mathematical Association of America. [Google Scholar]) and solving with Lemke (1962 Lemke, C.E. (1962). A method of solution for quadratic programs. Manage. Sci. 8(4):442–453.[Crossref], [Web of Science ®] , [Google Scholar]) algorithm. The purpose of this article is to present inequality constrained ridge regression (ICRR) estimator with correlated errors and inequality constrained two-stage and three-stage ridge regression estimators in the presence of multicollinearity. Untruncated variance–covariance matrix and mean square error are derived for the ICRR estimator with correlated errors, and its superiority over the GICLS estimator is examined via Monte Carlo simulation.     

A new biased estimator based on ridge estimation

In this paper we introduce a new biased estimator for the vector of parameters in a linear regression model and discuss its properties. We show that our new biased estimator is superior, in the mean square error(mse) sense, to the ordinary least squares (OLS) estimator, the ordinary ridge regression (ORR) estimator and the Liu estimator. We also compare the performance of our new biased estimator with two other special Liu-type estimators proposed in Liu (2003). We illustrate our findings with a numerical example based on the widely analysed dataset on Portland cement.     

Optimal generalized logistic estimator

In this paper, we propose a new efficient estimator namely Optimal Generalized Logistic Estimator (OGLE) for estimating the parameter in a logistic regression model when there exists multicollinearity among explanatory variables. Asymptotic properties of the proposed estimator are also derived. The performance of the proposed estimator over the other existing estimators in respect of Scalar Mean Square Error criterion is examined by conducting a Monte Carlo simulation.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Comment on a generalized stochastic restricted ridge regression estimator

In this note, we make some comments about the paper of Alheety and Kibria (2014 Alheety, M.I., Kibria, B.M.G. (2014). A generalized stochastic restricted ridge regression estimator. Commun. Stat. Theor. Meth. 43:4415–4427.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) and correct the wrongly proved Theorems in that paper.     

A restricted Liu estimator for binary regression models and its application to an applied demand system

In this article, we propose a restricted Liu regression estimator (RLRE) for estimating the parameter vector, β, in the presence of multicollinearity, when the dependent variable is binary and it is suspected that β may belong to a linear subspace defined by Rβ?=?r. First, we investigate the mean squared error (MSE) properties of the new estimator and compare them with those of the restricted maximum likelihood estimator (RMLE). Then we suggest some estimators of the shrinkage parameter, and a simulation study is conducted to compare the performance of the different estimators. Finally, we show the benefit of using RLRE instead of RMLE when estimating how changes in price affect consumer demand for a specific product.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.     

Liu-Type Logistic Estimator

It is known that multicollinearity inflates the variance of the maximum likelihood estimator in logistic regression. Especially, if the primary interest is in the coefficients, the impact of collinearity can be very serious. To deal with collinearity, a ridge estimator was proposed by Schaefer et al. The primary interest of this article is to introduce a Liu-type estimator that had a smaller total mean squared error (MSE) than the Schaefer's ridge estimator under certain conditions. Simulation studies were conducted that evaluated the performance of this estimator. Furthermore, the proposed estimator was applied to a real-life dataset.     

Seemingly unrelated regressions with covariance matrix of cross-equation ridge regression residuals

Generalized least squares estimation of a system of seemingly unrelated regressions is usually a two-stage method: (1) estimation of cross-equation covariance matrix from ordinary least squares residuals for transforming data, and (2) application of least squares on transformed data. In presence of multicollinearity problem, conventionally ridge regression is applied at stage 2. We investigate the usage of ridge residuals at stage 1, and show analytically that the covariance matrix based on the least squares residuals does not always result in more efficient estimator. A simulation study and an application to a system of firms' gross investment support our finding.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

Some Modifications for Choosing Ridge Parameters

Standard least square regression can produce estimates having a large mean squares error (MSE) when predictor variables are highly correlated or multicollinear. In this article, we propose four modifications to choose the ridge parameter (K) when multicollinearity exists among the columns of the design matrix. The proposed new estimators are extended versions of that suggested by Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameter for regression problems . Commun. Statist. A 34 : 1177 – 1182 . [CSA] [Taylor & Francis Online] , [Google Scholar]). The properties of these estimators are compared with those of Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. . 12 : 55 – 67 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the OLS using the MSE criterion. All estimators under consideration are evaluated using simulation techniques under certain conditions where a number of factors that may affect their properties have been varied. In addition, it is shown that at least one of the proposed estimators either has a smaller MSE than the others or is the next best otherwise.     

Performance of some ridge regression estimators for the multinomial logit model

This article considers several estimators for estimating the ridge parameter k for multinomial logit model based on the work of Khalaf and Shukur (2005 Khalaf, G., and G. Shukur. 2005. Choosing ridge parameters for regression problems. Commun. Statist. Theor. Meth., 34:1177–1182.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi, M., G. Khalaf, and G. Shukur. 2006. Some modifications for choosing ridge parameters. Commun. Statist. Theor. Meth. 35:2005–2020.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2012 Muniz, G., B. M. G. Kibria, K. Månsson, and G. Shukur. 2012. On developing ridge regression parameters: A graphical investigation. in SORT. 36: 115–138.[Web of Science ®] , [Google Scholar]). The mean square error (MSE) is considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that increasing the correlation between the independent variables and the number of regressors has negative effect on the MSE. However, when the sample size increases the MSE decreases even when the correlation between the independent variables is large. Based on the minimum MSE criterion some useful estimators for estimating the ridge parameter k are recommended for the practitioners.