On the Choice of the Ridge Parameter: A Maximum Entropy Approach

In this article, a new method to estimate the ridge parameter, based on the ridge trace and an analytical method borrowed from maximum entropy, is presented. The performance of the new estimator is illustrated through a Monte Carlo simulation study and an empirical application to the well-known Portland cement data set.     

Selection of the Ridge Parameter Using Mathematical Programming

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates and their standard errors in order to reach acceptable results. Selection of the ridge parameter was done using several subjective and objective techniques that are concerned with certain criteria. In this study, selection of the ridge parameter depends on other important statistical measures to reach a better value of the ridge parameter. The proposed ridge parameter selection technique depends on a mathematical programming model and the results are evaluated using a simulation study. The performance of the proposed method is good when the error variance is greater than or equal to one; the sample consists of 20 observations, the number of explanatory variables in the model is 2, and there is a very strong correlation between the two explanatory variables.     

A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

Flexible Tweedie regression models for continuous data

Tweedie regression models (TRMs) provide a flexible family of distributions to deal with non-negative right-skewed data and can handle continuous data with probability mass at zero. Estimation and inference of TRMs based on the maximum likelihood (ML) method are challenged by the presence of an infinity sum in the probability function and non-trivial restrictions on the power parameter space. In this paper, we propose two approaches for fitting TRMs, namely quasi-likelihood (QML) and pseudo-likelihood (PML). We discuss their asymptotic properties and perform simulation studies to compare our methods with the ML method. We show that the QML method provides asymptotically efficient estimation for regression parameters. Simulation studies showed that the QML and PML approaches present estimates, standard errors and coverage rates similar to the ML method. Furthermore, the second-moment assumptions required by the QML and PML methods enable us to extend the TRMs to the class of quasi-TRMs in Wedderburn's style. It allows to eliminate the non-trivial restriction on the power parameter space, and thus provides a flexible regression model to deal with continuous data. We provide an R implementation and illustrate the application of TRMs using three data sets.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.     

Two-parameter ridge estimator in the binary logistic regression

The binary logistic regression is a commonly used statistical method when the outcome variable is dichotomous or binary. The explanatory variables are correlated in some situations of the logit model. This problem is called multicollinearity. It is known that the variance of the maximum likelihood estimator (MLE) is inflated in the presence of multicollinearity. Therefore, in this study, we define a new two-parameter ridge estimator for the logistic regression model to decrease the variance and overcome multicollinearity problem. We compare the new estimator to the other well-known estimators by studying their mean squared error (MSE) properties. Moreover, a Monte Carlo simulation is designed to evaluate the performances of the estimators. Finally, a real data application is illustrated to show the applicability of the new method. According to the results of the simulation and real application, the new estimator outperforms the other estimators for all of the situations considered.     

Seemingly unrelated regressions with covariance matrix of cross-equation ridge regression residuals

Generalized least squares estimation of a system of seemingly unrelated regressions is usually a two-stage method: (1) estimation of cross-equation covariance matrix from ordinary least squares residuals for transforming data, and (2) application of least squares on transformed data. In presence of multicollinearity problem, conventionally ridge regression is applied at stage 2. We investigate the usage of ridge residuals at stage 1, and show analytically that the covariance matrix based on the least squares residuals does not always result in more efficient estimator. A simulation study and an application to a system of firms' gross investment support our finding.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Confidence Interval for Shrinkage Parameters in Ridge Regression

In ridge regression, the estimation of ridge parameter k is an important problem. There are several methods available in the literature to do this job some what efficiently. However, no attempts were made to suggest a confidence interval for the ridge parameter using the knwoledge from the data. In this article, we propose a data dependent confidence interval for the ridge parameter k. The method of obtaining the confidence interval is illustrated with the help of a data set. A simulation study indicates that the empirical coverage probability of the suggested confidence intervals are quite high.     

A New Asymmetric Interaction Ridge (AIR) Regression Method

Despite that interaction terms are standard tools of regression analysis, the side effects of the inclusion of these terms in models estimated by ordinary least squares (OLS) are yet not fully penetrated. The inclusion of interaction effects induces multicollinearity problems since all non zero values are equal between the interaction term and the regressor. In this article, we propose a procedure to remedy this problem by the use of new ridge regression (RR) shrinkage parameters—which we call the asymmetric interaction ridge (AIR) regression method. By means of Monte Carlo simulations we evaluate both OLS and AIR using the mean square error (MSE) performance criterion. The result from the simulation study confirms our hypothesis that AIR always should be preferred to OLS since it has a lower estimated MSE. Moreover, the advantages of our new method are demonstrated in an empirical application where positive asymmetric price transmission effects are exposed for the mortgage interest rates of Handelsbanken Stadshypotek. It is observed that the mortgage interest rates increase more fully and rapidly to an increase in the bank's borrowing costs than to a decrease. This asymmetry is defined as positive asymmetric price transmission (APT).     

A simple way to deal with multicollinearity

Despite the long and frustrating history of struggling with the wrong signs or other types of implausible estimates under multicollinearity, it turns out that the problem can be solved in a surprisingly easy way. This paper presents a simple approach that ensures both statistically sound and theoretically consistent estimates under multicollinearity. The approach is simple in the sense that it requires nothing but basic statistical methods plus a piece of a priori knowledge. In addition, the approach is robust even to the extreme case when the a priori knowledge is wrong. A simulation test shows astonishingly superior performance of the method in repeated samples comparing to the OLS, the Ridge Regression and the Dropping-Variable approach.     

Comparisons of the alternative biased estimators for the distributed lag models

The finite distributed lag models include highly correlated variables as well as lagged and unlagged values of the same variables. Some problems are faced for this model when applying the ordinary least squares (OLS) method or econometric models such as Almon and Koyck models. The primary aim of this study is to compare performances of alternative estimators to the OLS estimator defined by combining the Almon estimator with some estimators using Almon (1965) data. A simulation study with different model parameters is performed and the estimators are compared according to the root mean square error (RMSE) and prediction mean square error (PMSE).     

A simulation study of least squares and ridge estimaors for normal and nonnormal autocorrelated disturbances

The purpose of this paper is to examine the small sample properties of various ridge estimators along with least squares, in some special settings.Specifically, we consider a first order autoregressive structuure for normal and nonnormal disturbances, and report on a Monte Carlo study the small sample behavior of these estimators according to the criteria of bias and dispersion.The results suggest that under all the examined settings and for all the criteria used the HKB estimator exhibited a superior performance compared to the other estimators, while the LS and LW estimators gave consistently poor results.Also if the error term is only moderately autocorrelated the performance of the ridge estimators that do not account for autocorrelation outperform their counterparts as well as least squares that account for autocorrelation.     

Ridge Regression – A Simulation Study

In this article we assess the suitability of two new ridge estimators by means of a simulation study. We compare these estimators with well-known ridge estimators. We also make direct comparisons between the ordinary least squares (OLS) estimator and the ridge estimators by using ratio of the average total mean square error of the OLS estimator and the ridge estimators. We find that the new estimators perform well under certain conditions.     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.