A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

The Mean Squared Error of the Generalized Ridge Regression Estimator and the Orientation of β

Newhouse and Oman (1971) identified the orientations with respect to the eigenvectors of X'X of the true coefficient vector of the linear regression model for which the ordinary ridge regression estimator performs best and performs worse when mean squared error is the measure of performance. In this paper the corresponding result is derived for generalized ridge regression for two risk functions: mean squared error and mean squared error of prediction.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.     

Some ridge regression estimators for the zero-inflated Poisson model

The zero-inflated Poisson regression model is commonly used when analyzing economic data that come in the form of non-negative integers since it accounts for excess zeros and overdispersion of the dependent variable. However, a problem often encountered when analyzing economic data that has not been addressed for this model is multicollinearity. This paper proposes ridge regression (RR) estimators and some methods for estimating the ridge parameter k for a non-negative model. A simulation study has been conducted to compare the performance of the estimators. Both mean squared error and mean absolute error are considered as the performance criteria. The simulation study shows that some estimators are better than the commonly used maximum-likelihood estimator and some other RR estimators. Based on the simulation study and an empirical application, some useful estimators are recommended for practitioners.     

EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

Selecting the optimum k in ridge regression

Two ridge rules are proposed for selecting the optimal k in ridge regression . Since the sampling distribution of the proposed rules are mathematically in tractable , a Monte Carlo study is conducted to examine their statisticl properties . Numerical results of the simulations in dicate that the performance of ridge rules depends upon the risk function used. Nevertheless, one of the ridge rules does produce a smaller mean squared error than the least squares estimator with the probability greater than 0.57 for all situations.     

Ridge estimation in generalized linear models and proportional hazards regressions

Conventionally, a ridge parameter is estimated as a function of regression parameters based on ordinary least squares. In this article, we proposed an iterative procedure instead of the one-step or conventional ridge method. Additionally, we construct an indicator that measures the potential degree of improvement in mean squared error when ridge estimates are employed. Simulations show that our methods are appropriate for a wide class of non linear models including generalized linear models and proportional hazards (PHs) regressions. The method is applied to a PH regression with highly collinear covariates in a cancer recurrence study.     

On the almost unbiased ridge regression estimator

The purpose of this paper is two-fold. One is to compare the almost unbiased generalized ridge regression (AUGRR) estimator proposed by Singh, Chaubey and Dwivedi (1986) with the generalized ridge regression (GRR) estimator and with the ordinary least squares (OLS) estimator in terms of the mean squared error criterion. Second is to examine small sample properties of the operational almost unbiased ordinary ridge regression (AUORR) estimator by Monte Carlo experiments.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

Ridge Regression Estimation for Survey Samples

This paper describes procedure for constructing a vector of regression weights. Under the regression superpopulation model, the ridge regression estimator that has minimum model mean squared error is derived. Through a simulation study, we compare the ridge regression weights, regression weights, quadratic programming weights, and raking ratio weights. The ridge regression procedure with weights bounded by zero performed very well.     

A new kind of stochastic restricted biased estimator for logistic regression model

In the logistic regression model, the variance of the maximum likelihood estimator is inflated and unstable when the multicollinearity exists in the data. There are several methods available in literature to overcome this problem. We propose a new stochastic restricted biased estimator. We study the statistical properties of the proposed estimator and compare its performance with some existing estimators in the sense of scalar mean squared criterion. An example and a simulation study are provided to illustrate the performance of the proposed estimator.KEYWORDS: Logistic regression, maximum likelihood estimator, mean squared error matrix, ridge regression, simulation study, stochastic restricted estimatorMathematics Subject Classifications: Primary 62J05, Secondary 62J07     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.     

On exact small sample properties of the minimax generalized ridge regression estimators

The purpose of this paper is to compare sampling performance of the minimax generalized ridge regression estimators considered by Casella (1985) with that of ordinary least squares estimator by numerical calculations of exact mean squared error of these estimators.     

Bounds on minimum mean squared error in ridge regression

The minimum MSE (mean squared error) of ridge regression coefficient estimates (for a given set of eigenvalues and variance) is a function of the transformed coefficient vector. In this paper, the authors prove that the minimum MSE is bounded, for a given coefficient vector length, by the two cases corresponding to the signal completely contained in the component associated with the smallest or largest eigenvalue. The implication for evaluating proposed estimators of the ridge regression biasing parameter is discussed.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Estimation in linear models using gradient descent with early stopping

A new shrinkage estimator of the coefficients of a linear model is derived. The estimator is motivated by the gradient-descent algorithm used to minimize the sum of squared errors and results from early stopping of the algorithm. The statistical properties of the estimator are examined and compared with other well-established methods such as least squares and ridge regression, both analytically and through a simulation study. An important result is that the new estimator is shown to be comparable to other shrinkage estimators in terms of mean squared error of parameters and of predictions, and superior under certain circumstances.Supported by the Greek State Scholarships Foundation     

A simulation study of deterministic ridge estimators

Some deterministic ridge rules are proposed and their finite sample properties are studied. Further, a simulation study is also conducted. Based on the simulation results, the proposed ridge estimators can improve the mean squared error over the least squares estimator, provided that the condition number of correlation matrices in the regression model is large, say at least 1,000.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Ridge estimation in regression problems with autocorrelated errors: A monte carlo study

This paper presents the results of a Monte Carlo study of OLS and GLS based adaptive ridge estimators for regression problems in which the independent variables are collinear and the errors are autocorrelated. It studies the effects of degree of collinearity, magnitude of error variance, orientation of the parameter vector and serial correlation of the independent variables on the mean squared error performance of these estimators. Results suggest that such estimators produce greatly improved performance in favorable portions of the parameter space. The GLS based methods are best when the independent variables are also serially correlated.