Developing a Liu estimator for the negative binomial regression model: method and application

This paper introduces a new shrinkage estimator for the negative binomial regression model that is a generalization of the estimator proposed for the linear regression model by Liu [A new class of biased estimate in linear regression, Comm. Stat. Theor. Meth. 22 (1993), pp. 393–402]. This shrinkage estimator is proposed in order to solve the problem of an inflated mean squared error of the classical maximum likelihood (ML) method in the presence of multicollinearity. Furthermore, the paper presents some methods of estimating the shrinkage parameter. By means of Monte Carlo simulations, it is shown that if the Liu estimator is applied with these shrinkage parameters, it always outperforms ML. The benefit of the new estimation method is also illustrated in an empirical application. Finally, based on the results from the simulation study and the empirical application, a recommendation regarding which estimator of the shrinkage parameter that should be used is given.     

Performance of some new Liu parameters for the linear regression model

Abstract

This article introduces some Liu parameters in the linear regression model based on the work of Shukur, Månsson, and Sjölander. These methods of estimating the Liu parameter d increase the efficiency of Liu estimator. The comparison of proposed Liu parameters and available methods has done using Monte Carlo simulation and a real data set where the mean squared error, mean absolute error and interval estimation are considered as performance criterions. The simulation study shows that under certain conditions the proposed Liu parameters perform quite well as compared to the ordinary least squares estimator and other existing Liu parameters.     

More on the restricted Liu estimator in the logistic regression model

?iray et al. proposed a restricted Liu estimator to overcome multicollinearity in the logistic regression model. They also used a Monte Carlo simulation to study the properties of the restricted Liu estimator. However, they did not present the theoretical result about the mean squared error properties of the restricted estimator compared to MLE, restricted maximum likelihood estimator (RMLE) and Liu estimator. In this article, we compare the restricted Liu estimator with MLE, RMLE and Liu estimator in the mean squared error sense and we also present a method to choose a biasing parameter. Finally, a real data example and a Monte Carlo simulation are conducted to illustrate the benefits of the restricted Liu estimator.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

On the restricted almost unbiased Liu estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, in the context of biased shrinkage Liu estimation, Chang introduced an almost unbiased Liu estimator in the logistic regression model. Making use of his approach, when some prior knowledge in the form of linear restrictions are also available, we introduce a restricted almost unbiased Liu estimator in the logistic regression model. Statistical properties of this newly defined estimator are derived and some comparison results are also provided in the form of theorems. A Monte Carlo simulation study along with a real data example are given to investigate the performance of this estimator.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

The Almon two parameter estimator for the distributed lag models

The two parameter estimator proposed by Özkale and Kaç?ranlar [The restricted and unrestricted two parameter estimators. Comm Statist Theory Methods. 2007;36(15):2707–2725] is a general estimator which includes the ordinary least squares, the ridge and the Liu estimators as special cases. In the present paper we introduce Almon two parameter estimator based on the two parameter estimation procedure to deal with the problem of multicollinearity for the distiributed lag models. This estimator outperforms the Almon estimator according to the matrix mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters.     

Liu estimation approach to the measurement error models

ABSTRACT

In this paper, we consider the estimation of the parameters of measurement error (ME) models when the multicollinearity exists. To remedy the problem of multicollinearity in ME models, we consider the Liu estimation approach. We define Liu and restricted Liu estimators and also examine the asymptotic properties of proposed estimators in ME models. Moreover, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean squared error criterion.     

On the Stochastic Restricted Almost Unbiased Estimators in Linear Regression Model

In this article, the stochastic restricted almost unbiased ridge regression estimator and stochastic restricted almost unbiased Liu estimator are proposed to overcome the well-known multicollinearity problem in linear regression model. The quadratic bias and mean square error matrix of the proposed estimators are derived and compared. Furthermore, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

On the Restricted Liu Estimator in the Logistic Regression Model

The logistic regression model is used when the response variables are dichotomous. In the presence of multicollinearity, the variance of the maximum likelihood estimator (MLE) becomes inflated. The Liu estimator for the linear regression model is proposed by Liu to remedy this problem. Urgan and Tez and Mansson et al. examined the Liu estimator (LE) for the logistic regression model. We introduced the restricted Liu estimator (RLE) for the logistic regression model. Moreover, a Monte Carlo simulation study is conducted for comparing the performances of the MLE, restricted maximum likelihood estimator (RMLE), LE, and RLE for the logistic regression model.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Robust two parameter ridge M-estimator for linear regression

The problem of multicollinearity and outliers in the data set produce undesirable effects on the ordinary least squares estimator. Therefore, robust two parameter ridge estimation based on M-estimator (ME) is introduced to deal with multicollinearity and outliers in the y-direction. The proposed estimator outperforms ME, two parameter ridge estimator and robust ridge M-estimator according to mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

A new linearized ridge Poisson estimator in the presence of multicollinearity

Poisson regression is a very commonly used technique for modeling the count data in applied sciences, in which the model parameters are usually estimated by the maximum likelihood method. However, the presence of multicollinearity inflates the variance of maximum likelihood (ML) estimator and the estimated parameters give unstable results. In this article, a new linearized ridge Poisson estimator is introduced to deal with the problem of multicollinearity. Based on the asymptotic properties of ML estimator, the bias, covariance and mean squared error of the proposed estimator are obtained and the optimal choice of shrinkage parameter is derived. The performance of the existing estimators and proposed estimator is evaluated through Monte Carlo simulations and two real data applications. The results clearly reveal that the proposed estimator outperforms the existing estimators in the mean squared error sense.KEYWORDS: Poisson regression, multicollinearity, ridge Poisson estimator, linearized ridge regression estimator, mean squared errorMathematics Subject Classifications: 62J07, 62F10     

Liu estimator in partly linear regression models with correlated errors

This article is concerned with the parameter estimation in partly linear regression models when the errors are dependent. To overcome the multicollinearity problem, a generalized Liu estimator is proposed. The theoretical properties of the proposed estimator and its relationship with some existing methods designed for partly linear models are investigated. Finally, a hypothetical data is conducted to illustrate some of the theoretical results.     

A new stochastic mixed Liu estimator in linear regression model

Abstract

To overcome multicollinearity, a new stochastic mixed Liu estimator is presented and its efficiency is considered. We also compare the proposed estimators in the sense of matrix mean squared error criteria. Finally a numerical example and a simulation study are given to show the performance of the estimators.     

Two-stage Liu estimator in a simultaneous equations model

Two-stage least squares estimation in a simultaneous equations model has several desirable properties under the problem of multicollinearity. So, various kinds of improved estimation techniques can be developed to deal with the problem of multicollinearity. One of them is ridge regression estimation that can be applied at both stages and defined in Vinod and Ullah [Recent advances in regression methods. New York: Marcel Dekker; 1981]. We propose three different kinds of Liu estimators that are named by their implementation stages. Mean square errors are derived to compare the performances of the mentioned estimators and two different choices of the biasing parameter are offered. Moreover, a numerical example is given with a data analysis based on the Klein Model I and a Monte Carlo experiment is conducted.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Liu-type estimator in semiparametric partially linear additive models

Partially linear additive model is useful in statistical modelling as a multivariate nonparametric fitting technique. This paper considers statistical inference for the semiparametric model in the presence of multicollinearity. Based on the profile least-squares (PL) approach and Liu estimation method, we propose a PL Liu estimator for the parametric component. When some additional linear restrictions on the parametric component are available, the corresponding restricted Liu estimator for the parametric component is constructed. The properties of the proposed estimators are derived. Some simulations are conducted to assess the performance of the proposed procedures and the results are satisfactory. Finally, a real data example is analysed.     

On preliminary test almost unbiased two-parameter estimator in linear regression model with student's t errors

In this paper, the preliminary test approach to the estimation of the linear regression model with student's t errors is considered. The preliminary test almost unbiased two-parameter estimator is proposed, when it is suspected that the regression parameter may be restricted to a constraint. The quadratic biases and quadratic risks of the proposed estimators are derived and compared under both null and alternative hypotheses. The conditions of superiority of the proposed estimators for departure parameter and biasing parameters k and d are derived, respectively. Furthermore, a real data example and a Monte Carlo simulation study are provided to illustrate some of the theoretical results.