A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

On ridge parameter estimators under stochastic subspace hypothesis

This paper considers several estimators for estimating the restricted ridge parameter estimators. A simulation study has been conducted to compare the performance of these estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has positive effect on the mean square error (MSE). However, increasing the value of ρ has negative effect on MSE. When the sample size increases, the MSE decreases even when the correlation between the independent variables is large. Two real life examples have been considered to illustrate the performance of the estimators.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Some ridge regression estimators for the zero-inflated Poisson model

The zero-inflated Poisson regression model is commonly used when analyzing economic data that come in the form of non-negative integers since it accounts for excess zeros and overdispersion of the dependent variable. However, a problem often encountered when analyzing economic data that has not been addressed for this model is multicollinearity. This paper proposes ridge regression (RR) estimators and some methods for estimating the ridge parameter k for a non-negative model. A simulation study has been conducted to compare the performance of the estimators. Both mean squared error and mean absolute error are considered as the performance criteria. The simulation study shows that some estimators are better than the commonly used maximum-likelihood estimator and some other RR estimators. Based on the simulation study and an empirical application, some useful estimators are recommended for practitioners.     

A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

A comparison of biased regression estimators using a pitman nearness criterion

Biased regression estimators have traditionally benn studied using the Mean Square Error (MSE) criterion. Usually these comparisons have been based on the sum of the MSE's of each of the individual parameters, i.e., a scaler valued measure that is the trace of the MSE matrix. However, since this summed MSE does not consider the covariance structure of the estimators, we propose the use of a Pitman Measure of Closeness (PMC) criterion (Keating and Gupta, 1984; Keating and Mason, 1985). In this paper we consider two versions of PMC. One of these compares the estimates and the other compares the resultant predicted values for 12 different regression estimators. These estimators represent three classes of estimators, namely, ridge, shrunken, and principal component estimators. The comparisons of these estimators using the PMC criteria are contrasted with the usual MSE criteria as well as the prediction mean square error. Included in the estimators is a relatively new estimator termed the generalized principal component estimator proposed by Jolliffe. This estimator has previously received little attention in the literature.     

Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses

It is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

Finite sample correction factors for several simple robust estimators of normal standard deviation

Estimation of the standard deviation of a normal population is an important practical problem that in industrial practice must often be done from small and possibly contaminated data sets. Using multiple estimators is useful, as differences in the estimates may indicate whether the data set is contaminated and the form of the contamination. In this paper, finite sample correction factors have been estimated by simulation for several simple robust estimators of the standard deviation of a normal population. The estimators are the median absolute deviation, interquartile range, shortest half interval (Shorth), and median moving range. Finite sample correction factors have also been estimated for the commonly used non-robust estimators: mean absolute deviation and mean moving range. The simulation has been benchmarked against finite sample correction factors for the sample standard deviation and the sample range.     

On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances

In this paper, we study the properties of the preliminary test, restricted and unrestricted ridge regression estimators of the linear regression model with non-normal disturbances. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology, when it is suspected that the regression coefficients may be restricted to a subspace and the regression error is distributed as multivariate t. Accordingly we consider three estimators, namely the Unrestricted Ridge Regression Estimator (URRRE), the Restricted Ridge Regression Estimator (RRRE) and finally the Preliminary test Ridge Regression Estimator (PTRRE). The biases and the mean square error (MSE) of the estimators are derived under the null and alternative hypotheses and compared with the usual estimators. By studying the MSE criterion, the regions of optimahty of the estimators are determined.     

Probability density estimation with data missing at random when covariables are present

This paper addresses the problem of the probability density estimation in the presence of covariates when data are missing at random (MAR). The inverse probability weighted method is used to define a nonparametric and a semiparametric weighted probability density estimators. A regression calibration technique is also used to define an imputed estimator. It is shown that all the estimators are asymptotically normal with the same asymptotic variance as that of the inverse probability weighted estimator with known selection probability function and weights. Also, we establish the mean squared error (MSE) bounds and obtain the MSE convergence rates. A simulation is carried out to assess the proposed estimators in terms of the bias and standard error.     

Small sample properties of a ridge regression estimator when there exist omitted variables

In this paper, we derive the exact formulae for moments of the ridge regression estimator proposed by Huang (Econ Lett 62:261–264, 1999), when there exist omitted variables. We show the conditions under which the ridge regression estimator has smaller mean squared error (MSE) than the ordinary least squares estimator. Based on the exact formulae for moments, we compare the bias and MSE performances of both estimators by numerical evaluations.     

Unbiased estimation of the MSE matrices of improved estimators in linear regression

Stein-rule and other improved estimators have scarcely been used in empirical work. One major reason is that it is not easy to obtain precision measures for these estimators. In this paper, we derive unbiased estimators for both the mean squared error (MSE) and the scaled MSE matrices of a class of Stein-type estimators. Our derivation provides the basis for measuring the estimators' precision and constructing confidence bands. Comparisons are made between these MSE estimators and the least squares covariance estimator. For illustration, the methodology is applied to data on energy consumption.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

A New Estimator for Cox Proportional Hazard Regression Model in Presence of Collinearity

We propose a new approach to estimate the parameters of the Cox proportional hazards model in the presence of collinearity. Generally, a maximum partial likelihood estimator is used to estimate parameters for the Cox proportional hazards model. However, the maximum partial likelihood estimators can be seriously affected by the presence of collinearity since the parameter estimates result in large variances.

In this study, we develop a Liu-type estimator for Cox proportional hazards model parameters and compare it with a ridge regression estimator based on the scalar mean squared error (MSE). Finally, we evaluate its performance through a simulation study.     

On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models

This paper develops alternatives to maximum likelihood estimators (MLE) for logistic regression models and compares the mean squared error (MSE) of the estimators. The MLE for the vector of underlying success probabilities has low MSE only when the true probabilities are extreme (i.e., near 0 or 1). Extreme probabilities correspond to logistic regression parameter vectors which are large in norm. A competing “restricted” MLE and an empirical version of it are suggested as estimators with better performance than the MLE for central probabilities. An approximate EM-algorithm for estimating the restriction is described. As in the case of normal theory ridge estimators, the proposed estimators are shown to be formally derivable by Bayes and empirical Bayes arguments. The small sample operating characteristics of the proposed estimators are compared to the MLE via a simulation study; both the estimation of individual probabilities and of logistic parameters are considered.     

On the use of the Stein variance estimator in the double <Emphasis Type="Italic">k</Emphasis>-class estimator when each individual regression coefficient is estimated

In this paper we consider the double k-class estimator which incorporates the Stein variance estimator. This estimator is called the SVKK estimator. We derive the explicit formula for the mean squared error (MSE) of the SVKK estimator for each individual regression coefficient. It is shown analytically that the MSE performance of the Stein-rule estimator for each individual regression coefficient can be improved by utilizing the Stein variance estimator. Also, MSE’s of several estimators included in a family of the SVKK estimators are compared by numerical evaluations.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

The relative performances of improved ridge estimators and an empirical bayes estimator: some monte carlo results

The relative 'performances of improved ridge estimators and an empirical Bayes estimator are studied by means of Monte Carlo simulations. The empirical Bayes method is seen to perform consistently better in terms of smaller MSE and more accurate empirical coverage than any of the estimators considered here. A bootstrap method is proposed to obtain more reliable estimates of the MSE of ridge esimators. Some theorems on the bootstrap for the ridge estimators are also given and they are used to provide an analytical understanding of the proposed bootstrap procedure. Empirical coverages of the ridge estimators based on the proposed procedure are generally closer to the nominal coverage when compared to their earlier counterparts. In general, except for a few cases, these coverages are still less accurate than the empirical coverages of the empirical Bayes estimator.     

A sufficient condition for the MSE dominance of the positive-part shrinkage estimator when each individual regression coefficient is estimated in a misspecified linear regression model

In this paper, assuming that there exist omitted explanatory variables in the specified model, we derive the exact formula for the mean squared error (MSE) of a general family of shrinkage estimators for each individual regression coefficient. It is shown analytically that when our concern is to estimate each individual regression coefficient, the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators under some conditions even when the relevant regressors are omitted. Also, by numerical evaluations, we showed the effects of our theorem for several specific cases. It is shown that the positive-part shrinkage estimators have smaller MSE than the original shrinkage estimators for wide region of parameter space even when there exist omitted variables in the specified model.