Liu estimation approach to the measurement error models

ABSTRACT

In this paper, we consider the estimation of the parameters of measurement error (ME) models when the multicollinearity exists. To remedy the problem of multicollinearity in ME models, we consider the Liu estimation approach. We define Liu and restricted Liu estimators and also examine the asymptotic properties of proposed estimators in ME models. Moreover, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean squared error criterion.     

A new stochastic mixed Liu estimator in linear regression model

Abstract

To overcome multicollinearity, a new stochastic mixed Liu estimator is presented and its efficiency is considered. We also compare the proposed estimators in the sense of matrix mean squared error criteria. Finally a numerical example and a simulation study are given to show the performance of the estimators.     

Preliminary test almost unbiased two-parameter estimators with student’s t errors and conflicting test statistics

Abstract

In this paper, we consider the preliminary test approach to the estimation of the regression parameter in a multiple regression model under multicollinearity situation. The preliminary test almost unbiased two-parameter estimators based on the Wald, the Likelihood ratio, and the Lagrangian multiplier tests are given, when it is suspected that the regression parameter may be restricted to a subspace and the regression error is distributed with multivariate Student’s t errors. The bias and quadratic risk of the proposed estimators are derived and compared. Furthermore, a Monte Carlo simulation is provided to illustrate some of the theoretical results.     

Improved ridge estimators in a linear regression model

In this paper, the notion of the improved ridge estimator (IRE) is put forward in the linear regression model y=X β+e. The problem arises if augmenting the equation 0=c′α+ε instead of 0=C α+? to the model. Three special IREs are considered and studied under the mean-squared error criterion and the prediction error sum of squares criterion. The simulations demonstrate that the proposed estimators are effective and recommendable, especially when multicollinearity is severe.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Using Heteroscedasticity-Consistent Standard Errors for the Linear Regression Model with Correlated Regressors

The use of heteroscedasticity-consistent covariance matrix (HCCM) estimators is very common in practice to draw correct inference for the coefficients of a linear regression model with heteroscedastic errors. However, in addition to the problem of heteroscedasticity, linear regression models may also be plagued with some considerable degree of collinearity among the regressors when two or more regressors are considered. This situation causes many adverse effects on the least squares measures and alternatively, the ordinary ridge regression method is used as a common practice. But in the available literature, the problems of multicollinearity and heteroscedasticity have not been discussed as a combined issue especially, for the inference of the regression coefficients. The present article addresses the inference about the regression coefficients taking both the issues of multicollinearity and heteroscedasticity into account and suggests the use of HCCM estimators for the ridge regression. This article proposes t- and F-tests, based on these HCCM estimators, that perform adequately well in the numerical evaluation of the Monte Carlo simulations.     

A Generalized Diagonal Ridge-type Estimator in Linear Regression

This article introduces a general class of biased estimator, namely a generalized diagonal ridge-type (GDR) estimator, for the linear regression model when multicollinearity occurs. The estimator represents different kinds of biased estimators when different parameters are obtained. Some properties of this estimator are discussed and an iterative procedure is provided for selecting the parameters. A Monte Carlo simulation study and an application show that the GDR estimator performs much better than the ordinary least squares (OLS) estimator under the mean square error (MSE) criterion when severe multicollinearity is present.     

A Class of s–K Type Principal Components Estimators in the Linear Model

In this article, we introduce a new class of estimators called the s–K type principal components estimators to combat multicollinearity, which include the principal components regression (PCR) estimator, the r–k estimator and the s–K estimator as special cases. Necessary and sufficient conditions for the superiority of the new estimator over the PCR estimator, the r–k estimator and the s–K estimator are derived in the sense of the mean squared error matrix criterion. A Monte Carlo simulation study and a numerical example are given to illustrate the performance of the proposed estimator.     

The generalized maximum Tsallis entropy estimators and applications to the Portland cement dataset

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.     

Jackknife Estimation for Smooth Functions of the Parametric Component in Partially Linear Regression Models

Abstract

It is known that due to the existence of the nonparametric component, the usual estimators for the parametric component or its function in partially linear regression models are biased. Sometimes this bias is severe. To reduce the bias, we propose two jackknife estimators and compare them with the naive estimator. All three estimators are shown to be asymptotically equivalent and asymptotically normally distributed under some regularity conditions. However, through simulation we demonstrate that the jackknife estimators perform better than the naive estimator in terms of bias when the sample size is small to moderate. To make our results more useful, we also construct consistent estimators of the asymptotic variance, which are robust against heterogeneity of the error variances.     

On the euclidean distance between biased estimators

The purpose of this paper is to extend the results achieved by Hsuan (1981) to a wide class of biased estimators. It is shown that in the case of multicollinearity ridge and Iteration estimator can be made very close to the principal component estimator whereas the shrunken estimator does not have this property.     

Parameter estimation approaches to tackling measurement error and multicollinearity in ordinal probit models

Abstract

The regression model with ordinal outcome has been widely used in a lot of fields because of its significant effect. Moreover, predictors measured with error and multicollinearity are long-standing problems and often occur in regression analysis. However there are not many studies on dealing with measurement error models with generally ordinal response, even fewer when they suffer from multicollinearity. The purpose of this article is to estimate parameters of ordinal probit models with measurement error and multicollinearity. First, we propose to use regression calibration and refined regression calibration to estimate parameters in ordinal probit models with measurement error. Second, we develop new methods to obtain estimators of parameters in the presence of multicollinearity and measurement error in ordinal probit model. Furthermore we also extend all the methods to quadratic ordinal probit models and talk about the situation in ordinal logistic models. These estimators are consistent and asymptotically normally distributed under general conditions. They are easy to compute, perform well and are robust against the normality assumption for the predictor variables in our simulation studies. The proposed methods are applied to some real datasets.     

A restricted Liu estimator for binary regression models and its application to an applied demand system

In this article, we propose a restricted Liu regression estimator (RLRE) for estimating the parameter vector, β, in the presence of multicollinearity, when the dependent variable is binary and it is suspected that β may belong to a linear subspace defined by Rβ?=?r. First, we investigate the mean squared error (MSE) properties of the new estimator and compare them with those of the restricted maximum likelihood estimator (RMLE). Then we suggest some estimators of the shrinkage parameter, and a simulation study is conducted to compare the performance of the different estimators. Finally, we show the benefit of using RLRE instead of RMLE when estimating how changes in price affect consumer demand for a specific product.     

Restricted minimum bias linear estimation in regression

In this article a class of restricted minimum bias linear estimators of the vector of unknown regression coefficients when multicollinearity among the columns of the design matrix exists, is obtained. The ordinary ridge regression, principal components and shrinkage estimators are members of this class. Moreover, our ap-proach can be used to improve, in some sense, certain classes of generalized ridge and shrinkage estimators of the vector of un-known parameters in linear models.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

MORE ON THE PRE-TEST ESTIMATOR IN RIDGE REGRESSION

ABSTRACT

The problem of estimation of the regression coefficients in a multiple regression model is considered under a multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. The objective of this paper is to compare the usual preliminary test estimator and the preliminary test ridge regression estimator in the sense of the dispersion matrix of one dominating that of the other. In particular we proved two results giving necessary and sufficient conditions for the superiority of the preliminary test ridge regression estimator over the preliminary test estimator associated with the δ = 0 (or Δ = 0) and δ ≠ 0 (or Δ ≠ 0).     

ESTIMATION OF EXPONENTIATED WEIBULL SHAPE PARAMETERS UNDER LINEX LOSS FUNCTION

ABSTRACT

The paper deals with Bayes estimation of the exponentiated Weibull shape parameters under linex loss function when independent non-informative type of priors are available for the parameters. Generalized maximum likelihood estimators have also been obtained. Performances of the proposed Bayes estimator, generalized maximum likelihood estimators, posterior mean (i.e., Bayes estimator under squared error loss function) and maximum likelihood estimators have been studied on the basis of their risks under linex loss function. The comparison is based on a simulation study because the expressions for risk functions of these estimators cannot be obtained in nice closed forms.     

Monte Carlo Simulation Study of Biased Estimators in the Linear Regression Models with Correlated or Heteroscedastic Errors

In this study, the performance of the estimators proposed in the presence of multicollinearity in the linear regression model with heteroscedastic or correlated or both error terms is investigated under the matrix mean square error criterion. Structures of the autocorrelated error terms are given and a Monte Carlo simulation study is conducted to examine the relative efficiency of the estimators against each other.     

Superiority of the r–k Class Estimator Over Some Estimators In A Linear Model

In regression analysis, to overcome the problem of multicollinearity, the r ? k class estimator is proposed as an alternative to the ordinary least squares estimator which is a general estimator including the ordinary ridge regression estimator, the principal components regression estimator and the ordinary least squares estimator. In this article, we derive the necessary and sufficient conditions for the superiority of the r ? k class estimator over each of these estimators under the Mahalanobis loss function by the average loss criterion. Then, we compare these estimators with each other using the same criterion. Also, we suggest to test to verify if these conditions are indeed satisfied. Finally, a numerical example and a Monte Carlo simulation are done to illustrate the theoretical results.