Liu estimation approach to the measurement error models

ABSTRACT

In this paper, we consider the estimation of the parameters of measurement error (ME) models when the multicollinearity exists. To remedy the problem of multicollinearity in ME models, we consider the Liu estimation approach. We define Liu and restricted Liu estimators and also examine the asymptotic properties of proposed estimators in ME models. Moreover, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean squared error criterion.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

On a generalization of the test of endogeneity in a two stage least squares estimation

In situations that the predictors are correlated with the error term, we propose a bridge estimator in the two-stage least squares estimation. We apply this estimator to overcome the multicollinearity and sparsity of the explanatory variables, when the endogeneity problem is present.The proposed estimator was applied to modify the Durbin-Wu-Hausman (DWH) test of endogeneity in the presence of multicollinearity. To compare our modified test with the existing DWH for detection of an endogenous problem in multi-collinear data, some numerical assessments are carried out. The numerical results showed that the proposed estimators and the suggested test perform better for the multi-collinear data. Finally, a genetical data set is applied for illustration the our results by estimating the coefficients parameters in the presence of endogeneity and multicollinearity.     

A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.     

A New Asymmetric Interaction Ridge (AIR) Regression Method

Despite that interaction terms are standard tools of regression analysis, the side effects of the inclusion of these terms in models estimated by ordinary least squares (OLS) are yet not fully penetrated. The inclusion of interaction effects induces multicollinearity problems since all non zero values are equal between the interaction term and the regressor. In this article, we propose a procedure to remedy this problem by the use of new ridge regression (RR) shrinkage parameters—which we call the asymmetric interaction ridge (AIR) regression method. By means of Monte Carlo simulations we evaluate both OLS and AIR using the mean square error (MSE) performance criterion. The result from the simulation study confirms our hypothesis that AIR always should be preferred to OLS since it has a lower estimated MSE. Moreover, the advantages of our new method are demonstrated in an empirical application where positive asymmetric price transmission effects are exposed for the mortgage interest rates of Handelsbanken Stadshypotek. It is observed that the mortgage interest rates increase more fully and rapidly to an increase in the bank's borrowing costs than to a decrease. This asymmetry is defined as positive asymmetric price transmission (APT).     

Two-parameter ridge estimator in the binary logistic regression

The binary logistic regression is a commonly used statistical method when the outcome variable is dichotomous or binary. The explanatory variables are correlated in some situations of the logit model. This problem is called multicollinearity. It is known that the variance of the maximum likelihood estimator (MLE) is inflated in the presence of multicollinearity. Therefore, in this study, we define a new two-parameter ridge estimator for the logistic regression model to decrease the variance and overcome multicollinearity problem. We compare the new estimator to the other well-known estimators by studying their mean squared error (MSE) properties. Moreover, a Monte Carlo simulation is designed to evaluate the performances of the estimators. Finally, a real data application is illustrated to show the applicability of the new method. According to the results of the simulation and real application, the new estimator outperforms the other estimators for all of the situations considered.     

New Shrinkage Parameters for the Liu-type Logistic Estimators

The binary logistic regression is a widely used statistical method when the dependent variable has two categories. In most of the situations of logistic regression, independent variables are collinear which is called the multicollinearity problem. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Therefore, this article introduces new shrinkage parameters for the Liu-type estimators in the Liu (2003) in the logistic regression model defined by Huang (2012) in order to decrease the variance and overcome the problem of multicollinearity. A Monte Carlo study is designed to show the goodness of the proposed estimators over MLE in the sense of mean squared error (MSE) and mean absolute error (MAE). Moreover, a real data case is given to demonstrate the advantages of the new shrinkage parameters.     

Robust Liu-type estimator for regression based on M-estimator

The problem of multicollinearity and outliers in the dataset can strongly distort ordinary least-square estimates and lead to unreliable results. We propose a new Robust Liu-type M-estimator to cope with this combined problem of multicollinearity and outliers in the y-direction. Our new estimator has advantages over two-parameter Liu-type estimator, Ridge-type M-estimator, and M-estimator. Furthermore, we give a numerical example and a simulation study to illustrate some of the theoretical results.     

Minimax Empirical Bayes Estimators in Multivariate Mixed Linear Models with Unequal Replications

Predictive influence of explanatory variables has been studied in both univariate and multivariate distributions. In the Bayesian approach, the same problem is considered in absence of multicollinearity in the dataset. The aim of this article is to study the same in the presence of perfect multicollinearity. To do this, we first derived the predictive distributions for full model and reduced model using vague prior density. Then the discrepancies between these predictive distributions are measured by the Kullback–Leibler (K–L) directed measure of divergence to assess the influence of deleted explanatory variables. Finally, distribution of the discrepancies is derived and the test procedure is performed.     

On the Restricted Liu Estimator in the Logistic Regression Model

The logistic regression model is used when the response variables are dichotomous. In the presence of multicollinearity, the variance of the maximum likelihood estimator (MLE) becomes inflated. The Liu estimator for the linear regression model is proposed by Liu to remedy this problem. Urgan and Tez and Mansson et al. examined the Liu estimator (LE) for the logistic regression model. We introduced the restricted Liu estimator (RLE) for the logistic regression model. Moreover, a Monte Carlo simulation study is conducted for comparing the performances of the MLE, restricted maximum likelihood estimator (RMLE), LE, and RLE for the logistic regression model.     

A modified ridge m-estimator for linear regression model with multicollinearity and outliers

The ordinary least-square estimators for linear regression analysis with multicollinearity and outliers lead to unfavorable results. In this article, we propose a new robust modified ridge M-estimator (MRME) based on M-estimator (ME) to deal with the combined problem resulting from multicollinearity and outliers in the y-direction. MRME outperforms modified ridge estimator, robust ridge estimator and ME, according to mean squares error criterion. Furthermore, a numerical example and a Monte Carlo simulation experiment are given to illustrate some of the theoretical results.     

基于粗集理论的线性回归方法及实证分析

通过分析实际问题中经济变量间往往出现多重共线性的现象，将粗集理论的约简思想引入线性回归分析，提出了基于粗集理论的线性回归模型来解决多重共线性问题，并通过实证分析来验证模型的可行性，为人们进行科学的预测和决策提供了一种新的思想和方法。     

基于投入——产出思想的多重共线问题的解决

多重共线是回归分析中常见的问题,尤其是近似多重共线,对预测精度和稳定性影响很大。本文着重论述了近似多重共线的危害、诊断以及几种常用的解决办法,并根据投入——产出中直接消耗、间接消耗的思想,提出一种新的解决思路。     

Robust Linearized Ridge M-estimator for Linear Regression Model

In the multiple linear regression, multicollinearity and outliers are commonly occurring problems. They produce undesirable effects on the ordinary least squares estimator. Many alternative parameter estimation methods are available in the literature which deals with these problems independently. In practice, it may happen that the multicollinearity and outliers occur simultaneously. In this article, we present a new estimator called as Linearized Ridge M-estimator which combats the problem of simultaneous occurrence of multicollinearity and outliers. A real data example and a simulation study is carried out to illustrate the performance of the proposed estimator.     

Performance of some new preliminary test ridge regression estimators and their properties

The problem of estimation of the regression coefficients in a multiple regression model is considered under multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology. Accordingly, we consider three estimators, namely, the unrestricted ridge regression estimator (URRE), the restricted ridge regression estimator (RRRE), and finally, the preliminary test ridge regression estimator (PTRRE). The biases, variancematrices and mean square errors (mse) of the estimators are derived and compared with the usual estimators. Regions of optimality of the estimators are determined by studying the mse criterion. The conditions of superiority of the estimators over the traditional estimators as in Saleh and Han (1990) and Ali and Saleh (1991) have also been discussed.     

Modified ridge-type for the Poisson regression model: simulation and application

The Poisson regression model (PRM) is employed in modelling the relationship between a count variable (y) and one or more explanatory variables. The parameters of PRM are popularly estimated using the Poisson maximum likelihood estimator (PMLE). There is a tendency that the explanatory variables grow together, which results in the problem of multicollinearity. The variance of the PMLE becomes inflated in the presence of multicollinearity. The Poisson ridge regression (PRRE) and Liu estimator (PLE) have been suggested as an alternative to the PMLE. However, in this study, we propose a new estimator to estimate the regression coefficients for the PRM when multicollinearity is a challenge. We perform a simulation study under different specifications to assess the performance of the new estimator and the existing ones. The performance was evaluated using the scalar mean square error criterion and the mean squared error prediction error. The aircraft damage data was adopted for the application study and the estimators’ performance judged by the SMSE and the mean squared prediction error. The theoretical comparison shows that the proposed estimator outperforms other estimators. This is further supported by the simulation study and the application result.KEYWORDS: Poisson regression model, Poisson maximum likelihood estimator, multicollinearity, Poisson ridge regression, Liu estimator, simulation     

The corrected VIF (CVIF)

In this paper, we propose a new corrected variance inflation factor (VIF) measure to evaluate the impact of the correlation among the explanatory variables in the variance of the ordinary least squares estimators. We show that the real impact on variance can be overestimated by the traditional VIF when the explanatory variables contain no redundant information about the dependent variable and a corrected version of this multicollinearity indicator becomes necessary.     

On the Principal Component Liu-type Estimator in Linear Regression

In this article, we present a principal component Liu-type estimator (LTE) by combining the principal component regression (PCR) and LTE to deal with the multicollinearity problem. The superiority of the new estimator over the PCR estimator, the ordinary least squares estimator (OLSE) and the LTE are studied under the mean squared error matrix. The selection of the tuning parameter in the proposed estimator is also discussed. Finally, a numerical example is given to explain our theoretical results.     

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.