Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.     

Generalised Rank Regression Estimator with Standard Error Adjusted Lasso

One of the standard variable selection procedures in multiple linear regression is to use a penalisation technique in least‐squares (LS) analysis. In this setting, many different types of penalties have been introduced to achieve variable selection. It is well known that LS analysis is sensitive to outliers, and consequently outliers can present serious problems for the classical variable selection procedures. Since rank‐based procedures have desirable robustness properties compared to LS procedures, we propose a rank‐based adaptive lasso‐type penalised regression estimator and a corresponding variable selection procedure for linear regression models. The proposed estimator and variable selection procedure are robust against outliers in both response and predictor space. Furthermore, since rank regression can yield unstable estimators in the presence of multicollinearity, in order to provide inference that is robust against multicollinearity, we adjust the penalty term in the adaptive lasso function by incorporating the standard errors of the rank estimator. The theoretical properties of the proposed procedures are established and their performances are investigated by means of simulations. Finally, the estimator and variable selection procedure are applied to the Plasma Beta‐Carotene Level data set.     

A modified ridge m-estimator for linear regression model with multicollinearity and outliers

The ordinary least-square estimators for linear regression analysis with multicollinearity and outliers lead to unfavorable results. In this article, we propose a new robust modified ridge M-estimator (MRME) based on M-estimator (ME) to deal with the combined problem resulting from multicollinearity and outliers in the y-direction. MRME outperforms modified ridge estimator, robust ridge estimator and ME, according to mean squares error criterion. Furthermore, a numerical example and a Monte Carlo simulation experiment are given to illustrate some of the theoretical results.     

Robust two parameter ridge M-estimator for linear regression

The problem of multicollinearity and outliers in the data set produce undesirable effects on the ordinary least squares estimator. Therefore, robust two parameter ridge estimation based on M-estimator (ME) is introduced to deal with multicollinearity and outliers in the y-direction. The proposed estimator outperforms ME, two parameter ridge estimator and robust ridge M-estimator according to mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented.     

Robust Liu-type estimator for regression based on M-estimator

The problem of multicollinearity and outliers in the dataset can strongly distort ordinary least-square estimates and lead to unreliable results. We propose a new Robust Liu-type M-estimator to cope with this combined problem of multicollinearity and outliers in the y-direction. Our new estimator has advantages over two-parameter Liu-type estimator, Ridge-type M-estimator, and M-estimator. Furthermore, we give a numerical example and a simulation study to illustrate some of the theoretical results.     

Robust Principal Component Functional Logistic Regression

In this article, we discuss the estimation of the parameter function for a functional logistic regression model in the presence of outliers. We consider ways that allow for the parameter estimator to be resistant to outliers, in addition to minimizing multicollinearity and reducing the high dimensionality, which is inherent with functional data. To achieve this, the functional covariates and functional parameter of the model are approximated in a finite-dimensional space generated by an appropriate basis. This approach reduces the functional model to a standard multiple logistic model with highly collinear covariates and potential high-dimensionality issues. The proposed estimator tackles these issues and also minimizes the effect of functional outliers. Results from a simulation study and a real world example are also presented to illustrate the performance of the proposed estimator.     

Robust Liu estimator for regression based on an M-estimator

Consider the regression model y = beta 0 1 + Xbeta + epsilon. Recently, the Liu estimator, which is an alternative biased estimator beta L (d) = (X'X + I) -1 (X'X + dI)beta OLS , where 0<d<1 is a parameter, has been proposed to overcome multicollinearity . The advantage of beta L (d) over the ridge estimator beta R (k) is that beta L (d) is a linear function of d. Therefore, it is easier to choose d than to choose k in the ridge estimator. However, beta L (d) is obtained by shrinking the ordinary least squares (OLS) estimator using the matrix (X'X + I) -1 (X'X + dI) so that the presence of outliers in the y direction may affect the beta L (d) estimator. To cope with this combined problem of multicollinearity and outliers, we propose an alternative class of Liu-type M-estimators (LM-estimators) obtained by shrinking an M-estimator beta M , instead of the OLS estimator using the matrix (X'X + I) -1 (X'X + dI).     

A robust Liu regression estimator

The least-squares regression estimator can be very sensitive in the presence of multicollinearity and outliers in the data. We introduce a new robust estimator based on the MM estimator. By considering weights, also the resulting MM-Liu estimator is highly robust, but also the estimation of the biasing parameter is robustified. Also for high-dimensional data, a robust Liu-type estimator is introduced, based on the Partial Robust M-estimator. Simulation experiments and a real dataset show the advantages over the standard estimators and other robustness proposals.     

Improved robust ridge M-estimation

It is developed that non-sample prior information about regression vector-parameter, usually in the form of constraints, improves the risk performance of the ordinary least squares estimator (OLSE) when it is shrunken. However, in practice, it may happen that both multicollinearity and outliers exist simultaneously in the data. In such a situation, the use of robust ridge estimator is suggested to overcome the undesirable effects of the OLSE. In this article, some prior information in the form of constraints is employed to improve the performance of this estimator in the multiple regression model. In this regard, shrinkage ridge robust estimators are defined. Advantages of the proposed estimators over the usual robust ridge estimator are also investigated using Monte-Carlo simulation as well as a real data example.     

A useful approach to identify the multicollinearity in the presence of outliers

The presence of outliers in the data sets affects the structure of multicollinearity which arises from a high degree of correlation between explanatory variables in a linear regression analysis. This affect could be seen as an increase or decrease in the diagnostics used to determine multicollinearity. Thus, the cases of outliers reduce the reliability of diagnostics such as variance inflation factors, condition numbers and variance decomposition proportions. In this study, we propose to use a robust estimation of the correlation matrix obtained by the minimum covariance determinant method to determine the diagnostics of multicollinearity in the presence of outliers. As a result, the present paper demonstrates that the diagnostics of multicollinearity obtained by the robust estimation of the correlation matrix are more reliable in the presence of outliers.     

The feasible generalized restricted ridge regression estimator

The presence of autocorrelation in errors and multicollinearity among the regressors have undesirable effects on the least-squares regression. There are a wide range of methods which are proposed to overcome the usefulness of the ordinary least-squares estimator or the generalized least-squares estimator, such as the Stein-rule, restricted least-squares or ridge estimator. Therefore, we introduce a new feasible generalized restricted ridge regression (FGRR) estimator to examine multicollinearity and autocorrelation problems simultaneously for the general linear regression model. We also derive some statistical properties of the FGRR estimator and comparisons have been conducted using matrix mean-square error. Moreover, a Monte Carlo simulation experiment is performed to investigate the performance of the proposed estimator over the others.     

r − k Class estimator in the linear regression model with correlated errors

Autocorrelation in errors and multicollinearity among the regressors are serious problems in regression analysis. The aim of this paper is to examine multicollinearity and autocorrelation problems concurrently and to compare the r ? k class estimator to the generalized least squares estimator, the principal components regression estimator and the ridge regression estimator by the scalar and matrix mean square error criteria in the linear regression model with correlated errors.     

Factor analysis regression

In the presence of multicollinearity the literature points to principal component regression (PCR) as an estimation method for the regression coefficients of a multiple regression model. Due to ambiguities in the interpretation, involved by the orthogonal transformation of the set of explanatory variables, the method could not yet gain wide acceptance. Factor analysis regression (FAR) provides a model-based estimation method which is particularly tailored to overcome multicollinearity in an errors-in-variables setting. In this paper two feasible versions of a FAR estimator are compared with the OLS estimator and the PCR estimator by means of Monte Carlo simulation. While the PCR estimator performs best in cases of strong and high multicollinearity, the Thomson-based FAR estimator proves to be superior when the regressors are moderately correlated.     

Fractional principal components regression: a general approach to biased estimators

Several biased estimators have been proposed as alternatives to the least squares estimator when multicollinearity is present in the multiple linear regression model. The ridge estimator and the principal components estimator are two techniques that have been proposed for such problems. In this paper the class of fractional principal component estimators is developed for the multiple linear regression model. This class contains many of the biased estimators commonly used to combat multicollinearity. In the fractional principal components framework, two new estimation techniques are introduced. The theoretical performances of the new estimators are evaluated and their small sample properties are compared via simulation with the ridge, generalized ridge and principal components estimators     

A graphical evaluation of logistic ridge estimator in mixture experiments

In comparison to other experimental studies, multicollinearity appears frequently in mixture experiments, a special study area of response surface methodology, due to the constraints on the components composing the mixture. In the analysis of mixture experiments by using a special generalized linear model, logistic regression model, multicollinearity causes precision problems in the maximum-likelihood logistic regression estimate. Therefore, effects due to multicollinearity can be reduced to a certain extent by using alternative approaches. One of these approaches is to use biased estimators for the estimation of the coefficients. In this paper, we suggest the use of logistic ridge regression (RR) estimator in the cases where there is multicollinearity during the analysis of mixture experiments using logistic regression. Also, for the selection of the biasing parameter, we use fraction of design space plots for evaluating the effect of the logistic RR estimator with respect to the scaled mean squared error of prediction. The suggested graphical approaches are illustrated on the tumor incidence data set.     

Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Efficiency of the QR class estimator in semiparametric regression models to combat multicollinearity

There are some classes of biased estimators for solving the multicollinearity among the predictor variables in statistical literature. In this research, we propose a modified estimator based on the QR decomposition in the semiparametric regression models, to combat the multicollinearity problem of design matrix which makes the data to be less distorted than the other methods. We derive the properties of the proposed estimator, and then, the necessary and sufficient condition for the superiority of the partially generalized QR-based estimator over partially generalized least-squares estimator is obtained. In the biased estimators, selection of shrinkage parameters plays an important role in data analysing. We use generalized cross-validation criterion for selecting the optimal shrinkage parameter and the bandwidth of the kernel smoother. Finally, the Monté-Carlo simulation studies and a real application related to bridge construction data are conducted to support our theoretical discussion.     

On a generalization of the test of endogeneity in a two stage least squares estimation

In situations that the predictors are correlated with the error term, we propose a bridge estimator in the two-stage least squares estimation. We apply this estimator to overcome the multicollinearity and sparsity of the explanatory variables, when the endogeneity problem is present.The proposed estimator was applied to modify the Durbin-Wu-Hausman (DWH) test of endogeneity in the presence of multicollinearity. To compare our modified test with the existing DWH for detection of an endogenous problem in multi-collinear data, some numerical assessments are carried out. The numerical results showed that the proposed estimators and the suggested test perform better for the multi-collinear data. Finally, a genetical data set is applied for illustration the our results by estimating the coefficients parameters in the presence of endogeneity and multicollinearity.