Influence in the Elliptical Modified Ridge Regression

In this article, we present diagnostic methods for the modified ridge regression under elliptical model based on the pseudo-likelihood function. The maximum likelihood estimators of the parameters in the modified ridge elliptical model are given and local influence measures are developed. Finally, illustration of our methodology is given through a numerical example.     

Local Influence Diagnostics in the Modified Ridge Regression

Calculations of local influence curvatures have been well developed when the ordinary least squares method is applied. In this article, we discuss the assessment of local influence under the modified ridge regression. Using a pseudo-likelihood function, we express the normal curvatures of local influence for three useful perturbation schemes in interpretable forms. Two illustrative examples are analyzed by the methodology developed in the article.     

A Tobit Ridge Regression Estimator

This article analyzes the effects of multicollienarity on the maximum likelihood (ML) estimator for the Tobit regression model. Furthermore, a ridge regression (RR) estimator is proposed since the mean squared error (MSE) of ML becomes inflated when the regressors are collinear. To investigate the performance of the traditional ML and the RR approaches we use Monte Carlo simulations where the MSE is used as performance criteria. The simulated results indicate that the RR approach should always be preferred to the ML estimation method.     

Jackknifing the Ridge Regression Estimator: A Revisit

Singh et al. (1986 Singh, B., Chaubey, Y.P., Dwivedi, T.D. (1986). An almost unbiased ridge estimator. Sankhya B48: 34236. [Google Scholar]) proposed an almost unbiased ridge estimator using Jackknife method that required transformation of the regression parameters. This article shows that the same method can be used to derive the Jackknifed ridge estimator of the original (untransformed) parameter without transformation. This method also leads in deriving easily the second-order Jackknifed ridge that may reduce the bias further. We further investigate the performance of these estimators along with a recent method by Batah et al. (2008 Batah, F. S.M., Ramanathan, T.V., Gore, S.D. (2008). The efficiency of modified Jack-knife and ridge type regression estimators: a comparison. Surv. Math. Applic. 3:111122. [Google Scholar]) called modified Jackknifed ridge theoretically as well as numerically.     

Local Influence Analysis for The Ridge Regression Under Stochastic Linear Restrictions

In this article, we assess the local influence for the ridge regression of linear models with stochastic linear restrictions in the spirit of Cook by using the log-likelihood of the stochastic restricted ridge regression estimator. The diagnostics under the perturbations of constant variance, responses and individual explanatory variables are derived. We also assess the local influence of the stochastic restricted ridge regression estimator under the approach suggested by Billor and Loynes. At the end, a numerical example on the Longley data is given to illustrate the theoretic results.     

Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence

It is common for a linear regression model that the error terms display some form of heteroscedasticity and at the same time, the regressors are also linearly correlated. Both of these problems have serious impact on the ordinary least squares (OLS) estimates. In the presence of heteroscedasticity, the OLS estimator becomes inefficient and the similar adverse impact can also be found on the ridge regression estimator that is alternatively used to cope with the problem of multicollinearity. In the available literature, the adaptive estimator has been established to be more efficient than the OLS estimator when there is heteroscedasticity of unknown form. The present article proposes the similar adaptation for the ridge regression setting with an attempt to have more efficient estimator. Our numerical results, based on the Monte Carlo simulations, provide very attractive performance of the proposed estimator in terms of efficiency. Three different existing methods have been used for the selection of biasing parameter. Moreover, three different distributions of the error term have been studied to evaluate the proposed estimator and these are normal, Student's t and F distribution.     

Performance of Some Ridge Parameters for Probit Regression: With Application to Swedish Job Search Data

In ridge regression, the estimation of the ridge parameter is an important issue. This article generalizes some methods for estimating the ridge parameter for probit ridge regression (PRR) model based on the work of Kibria et al. (2011 Kibria, B. M. G., Månsson, K. and Shukur, G. 2011. Performance of some logistic ridge regression parameters. Computational Economics, DOI: 10.1007/s10614-011-9275-x [Google Scholar]). The performance of these new estimators is judged by calculating the mean squared error (MSE) using Monte Carlo simulations. In the design of the experiment, we chose to vary the sample size and the number of regressors. Furthermore, we generate explanatory variables that are linear combinations of other regressors, which is a common situation in economics. In an empirical application regarding Swedish job search data, we also illustrate the benefits of the new method.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Linearized Ridge Regression Estimator Under the Mean Squared Error Criterion in a Linear Regression Model

This article mainly aims to study the superiority of the notion of linearized ridge regression estimator (LRRE) under the mean squared error criterion in a linear regression model. Firstly, we derive uniform lower bound of MSE for the class of the generalized shrinkage estimator (GSE), based on which it is shown that the optimal LRRE is the best estimator in the class of GSE's. Secondly, we propose the notion of the almost unbiased completeness and show that LRRE possesses such a property. Thirdly, the simulation study is given, from which it indicates that the LRRE performs desirably. Finally, the main results are applied to the well known Hald data.     

A Note on a Commonly Used Ridge Regression Monte Carlo Design

Ridge estimators are usually examined through Monte Carlo simulations since their properties are difficult to obtain analytically. In this paper we argue that a simulation design commonly used in the literature will give biased results of Monte Carlo simulations in favor of ridge regression over ordinary least square estimators. Specifically, it is argued that the properties of ridge estimators that are functions of p distinct regressor eigenvalues should not be evaluated through Monte Carlo designs using only two distinct eigenvalues.     

Restricted Ridge Estimators of the Parameters in Semiparametric Regression Model

In this article, we introduce a ridge estimator for the vector of parameters β in a semiparametric model when additional linear restrictions on the parameter vector are assumed to hold. We also obtain the semiparametric restricted ridge estimator for the parametric component in the semiparametric regression model. The ideas in this article are illustrated with a data set consisting of housing prices and through a comparison of the performances of the proposed and related estimators via a Monte Carlo simulation.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.     

Ridge Estimation to the Restricted Linear Model

This article is concerned with the problem of multicollinearity in a linear model with linear restrictions. After introducing a spheral restricted condition, a new restricted ridge estimation method is proposed by minimizing the sum of squared residuals. The property of the new estimator in its superiority over the ordinary restricted least squares estimation is then theoretically analyzed. Furthermore, a sufficient and necessary condition for selecting the ridge parameter k is obtained. To simplify the selection of the ridge parameter, a sufficient condition is also given. Finally, a numerical example demonstrates the merit of the new method in the aspect of solving the multicollinearity over the ordinary restricted least squares estimation.     

A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions

Based on the work of Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameters for regression problems . Communications in Statistics – Theory and Methods 34 : 1177 – 1182 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi , M. , Khalaf , G. , Shukur , G. ( 2006 ). Some modifications for choosing ridge parameters . Communications in Statistics – Theory and Methods 35 : 2005 – 2020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2010 Muniz , G. , Kibria , B. M. G. , Shukur , G. ( 2010 ). On developing ridge regression parameters: a graphical Investigation. Submitted for Publication . [Google Scholar]), this article considers several estimators for estimating the ridge parameter k. This article differs from aforementioned articles in three ways: (1) Data are generated from Normal, Student's t, and F distributions with appropriate degrees of freedom; (2) The number of regressors considered are from 4–12 instead of 2–4, which are the usual practice; (3) Both mean square error (MSE) and prediction sum of square (PRESS) are considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has negative effect on the MSE and PRESS. However, increasing the number of regressors has positive effect on MSE and PRESS. When the sample size increases the MSE decreases even when the correlation between the independent variables is large. It is interesting to note that the dominance pictures of the estimators are remained the same under both the MSE and PRESS criterion. However, the performance of the estimators depends on the choice of the assumption of the error distribution of the regression model.     

Modified Ridge Parameters for Seemingly Unrelated Regression Model

In this article, we modify a number of new biased estimators of seemingly unrelated regression (SUR) parameters which are developed by Alkhamisi and Shukur (2008 Alkhamisi , M. A. , Shukur , G. ( 2008 ). Developing ridge parameters for SUR model . Commun. Statist. Theor. Meth. 37 ( 4 ): 544 – 564 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), AS, when the explanatory variables are affected by multicollinearity. Nine estimators of the ridge parameters have been modified and compared in terms of the trace mean squared error (TMSE) and (PR) criterion. The results from this extended study are the also compared with those founded by AS. A simulation study has been conducted to compare the performance of the modified estimators of the ridge parameters. The results showed that under certain conditions the performance of the multivariate ridge regression estimators based on SUR ridge R _MSmax is superior to other estimators in terms of TMSE and PR criterion. In large samples and when the collinearity between the explanatory variables is not high, the unbiased SUR, estimator produces a smaller TMSEs.     

A Monte Carlo Study of Recent Ridge Parameters

A number of procedures have been developed for finding biased estimators of regression parameters. One of these procedures is the ridge regression. In this article, a new approach to obtain the ridge parameter K is suggested and then evaluated by Monte Carlo simulations. A number of different models are investigated for different number of observations, the strength of correlation between the explanatory variables, and distribution of the error terms. The mean squared error (MSE) criterion is used to examine the performance of the proposed estimators when compared with other well-known estimators. Under certain conditions, it is shown that at least one of the proposed estimators have a smaller MSE than the ordinary least squared estimator (OLS) and Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) estimator (HK).     

Robust Linearized Ridge M-estimator for Linear Regression Model

In the multiple linear regression, multicollinearity and outliers are commonly occurring problems. They produce undesirable effects on the ordinary least squares estimator. Many alternative parameter estimation methods are available in the literature which deals with these problems independently. In practice, it may happen that the multicollinearity and outliers occur simultaneously. In this article, we present a new estimator called as Linearized Ridge M-estimator which combats the problem of simultaneous occurrence of multicollinearity and outliers. A real data example and a simulation study is carried out to illustrate the performance of the proposed estimator.     

A New Two-Parameter Estimator in Linear Regression

This article is concerned with the parameter estimation in linear regression model. To overcome the multicollinearity problem, a new two-parameter estimator is proposed. This new estimator is a general estimator which includes the ordinary least squares (OLS) estimator, the ridge regression (RR) estimator, and the Liu estimator as special cases. Necessary and sufficient conditions for the superiority of the new estimator over the OLS, RR, Liu estimators, and the two-parameter estimator proposed by Ozkale and Kaciranlar (2007 Ozkale , M. R. , Kaciranlar , S. ( 2007 ). The restricted and unrestricted two-parameter estimators . Commun. Statist. Theor. Meth. 36 : 2707 – 2725 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) in the mean squared error matrix (MSEM) sense are derived. Furthermore, we obtain the estimators of the biasing parameters and give a numerical example to illustrate some of the theoretical results.     

A New Ridge Regression Causality Test in the Presence of Multicollinearity

The VAR lag structure applied for the traditional Granger causality (GC) test is always severely affected by multicollinearity due to autocorrelation among the lags. Therefore, as a remedy to this problem we introduce a new Ridge Regression Granger Causality (RRGC) test, which is compared to the GC test by means of Monte Carlo simulations. Based on the simulation study we conclude that the traditional OLS version of the GC test over-rejects the true null hypothesis when there are relatively high (but empirically normal) levels of multicollinearity, while the new RRGC test will remedy or substantially decrease this problem.     

Influence Diagnostics in log-Birnbaum-Saunders Regression Models

In this paper we present various diagnostic methods for a linear regression model under a logarithmic Birnbaum-Saunders distribution for the errors, which may be applied for accelerated life testing or to compare the median lives of several populations. Some influence methods, such as the local influence, total local influence of an individual and generalized leverage are derived, analysed and discussed. We also present a connection between the local influence and generalized leverage methods. A discussion of the computation of the likelihood displacement as well as the normal curvature in the local influence method are presented. Finally, an example with real data is given for illustration.