An evaluation of ridge estimator in linear mixed models: an example from kidney failure data

This paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model.     

Restricted Two Parameter Ridge Estimator

In 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.     

Comparison of estimators of the location parameter using pitman's closeness criterion

Necessary and sufficient conditions for a linear estimator to dominate another linear estimator of a location parameter under the Pitman's criterion of comparison are discussed. Consequently it is demonstrated that a linear biased estimator can not dominate a linear unbiased estimator under Pitman's criterion and that the sample mean is the Closest Linear Unbiased Estimator (CLUE). It is also shown that the ridge regression estimator with a known biasing constant can not dominate the ordinary least squares estimator. If an estimator δdominates an estimator δin the average loss sense then sufficient conditions are obtained under which δis also preferred over δunder Pitman's criterion. Further we obtain sufficient conditions under which preference under the Pitman's criterion will lead to preference under the mean squared error sense.     

Preliminary testing of the Cobb–Douglas production function and related inferential issues

In this article, we consider the multiple regression model in the presence of multicollinearity and study the performance of the preliminary test estimator (PTE) both analytically and computationally, when it is a priori suspected that some constraints may hold on the vector parameter space. The performance of the PTE is further analyzed by comparing the risk of some well-known estimators of the ridge parameter through an extensive Monte Carlo simulation study under some bounded and or asymmetric loss functions. An application of the Cobb–Douglas production function is included and from these results as well as the simulation studies, it is clear that the bounded linear exponential loss function outperforms the other loss functions across all the proposed ridge parameters by comparing the risk values.     

The best subset of parameters in min-max linear regression

These Fortran-77 subroutines provide building blocks for Generalized Cross-Validation (GCV) (Craven and Wahba, 1979) calculations in data analysis and data smoothing including ridge regression (Golub, Heath, and Wahba, 1979), thin plate smoothing splines (Wahba and Wendelberger, 1980), deconvolution (Wahba, 1982d), smoothing of generalized linear models (O'sullivan, Yandell and Raynor 1986, Green 1984 and Green and Yandell 1985), and ill-posed problems (Nychka et al., 1984, O'sullivan and Wahba, 1985). We present some of the types of problems for which GCV is a useful method of choosing a smoothing or regularization parameter and we describe the structure of the subroutines.Ridge Regression: A familiar example of a smoothing parameter is the ridge parameter X in the ridge regression problem which we write.     

Cross validation of ridge regression estimator in autocorrelated linear regression models

ABSTRACT

In this paper, we investigated the cross validation measures, namely OCV, GCV and Cp under the linear regression models when the error structure is autocorrelated and regressor data are correlated. The best performed ridge regression estimator is obtained by getting the optimal ridge parameter so as to minimize these measures. A Monte Carlo simulation study is given to see how the optimal ridge parameter is affected by autocorrelation and the strength of multicollinearity.     

Collinearity: revisiting the variance inflation factor in ridge regression

Ridge regression has been widely applied to estimate under collinearity by defining a class of estimators that are dependent on the parameter k. The variance inflation factor (VIF) is applied to detect the presence of collinearity and also as an objective method to obtain the value of k in ridge regression. Contrarily to the definition of the VIF, the expressions traditionally applied in ridge regression do not necessarily lead to values of VIFs equal to or greater than 1. This work presents an alternative expression to calculate the VIF in ridge regression that satisfies the aforementioned condition and also presents other interesting properties.     

Selection of the Ridge Parameter Using Mathematical Programming

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates and their standard errors in order to reach acceptable results. Selection of the ridge parameter was done using several subjective and objective techniques that are concerned with certain criteria. In this study, selection of the ridge parameter depends on other important statistical measures to reach a better value of the ridge parameter. The proposed ridge parameter selection technique depends on a mathematical programming model and the results are evaluated using a simulation study. The performance of the proposed method is good when the error variance is greater than or equal to one; the sample consists of 20 observations, the number of explanatory variables in the model is 2, and there is a very strong correlation between the two explanatory variables.     

Liu-Type Logistic Estimator

It is known that multicollinearity inflates the variance of the maximum likelihood estimator in logistic regression. Especially, if the primary interest is in the coefficients, the impact of collinearity can be very serious. To deal with collinearity, a ridge estimator was proposed by Schaefer et al. The primary interest of this article is to introduce a Liu-type estimator that had a smaller total mean squared error (MSE) than the Schaefer's ridge estimator under certain conditions. Simulation studies were conducted that evaluated the performance of this estimator. Furthermore, the proposed estimator was applied to a real-life dataset.     

A ridge-type estimator and good prior means

Swindel (1976) introduced a modified ridge regression estimator based on prior information. A necessary and sufficient condition is derived for Swindel's proposed estimator to have lower risk than the conventional ordinary ridge regression estimator when both estimators are computed using the same value of k.     

Confidence Interval for Shrinkage Parameters in Ridge Regression

In ridge regression, the estimation of ridge parameter k is an important problem. There are several methods available in the literature to do this job some what efficiently. However, no attempts were made to suggest a confidence interval for the ridge parameter using the knwoledge from the data. In this article, we propose a data dependent confidence interval for the ridge parameter k. The method of obtaining the confidence interval is illustrated with the help of a data set. A simulation study indicates that the empirical coverage probability of the suggested confidence intervals are quite high.     

Choice of the ridge factor from the correlation matrix determinant

Ridge regression is the alternative method to ordinary least squares, which is mostly applied when a multiple linear regression model presents a worrying degree of collinearity. A relevant topic in ridge regression is the selection of the ridge parameter, and different proposals have been presented in the scientific literature. Since the ridge estimator is biased, its estimation is normally based on the calculation of the mean square error (MSE) without considering (to the best of our knowledge) whether the proposed value for the ridge parameter really mitigates the collinearity. With this goal and different simulations, this paper proposes to estimate the ridge parameter from the determinant of the matrix of correlation of the data, which verifies that the variance inflation factor (VIF) is lower than the traditionally established threshold. The possible relation between the VIF and the determinant of the matrix of correlation is also analysed. Finally, the contribution is illustrated with three real examples.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

An example of ridge regression difficulties

A simple consumption function is used to illustrate two fundamental difficulties with ridge regression and similarly motivated procedures. The first is the ambiguity of multicollinearity measures for judging the data's “ill-conditioning”. The second is the sensitivity of the estimates to the arbitrary normalization of the model. Neither of these poses a problem for least squares or Bayesian estimates. The logical restructuring of ridge procedures to avoid these difficulties leads to a more explicitly Bayesian approach.     

The Bayes rule of the variance parameter of the hierarchical normal and inverse gamma model under Stein's loss

For the variance parameter of the hierarchical normal and inverse gamma model, we analytically calculate the Bayes rule (estimator) with respect to a prior distribution IG (alpha, beta) under Stein's loss function. This estimator minimizes the posterior expected Stein's loss (PESL). We also analytically calculate the Bayes rule and the PESL under the squared error loss. Finally, the numerical simulations exemplify that the PESLs depend only on alpha and the number of observations. The Bayes rules and PESLs under Stein's loss are unanimously smaller than those under the squared error loss.     

Ridge Estimation under the Stochastic Restriction

In linear programming and modeling of an economic system, there may occur some linear stochastic artificial or unnatural manners, which may need serious attentions. These stochastic unusual uncertainty, say stochastic constraints, definitely cause some changes in the estimators under work and their behaviors. In this approach, we are basically concerned with the problem of multicollinearity, when it is suspected that the parameter space may be restricted to some stochastic restrictions. We develop the estimation strategy form unbiasedness to some improved biased adjustment. In this regard, we study the performance of shrinkage estimators under the assumption of elliptically contoured errors and derive the region of optimality of each one. Lastly, a numerical example is taken to determine the adequate ridge parameter for each given estimator.     

Robust two parameter ridge M-estimator for linear regression

The problem of multicollinearity and outliers in the data set produce undesirable effects on the ordinary least squares estimator. Therefore, robust two parameter ridge estimation based on M-estimator (ME) is introduced to deal with multicollinearity and outliers in the y-direction. The proposed estimator outperforms ME, two parameter ridge estimator and robust ridge M-estimator according to mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented.     

Using improved estimation strategies to combat multicollinearity

In this approach, some generalized ridge estimators are defined based on shrinkage foundation. Completely under the suspicion that some sub-space restrictions may occur, we present the estimators of the regression coefficients combining the idea of preliminary test estimator and Stein-rule estimator with the ridge regression methodology for normal models. Their exact risk expressions in addition to biases are derived and the regions of optimality of the estimators are exactly determined along with some numerical analysis. In this regard, the ridge parameter is determined in different disciplines.     

Linearized Ridge Regression Estimator Under the Mean Squared Error Criterion in a Linear Regression Model

This article mainly aims to study the superiority of the notion of linearized ridge regression estimator (LRRE) under the mean squared error criterion in a linear regression model. Firstly, we derive uniform lower bound of MSE for the class of the generalized shrinkage estimator (GSE), based on which it is shown that the optimal LRRE is the best estimator in the class of GSE's. Secondly, we propose the notion of the almost unbiased completeness and show that LRRE possesses such a property. Thirdly, the simulation study is given, from which it indicates that the LRRE performs desirably. Finally, the main results are applied to the well known Hald data.