Generalized Leverage and its Applications

The generalized leverage of an estimator is defined in regression models as a measure of the importance of individual observations. We derive a simple but powerful result, developing an explicit expression for leverage in a general M -estimation problem, of which the maximum likelihood problems are special cases. A variety of applications are considered, most notably to the exponential family non-linear models. The relationship between leverage and local influence is also discussed. Numerical examples are given to illustrate our results     

Diagnostic Displays for Assessing Leverage and Influence in Generalized Linear Models

A general technique for assessing leverage and influential observations in Generalized Linear Models is described. The procedure takes the form of Half-Normal plots with envelopes derived from simulation to enhance overall assessment of the model. This procedure of assessment is more informative and provides additional insight compared with procedures based on the largest sample leverage and influence statistics. Application of the method is illustrated with an example in logistic regression.     

An evaluation of ridge estimator in linear mixed models: an example from kidney failure data

This paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model.     

The performance of diagnostic-robust generalized potentials for the identification of multiple high leverage points in linear regression

Leverage values are being used in regression diagnostics as measures of influential observations in the $X$-space. Detection of high leverage values is crucial because of their responsibility for misleading conclusion about the fitting of a regression model, causing multicollinearity problems, masking and/or swamping of outliers, etc. Much work has been done on the identification of single high leverage points and it is generally believed that the problem of detection of a single high leverage point has been largely resolved. But there is no general agreement among the statisticians about the detection of multiple high leverage points. When a group of high leverage points is present in a data set, mainly because of the masking and/or swamping effects the commonly used diagnostic methods fail to identify them correctly. On the other hand, the robust alternative methods can identify the high leverage points correctly but they have a tendency to identify too many low leverage points to be points of high leverages which is not also desired. An attempt has been made to make a compromise between these two approaches. We propose an adaptive method where the suspected high leverage points are identified by robust methods and then the low leverage points (if any) are put back into the estimation data set after diagnostic checking. The usefulness of our newly proposed method for the detection of multiple high leverage points is studied by some well-known data sets and Monte Carlo simulations.     

Ridge Regression and Generalized Maximum Entropy: An improved version of the Ridge–GME parameter estimator

In this article, the Ridge–GME parameter estimator, which combines Ridge Regression and Generalized Maximum Entropy, is improved in order to eliminate the subjectivity in the analysis of the ridge trace. A serious concern with the visual inspection of the ridge trace to define the supports for the parameters in the Ridge–GME parameter estimator is the misinterpretation of some ridge traces, in particular where some of them are very close to the axes. A simulation study and two empirical applications are used to illustrate the performance of the improved estimator. A MATLAB code is provided as supplementary material.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

Unbiased ridge estimation with prior information and ridge trace

A procedure is illustrated to incorporate prior information in the ridge regression model. Unbiased ridge estimators with prior information are defined and a robust estimate of the ridge parameter k is proposed.     

The Ubiquity of Statistics

The use of biased estimation in data analysis and model building is discussed. A review of the theory of ridge regression and its relation to generalized inverse regression is presented along with the results of a simulation experiment and three examples of the use of ridge regression in practice. Comments on variable selection procedures, model validation, and ridge and generalized inverse regression computation procedures are included. The examples studied here show that when the predictor variables are highly correlated, ridge regression produces coefficients which predict and extrapolate better than least squares and is a safe procedure for selecting variables.     

Confidence Interval for Shrinkage Parameters in Ridge Regression

In ridge regression, the estimation of ridge parameter k is an important problem. There are several methods available in the literature to do this job some what efficiently. However, no attempts were made to suggest a confidence interval for the ridge parameter using the knwoledge from the data. In this article, we propose a data dependent confidence interval for the ridge parameter k. The method of obtaining the confidence interval is illustrated with the help of a data set. A simulation study indicates that the empirical coverage probability of the suggested confidence intervals are quite high.     

Choosing shrinkage estimators for regression problems

A Bayesian formulation of the canonical form of the standard regression model is used to compare various Stein-type estimators and the ridge estimator of regression coefficients, A particular (“constant prior”) Stein-type estimator having the same pattern of shrinkage as the ridge estimator is recommended for use.     

THE CONSTRUCTION OF EQUILEVERAGE DESIGNS FOR MULTIPLE LINEAR REGRESSION

The hat matrix is widely used as a diagnostic tool in linear regression because it contains the leverages which the independent variables exert on the fitted values. In some experiments, cases with high leverage may be avoided by judicious choice of design for the independent variables. A variety of methods for constructing equileverage designs for linear regression are discussed. Such designs remove one of the factors, namely large leverage points, which can lead to nonrobust estimators and tests. In addition, a method is given for combining equileverage designs to test for lack of fit of the linear model.     

A predictive leverage statistic for quantile regression with measurement errors

Application of quantile regression models with measurement errors in predictors is becoming increasingly popular. High leverage points in predictors can have substantial impacts on these models. Here, we propose a predictive leverage statistic for these models, assuming that the measurement errors follow a multivariate normal distribution, and derive its exact distribution. We compare its performance versus known predictive leverage statistics using simulation and a real dataset. The proposed statistic is shown to have desirable features. It is also the first predictive leverage statistic having its distribution derived in a closed form.     

A Ridge-Regression Sidelight

A useful matrix result is applied in ridge regression to cast light on the form of the ridge-regression residual sum of squares as a power series in the ridge parameter. An example illustrates that the series may diverge or else converge very slowly, depending on the value of the ridge parameter.     

Choice of the ridge factor from the correlation matrix determinant

Ridge regression is the alternative method to ordinary least squares, which is mostly applied when a multiple linear regression model presents a worrying degree of collinearity. A relevant topic in ridge regression is the selection of the ridge parameter, and different proposals have been presented in the scientific literature. Since the ridge estimator is biased, its estimation is normally based on the calculation of the mean square error (MSE) without considering (to the best of our knowledge) whether the proposed value for the ridge parameter really mitigates the collinearity. With this goal and different simulations, this paper proposes to estimate the ridge parameter from the determinant of the matrix of correlation of the data, which verifies that the variance inflation factor (VIF) is lower than the traditionally established threshold. The possible relation between the VIF and the determinant of the matrix of correlation is also analysed. Finally, the contribution is illustrated with three real examples.     

An application of the local influence approach to ridge regression

In this study, the method of local influence, which was introduced by Cook as a general tool for assessing the influence of local departures from the underlying assumptions, is applied to ridge regression, by defining the maximum pseudo-likelihood ridge estimator obtained using the augmentation approach, because this method is suitable for likelihood-based models. In addition, an alternative local influence approach suggested by Billor and Loynes is applied to ridge regression. A comparison of these approaches and an example are given.     

最低工资对企业长、短期杠杆率的异质性影响

本文基于2007-2012年工业企业和全国县区级最低工资的匹配数据,探讨最低工资对企业杠杆率的影响,研究发现最低工资上升会对企业的长、短期杠杆率产生截然相反的影响。具体而言,最低工资标准每上升10%,企业长期杠杆率将上升0.42个百分点,而短期杠杆率将下降0.29个百分点。针对最低工资对企业长、短期杠杆率的异质性影响,本文给出了两种机制进行解释———“要素替代”和“盈利预期”效应。就作用渠道而言,“要素替代”效应下,最低工资上升意味着资本相对劳动更为便宜,这会促使企业更多地以资本替代劳动,表现为债务的增加,且主要反映在长期债务上。“盈利预期”效应下,企业生产成本上升会恶化企业盈利预期,使得企业减少投资,同时整体性地收缩债务,这主要对应于短期债务调整。进一步,将企业按照高/ 低劳动密集度以及高/ 低市场竞争度进行划分时,异质性分析结果表明,“要素替代”所导致的长期杠杆率上升只在高劳动密集度企业才成立,而“盈利预期”效应导致的短期杠杆率下降只在高市场竞争度企业才成立。     

Restricted Two Parameter Ridge Estimator

In 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.     

Influence Diagnostics in log-Birnbaum-Saunders Regression Models

In this paper we present various diagnostic methods for a linear regression model under a logarithmic Birnbaum-Saunders distribution for the errors, which may be applied for accelerated life testing or to compare the median lives of several populations. Some influence methods, such as the local influence, total local influence of an individual and generalized leverage are derived, analysed and discussed. We also present a connection between the local influence and generalized leverage methods. A discussion of the computation of the likelihood displacement as well as the normal curvature in the local influence method are presented. Finally, an example with real data is given for illustration.     

Good ridge estimators based on prior information

Ridge regression is re-examined and ridge estimators based on prior information are introduced. A necessary and sufficient condition is given for such ridge estimators to yield estimators of every nonnull linear combination of the regression coefficients with smaller mean square error than that of the Gauss-Markov best linear unbiased estimator.     

Reply

ABSTRACT

In the class of stochastic volatility (SV) models, leverage effects are typically specified through the direct correlation between the innovations in both returns and volatility, resulting in the dynamic leverage (DL) model. Recently, two asymmetric SV models based on threshold effects have been proposed in the literature. As such models consider only the sign of the previous return and neglect its magnitude, this paper proposes a dynamic asymmetric leverage (DAL) model that accommodates the direct correlation as well as the sign and magnitude of the threshold effects. A special case of the DAL model with zero direct correlation between the innovations is the asymmetric leverage (AL) model. The dynamic asymmetric leverage models are estimated by the Monte Carlo likelihood (MCL) method. Monte Carlo experiments are presented to examine the finite sample properties of the estimator. For a sample size of T = 2000 with 500 replications, the sample means, standard deviations, and root mean squared errors of the MCL estimators indicate only a small finite sample bias. The empirical estimates for S&;P 500 and TOPIX financial returns, and USD/AUD and YEN/USD exchange rates, indicate that the DAL class, including the DL and AL models, is generally superior to threshold SV models with respect to AIC and BIC, with AL typically providing the best fit to the data.