Fitting survival data with penalized Poisson regression

Cox’s proportional hazards model is the most common way to analyze survival data. The model can be extended in the presence of collinearity to include a ridge penalty, or in cases where a very large number of coefficients (e.g. with microarray data) has to be estimated. To maximize the penalized likelihood, optimal weights of the ridge penalty have to be obtained. However, there is no definite rule for choosing the penalty weight. One approach suggests maximization of the weights by maximizing the leave-one-out cross validated partial likelihood, however this is time consuming and computationally expensive, especially in large datasets. We suggest modelling survival data through a Poisson model. Using this approach, the log-likelihood of a Poisson model is maximized by standard iterative weighted least squares. We will illustrate this simple approach, which includes smoothing of the hazard function and move on to include a ridge term in the likelihood. We will then maximize the likelihood by considering tools from generalized mixed linear models. We will show that the optimal value of the penalty is found simply by computing the hat matrix of the system of linear equations and dividing its trace by a product of the estimated coefficients.     

Local Influence Analysis for The Ridge Regression Under Stochastic Linear Restrictions

In this article, we assess the local influence for the ridge regression of linear models with stochastic linear restrictions in the spirit of Cook by using the log-likelihood of the stochastic restricted ridge regression estimator. The diagnostics under the perturbations of constant variance, responses and individual explanatory variables are derived. We also assess the local influence of the stochastic restricted ridge regression estimator under the approach suggested by Billor and Loynes. At the end, a numerical example on the Longley data is given to illustrate the theoretic results.     

Cox regression analysis in presence of collinearity: an application to assessment of health risks associated with occupational radiation exposure

This paper considers the analysis of time to event data in the presence of collinearity between covariates. In linear and logistic regression models, the ridge regression estimator has been applied as an alternative to the maximum likelihood estimator in the presence of collinearity. The advantage of the ridge regression estimator over the usual maximum likelihood estimator is that the former often has a smaller total mean square error and is thus more precise. In this paper, we generalized this approach for addressing collinearity to the Cox proportional hazards model. Simulation studies were conducted to evaluate the performance of the ridge regression estimator. Our approach was motivated by an occupational radiation study conducted at Oak Ridge National Laboratory to evaluate health risks associated with occupational radiation exposure in which the exposure tends to be correlated with possible confounders such as years of exposure and attained age. We applied the proposed methods to this study to evaluate the association of radiation exposure with all-cause mortality.     

An application of the local influence approach to ridge regression

In this study, the method of local influence, which was introduced by Cook as a general tool for assessing the influence of local departures from the underlying assumptions, is applied to ridge regression, by defining the maximum pseudo-likelihood ridge estimator obtained using the augmentation approach, because this method is suitable for likelihood-based models. In addition, an alternative local influence approach suggested by Billor and Loynes is applied to ridge regression. A comparison of these approaches and an example are given.     

Using improved estimation strategies to combat multicollinearity

In this approach, some generalized ridge estimators are defined based on shrinkage foundation. Completely under the suspicion that some sub-space restrictions may occur, we present the estimators of the regression coefficients combining the idea of preliminary test estimator and Stein-rule estimator with the ridge regression methodology for normal models. Their exact risk expressions in addition to biases are derived and the regions of optimality of the estimators are exactly determined along with some numerical analysis. In this regard, the ridge parameter is determined in different disciplines.     

New Ridge Regression Estimator in Semiparametric Regression Models

In the context of ridge regression, the estimation of shrinkage parameter plays an important role in analyzing data. Many efforts have been put to develop the computation of risk function in different full-parametric ridge regression approaches using eigenvalues and then bringing an efficient estimator of shrinkage parameter based on them. In this respect, the estimation of shrinkage parameter is neglected for semiparametric regression model. Not restricted, but the main focus of this approach is to develop necessary tools for computing the risk function of regression coefficient based on the eigenvalues of design matrix in semiparametric regression. For this purpose the differencing methodology is applied. We also propose a new estimator for shrinkage parameter which is of harmonic type mean of ridge estimators. It is shown that this estimator performs better than all the existing ones for the regression coefficient. For our proposal, a Monte Carlo simulation study and a real dataset analysis related to housing attributes are conducted to illustrate the efficiency of shrinkage estimators based on the minimum risk and mean squared error criteria.     

An evaluation of ridge estimator in linear mixed models: an example from kidney failure data

This paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model.     

A small but practically useful modification to the Hodrick–Prescott filtering: A note

The Hodrick–Prescott (HP) filtering is widely applied to decompose macroeconomic time series, such as real Gross Domestic Product, into cyclical and trend components. This paper presents a small but practically useful modification to this approach. The reason why this modified filtering is of practical use is that it provides not only identical trend estimates as the HP filtering but also extrapolations of the trend. We provide a proof based on a ridge regression representation of the modified HP filtering. This is mainly because it enhances our understanding of the approach.     

Ridge Regression – A Simulation Study

In this article we assess the suitability of two new ridge estimators by means of a simulation study. We compare these estimators with well-known ridge estimators. We also make direct comparisons between the ordinary least squares (OLS) estimator and the ridge estimators by using ratio of the average total mean square error of the OLS estimator and the ridge estimators. We find that the new estimators perform well under certain conditions.     

Some finite sample properties of generalized ridge regression estimators

In this paper we study the mean square error properties of the generalized ridge estimator. We obtain the exact and the approximate bias and the mean square error of the operational generalized ridge estimator in terms of G( ) functions. We show, among other things, that the operational generalized ridge estimator does not dominate the ordinary least squares estimator up to a certain order of approximation. Finally, we note that the iterative procedures to obtain coverging ridge estimators should be used with caution.     

Generalized ridge regression and a generalization of the CP statistic

We consider a generalization of ridge regression and demonstrate advantages over ridge regression. We provide an empirical Bayes method for determining the ridge constants, using the Bayesian interpretation of ridge estimators, and show that this coincides with a method based on a generalization of the C_P statistic and the non-negative garrote. These provide an automatic variable selection procedure for the canonical variables.     

APPLICATIONS OF ROBUST ESTIMATION FOR VARIANCE COMPONENTS MODELS:THE DETECTION OF MAJOR GENE EFFECTS IN FINGER RIDGE COUNTS IN NORMAL AND FRAGILE X FAMILIES

A diagnostic technique is proposed to detect major gene effects and other systematic departures from a model for the trait means in the presence of outliers. The technique is based on the examination of residuals from fitting variance components models to quantitative pedigree data using robust statistical procedures. The approach is demonstrated using the total ridge count and ridge count of the middle finger from 54 extended families affected with the Fragile X syndrome, and a sample of 217 normal pedigrees.     

Bayesian Tobit quantile regression using g-prior distribution with ridge parameter

A Bayesian approach is proposed for coefficient estimation in the Tobit quantile regression model. The proposed approach is based on placing a g-prior distribution depends on the quantile level on the regression coefficients. The prior is generalized by introducing a ridge parameter to address important challenges that may arise with censored data, such as multicollinearity and overfitting problems. Then, a stochastic search variable selection approach is proposed for Tobit quantile regression model based on g-prior. An expression for the hyperparameter g is proposed to calibrate the modified g-prior with a ridge parameter to the corresponding g-prior. Some possible extensions of the proposed approach are discussed, including the continuous and binary responses in quantile regression. The methods are illustrated using several simulation studies and a microarray study. The simulation studies and the microarray study indicate that the proposed approach performs well.     

Impacts of auxiliary information on outliers in regression

Iheil and Goldberger (1961) and Theil (1963) founded the mixed regression approach, Their mixed regression estimator is essentially a large class of estimators that includes ridge, generalized ridge and shrinkage estimators, Properties of these estimators when data contain outliers have not been examined extensively. The present investigation shows that the mixed regression estimator, when observationsare subject to shift in means and variances, is uniformly superior, in terms of squared bias and variance, to the least squares estimator.     

Second-derivative functional regression with applications to near infra-red spectroscopy

A linear regression method to predict a scalar from a discretized smooth function is presented. The method takes into account the functional nature of the predictors and the importance of the second derivative in spectroscopic applications. This motivates a functional inner product that can be used as a roughness penalty. Using this inner product, we derive a linear prediction method that is similar to ridge regression but with different shrinkage characteristics. We describe its practical implementation and we address the problem of computing the second derivatives nonparametrically. We apply the method to a calibration example using near infra-red spectra. We conclude with a discussion comparing our approach with other regression algorithms.     

An example of ridge regression difficulties

A simple consumption function is used to illustrate two fundamental difficulties with ridge regression and similarly motivated procedures. The first is the ambiguity of multicollinearity measures for judging the data's “ill-conditioning”. The second is the sensitivity of the estimates to the arbitrary normalization of the model. Neither of these poses a problem for least squares or Bayesian estimates. The logical restructuring of ridge procedures to avoid these difficulties leads to a more explicitly Bayesian approach.     

More on the restricted ridge regression estimation

Several alternative methods for derivation of the restricted ridge regression estimator (RRRE) are provided. Theoretical comparison and relationship of RRRE with related methods for regression with the multicollinearity problem are described. We also find inter-connections among RRRE, ordinary ridge regression estimator (ORRE), restricted least squares estimator (RLSE), modified ridge regression estimator (MRRE) and restricted modified generalized ridge estimator (RMGRE). Finally, numerical comparison, in addition to theoretical derivation, is also conducted with a Monte Carlo simulation and a real data example.     

A Generalized View on Continuum Regression

ABSTRACT. We generalize the relationship between continuum regression (Stone & Brooks, 1990) and ridge regression, by showing that any optimization principle will yield a regressor proportional to a ridge regressor, provided only that the principle implies maximizing a function of the regressor's sample correlation coefficient and its sample variance. This relationship shows that continuum regression as defined via ridge regression ("least squares ridge regression") is a more generally valid methodology than previously realized, and also opens up for alternative choices of its second and subsequent factors.     

Local likelihood tracking of fault lines and boundaries

We suggest locally parametric methods for estimating curves, such as boundaries of density supports or fault lines in response surfaces, in a variety of spatial problems. The methods are based on spatial approximations to the local likelihood that the curve passes through a given point in the plane, as a function of that point. The local likelihood might be a regular likelihood computed locally, with kernel weights (e.g. in the case of support boundary estimation) or a local version of a likelihood ratio statistic (e.g. in fault line estimation). In either case, the local likelihood surface represents a function which is relatively large near the target curve, and relatively small elsewhere. Therefore, the curve may be estimated as a ridge line of the surface; we require only a numerical algorithm for tracking the projection of a ridge into the plane. This approach offers several potential advantages over alternative methods. First, the local (log-)likelihood surface can be graphed, and the degree of 'ridginess' assessed visually, to determine how the level of local smoothing should be varied in different spatial locations in order to emphasize the ridge and hence the curve adequately. Secondly, the local likelihood surface does not need to be computed in anything like its entirety; once we have a reasonable approximation to a point on the curve we may track it by numerically 'walking along' the ridge line. Thirdly, the method is appropriate without change for many different types of spatial explanatory variables—gridded, stochastic or otherwise. Three examples are explored in detail; fault lines in response surfaces and in intensity or density surfaces, and boundaries of supports of probability densities.     

Using Bayesian Techniques for Data Pooling in Regional Payroll Forecasting

This article adapts to the regional level a multicountry technique recently used by Garcia-Ferrer, Highfield, Palm, and Zellner (1987) and extended by Zellner and Hong (1987) to forecast the growth rates in gross national product across nine countries. This forecasting methodology is applied to the regional level by modeling payroll formation in seven Ohio metropolitan areas. We compare the forecasting performance of these procedures with that of a ridge estimator and find that the ridge estimator produces forecasts equal to or better than those from the newly proposed estimators. We conclude that the ridge estimator, which does not reference the pooled data information introduced by the newly proposed techniques, may serve as a benchmark against which to judge the relative importance of this kind of information in improving forecasts.