Bayesian Model Selection and Averaging in Additive and Proportional Hazards Models

Although Cox proportional hazards regression is the default analysis for time to event data, there is typically uncertainty about whether the effects of a predictor are more appropriately characterized by a multiplicative or additive model. To accommodate this uncertainty, we place a model selection prior on the coefficients in an additive-multiplicative hazards model. This prior assigns positive probability, not only to the model that has both additive and multiplicative effects for each predictor, but also to sub-models corresponding to no association, to only additive effects, and to only proportional effects. The additive component of the model is constrained to ensure non-negative hazards, a condition often violated by current methods. After augmenting the data with Poisson latent variables, the prior is conditionally conjugate, and posterior computation can proceed via an efficient Gibbs sampling algorithm. Simulation study results are presented, and the methodology is illustrated using data from the Framingham heart study.     

Checking a Semiparametric Additive Risk Model

McKeague and Sasieni [A partly parametric additive risk model. Biometrika 81 (1994) 501] propose a restriction of Aalen’s additive risk model by the additional hypothesis that some of the covariates have time-independent influence on the intensity of the observed counting process. We introduce goodness-of-fit tests for this semiparametric Aalen model. The asymptotic distribution properties of the test statistics are derived by means of martingale techniques. The tests can be adjusted to detect particular alternatives. As one of the most important alternatives we consider Cox’s proportional hazards model. We present simulation studies and an application to a real data set.     

DASSO: connections between the Dantzig selector and lasso

Summary.  We propose a new algorithm, DASSO, for fitting the entire coefficient path of the Dantzig selector with a similar computational cost to the least angle regression algorithm that is used to compute the lasso. DASSO efficiently constructs a piecewise linear path through a sequential simplex-like algorithm, which is remarkably similar to the least angle regression algorithm. Comparison of the two algorithms sheds new light on the question of how the lasso and Dantzig selector are related. In addition, we provide theoretical conditions on the design matrix X under which the lasso and Dantzig selector coefficient estimates will be identical for certain tuning parameters. As a consequence, in many instances, we can extend the powerful non-asymptotic bounds that have been developed for the Dantzig selector to the lasso. Finally, through empirical studies of simulated and real world data sets we show that in practice, when the bounds hold for the Dantzig selector, they almost always also hold for the lasso.     

Testing Covariate Effects in Aalen's Linear Hazard Model

The performance of tests in Aalen's linear regression model is studied using asymptotic power calculations and stochastic simulation. Aalen's original least squares test is compared to two modifications: a weighted least squares test with correct weights and a test where the variance is re-estimated under the null hypothesis. The test with re-estimated variance provides the highest power of the tests for the setting of this paper, and the gain is substantial for covariates following a skewed distribution like the exponential. It is further shown that Aalen's choice for weight function with re-estimated variance is optimal in the one-parameter case against proportional alternatives.     

The Dantzig Selector in Cox's Proportional Hazards Model

Abstract. The Dantzig selector (DS) is a recent approach of estimation in high‐dimensional linear regression models with a large number of explanatory variables and a relatively small number of observations. As in the least absolute shrinkage and selection operator (LASSO), this approach sets certain regression coefficients exactly to zero, thus performing variable selection. However, such a framework, contrary to the LASSO, has never been used in regression models for survival data with censoring. A key motivation of this article is to study the estimation problem for Cox's proportional hazards (PH) function regression models using a framework that extends the theory, the computational advantages and the optimal asymptotic rate properties of the DS to the class of Cox's PH under appropriate sparsity scenarios. We perform a detailed simulation study to compare our approach with other methods and illustrate it on a well‐known microarray gene expression data set for predicting survival from gene expressions.     

Component Selection in the Additive Regression Model

Abstract. Similar to variable selection in the linear model, selecting significant components in the additive model is of great interest. However, such components are unknown, unobservable functions of independent variables. Some approximation is needed. We suggest a combination of penalized regression spline approximation and group variable selection, called the group‐bridge‐type spline method (GBSM), to handle this component selection problem with a diverging number of correlated variables in each group. The proposed method can select significant components and estimate non‐parametric additive function components simultaneously. To make the GBSM stable in computation and adaptive to the level of smoothness of the component functions, weighted power spline bases and projected weighted power spline bases are proposed. Their performance is examined by simulation studies. The proposed method is extended to a partial linear regression model analysis with real data, and gives reliable results.     

Semiparametric Additive Accelerated Life Models

In this paper we investigate the asymptotic properties of estimators obtained for the semiparametric additive accelerated life model proposed by Bagdonavicius & Nikulin (1995). This model generalizes the well known additive hazards model of survival analysis and is close to the general transformation model (see Dabrowska & Doksum, 1988). Asymptotic properties of the estimator of the regression parameter and the estimator of the reliability function are given in the case of right censoring for discretized data and a numerical example illustrates these results.     

Lasso-type estimation for covariate-adjusted linear model

Lasso is popularly used for variable selection in recent years. In this paper, lasso-type penalty functions including lasso and adaptive lasso are employed in simultaneously variable selection and parameter estimation for covariate-adjusted linear model, where the predictors and response cannot be observed directly and distorted by some observable covariate through some unknown multiplicative smooth functions. Estimation procedures are proposed and some asymptotic properties are obtained under some mild conditions. It deserves noting that under appropriate conditions, the adaptive lasso estimator correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance, i.e. the adaptive lasso estimator has the oracle property in the sense of Fan and Li [6]. Simulation studies are carried out to examine its performance in finite sample situations and the Boston Housing data is analyzed for illustration.     

On Goodness-of-Fit Tests for Aalen's Additive Risk Model

Abstract.  In this paper we propose goodness-of-fit tests for Aalen's additive risk model. They are based on test statistics the asymptotic distributions of which are determined under both the null and alternative hypotheses. The results are derived using martingale techniques for counting processes. An important feature of these tests is that they can be adjusted to particular alternatives. One of the alternatives we consider is Cox's multiplicative risk model. It is perhaps remarkable that such a test needs no estimate of the baseline hazard in the Cox model. We present simulation studies which give an impression of the performance of the proposed tests. In addition, the tests are applied to real data sets.     

Variable Selection for Semiparametric Partially Linear Covariate-Adjusted Regression Models

In this article, the partially linear covariate-adjusted regression models are considered, and the penalized least-squares procedure is proposed to simultaneously select variables and estimate the parametric components. The rate of convergence and the asymptotic normality of the resulting estimators are established under some regularization conditions. With the proper choices of the penalty functions and tuning parameters, it is shown that the proposed procedure can be as efficient as the oracle estimators. Some Monte Carlo simulation studies and a real data application are carried out to assess the finite sample performances for the proposed method.     

Selection of a stroke risk model based on transcranial Doppler ultrasound velocity

Increased transcranial Doppler ultrasound (TCD) velocity is an indicator of cerebral infarction in children with sickle cell disease (SCD). In this article, the parallel genetic algorithm (PGA) is used to select a stroke risk model with TCD velocity as the response variable. Development of such a stroke risk model leads to the identification of children with SCD who are at a higher risk of stroke and their treatment in the early stages. Using blood velocity data from SCD patients, it is shown that the PGA is an easy-to-use computationally variable selection tool. The results of the PGA are also compared with those obtained from the stochastic search variable selection method, the Dantzig selector and conventional techniques such as stepwise selection and best subset selection.     

Sparse group variable selection based on quantile hierarchical Lasso

The group Lasso is a penalized regression method, used in regression problems where the covariates are partitioned into groups to promote sparsity at the group level [27 M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B 68 (2006), pp. 49–67. doi: 10.1111/j.1467-9868.2005.00532.x[Crossref] , [Google Scholar]]. Quantile group Lasso, a natural extension of quantile Lasso [25 Y. Wu and Y. Liu, Variable selection in quantile regression, Statist. Sinica 19 (2009), pp. 801–817.[Web of Science ®] , [Google Scholar]], is a good alternative when the data has group information and has many outliers and/or heavy tails. How to discover important features that are correlated with interest of outcomes and immune to outliers has been paid much attention. In many applications, however, we may also want to keep the flexibility of selecting variables within a group. In this paper, we develop a sparse group variable selection based on quantile methods which select important covariates at both the group level and within the group level, which penalizes the empirical check loss function by the sum of square root group-wise L₁-norm penalties. The oracle properties are established where the number of parameters diverges. We also apply our new method to varying coefficient model with categorial effect modifiers. Simulations and real data example show that the newly proposed method has robust and superior performance.     

The Additive Nonparametric and Semiparametric Aalen Model as the Rate Function for a Counting Process

We use the additive risk model of Aalen (Aalen, 1980) as a model for the rate of a counting process. Rather than specifying the intensity, that is the instantaneous probability of an event conditional on the entire history of the relevant covariates and counting processes, we present a model for the rate function, i.e., the instantaneous probability of an event conditional on only a selected set of covariates. When the rate function for the counting process is of Aalen form we show that the usual Aalen estimator can be used and gives almost unbiased estimates. The usual martingale based variance estimator is incorrect and an alternative estimator should be used. We also consider the semi-parametric version of the Aalen model as a rate model (McKeague and Sasieni, 1994) and show that the standard errors that are computed based on an assumption of intensities are incorrect and give a different estimator. Finally, we introduce and implement a test-statistic for the hypothesis of a time-constant effect in both the non-parametric and semi-parametric model. A small simulation study was performed to evaluate the performance of the new estimator of the standard error.     

Lassoing the HAR Model: A Model Selection Perspective on Realized Volatility Dynamics

Realized volatility computed from high-frequency data is an important measure for many applications in finance, and its dynamics have been widely investigated. Recent notable advances that perform well include the heterogeneous autoregressive (HAR) model which can approximate long memory, is very parsimonious, is easy to estimate, and features good out-of-sample performance. We prove that the least absolute shrinkage and selection operator (Lasso) recovers the lags structure of the HAR model asymptotically if it is the true model, and we present Monte Carlo evidence in finite samples. The HAR model's lags structure is not fully in agreement with the one found using the Lasso on real data. Moreover, we provide empirical evidence that there are two clear breaks in structure for most of the assets we consider. These results bring into question the appropriateness of the HAR model for realized volatility. Finally, in an out-of-sample analysis, we show equal performance of the HAR model and the Lasso approach.     

Adjusting and Comparing Survival Curves by Means of an Additive Risk Model

Survival curves may be adjusted for covariates using Aalen's additive risk model. Survival curves may be compared by taking the ratio of two adjusted survival curves; the ratio is denoted the generalized relative survival rate. Adjusting both survival curves for all but one of a common set of covariates gives the partial relative survival rate, which measures the covariate-specific contribution to the generalized relative survival rate. The generalized and partial relative survival rates have interpretations similar to the traditional relative survival rates frequently used in cancer epidemiology. In fact, the traditional relative survival rate can be generalized to a regression context using the additive risk model. This population-adjusted relative survival rate is an alternative and useful method for removing confounding effects of age, cohorts, and sex. The authors use a data set of malignant melanoma patients diagnosed from 1965 to 1974 in Norway. The 25-year survival of 1967 individuals is studied.     

Developing Ridge Parameters for SUR Model

This paper proposes a number of procedures for developing new biased estimators of the seemingly unrelated regression (SUR) parameters, when the explanatory variables are affected by multicollinearity. Several ridge parameters are proposed and then compared in terms of the trace mean squared error (TMSE) and (PR) criteria. The PR criterion is the proportion of replication (out of 1,000) for which the SUR version of the generalized least squares (SGLS) estimator has a smaller TMSE than others. The study was performed using Monte Carlo simulations where the number of equations in the system, the number of observations, the correlation among equations, and the correlation between explanatory variables have been varied. For each model, we performed 1,000 replications. Our results show that under certain conditions some of the proposed SUR ridge parameters, (R _Sgeom , R _Skmed , R _Sqarith , and R _Sqmax ), performed well when compared, in terms of TMSE and PR criteria, with other proposed and popular existing ridge parameters. In large samples and when the collinearity between the explanatory variables is not high, the unbiased SUR estimator (SGLS), performed better than the other ridge parameters.     

SNP selection for predicting a quantitative trait

Molecular markers combined with powerful statistical tools have made it possible to detect and analyze multiple loci on the genome that are responsible for the phenotypic variation in quantitative traits. The objectives of the study presented in this paper are to identify a subset of single nucleotide polymorphism (SNP) markers that are associated with a particular trait and to construct a model that can best predict the value of the trait given the genotypic information of the SNPs using a three-step strategy. In the first step, a genome-wide association test is performed to screen SNPs that are associated with the quantitative trait of interest. SNPs with p-values of less than 5% are then analyzed in the second step. In the second step, a large number of randomly selected models, each consisting of a fixed number of randomly selected SNPs, are analyzed using the least angle regression method. This step will further remove redundant SNPs due to the complicated association among SNPs. A subset of SNPs that are shown to have a significant effect on the response trait more often than by chance are considered for the third step. In the third step, two alternative methods are considered: the least angle shrinkage and selection operation and sparse partial least squares regression. For both methods, the predictive ability of the fitted model is evaluated by an independent test set. The performance of the proposed method is illustrated by the analysis of a real data set on Canadian Holstein cattle.     

Coverage Properties of Confidence Intervals for Generalized Additive Model Components

Abstract. We study the coverage properties of Bayesian confidence intervals for the smooth component functions of generalized additive models (GAMs) represented using any penalized regression spline approach. The intervals are the usual generalization of the intervals first proposed by Wahba and Silverman in 1983 and 1985, respectively, to the GAM component context. We present simulation evidence showing these intervals have close to nominal ‘across‐the‐function’ frequentist coverage probabilities, except when the truth is close to a straight line/plane function. We extend the argument introduced by Nychka in 1988 for univariate smoothing splines to explain these results. The theoretical argument suggests that close to nominal coverage probabilities can be achieved, provided that heavy oversmoothing is avoided, so that the bias is not too large a proportion of the sampling variability. The theoretical results allow us to derive alternative intervals from a purely frequentist point of view, and to explain the impact that the neglect of smoothing parameter variability has on confidence interval performance. They also suggest switching the target of inference for component‐wise intervals away from smooth components in the space of the GAM identifiability constraints.     

Nonparametric Additive Instrumental Variable Estimator: A Group Shrinkage Estimation Perspective

In this article, we study a nonparametric approach regarding a general nonlinear reduced form equation to achieve a better approximation of the optimal instrument. Accordingly, we propose the nonparametric additive instrumental variable estimator (NAIVE) with the adaptive group Lasso. We theoretically demonstrate that the proposed estimator is root-n consistent and asymptotically normal. The adaptive group Lasso helps us select the valid instruments while the dimensionality of potential instrumental variables is allowed to be greater than the sample size. In practice, the degree and knots of B-spline series are selected by minimizing the BIC or EBIC criteria for each nonparametric additive component in the reduced form equation. In Monte Carlo simulations, we show that the NAIVE has the same performance as the linear instrumental variable (IV) estimator for the truly linear reduced form equation. On the other hand, the NAIVE performs much better in terms of bias and mean squared errors compared to other alternative estimators under the high-dimensional nonlinear reduced form equation. We further illustrate our method in an empirical study of international trade and growth. Our findings provide a stronger evidence that international trade has a significant positive effect on economic growth.     

Comparisons among regression estimators under the generalized mean square error criterion

It is not always prossible to establish a preference ordering among regression estimators in terms of the generalized mean square error criterion. In the paper, we determine when it is feasible to use this criteion to couduct comparisons among ordinary least squares, principal components, ridge regression, and shrunken least squares estimators.