Lasso with convex loss: Model selection consistency and estimation

Abstract

Variable selection is a fundamental challenge in statistical learning if one works with data sets containing huge amount of predictors. In this artical we consider procedures popular in model selection: Lasso and adaptive Lasso. Our goal is to investigate properties of estimators based on minimization of Lasso-type penalized empirical risk with a convex loss function, in particular nondifferentiable. We obtain theorems concerning rate of convergence in estimation, consistency in model selection and oracle properties for Lasso estimators if the number of predictors is fixed, i.e. it does not depend on the sample size. Moreover, we study properties of Lasso and adaptive Lasso estimators on simulated and real data sets.     

Statistical Inference of Adaptive Designs with Binary Responses

Adaptive designs of clinical trials are ethical alternatives when the traditional randomization becomes ethically infeasible in desperate medical situations. However, such a design creates a dependency among trial data and its statistical analysis becomes more complex than the analysis for traditional randomized clinical trials. In this article, we examine adaptive designs with dichotomous responses from two treatments and extend some commonly used statistical methods for independent data. Under a regularity condition, the estimated odds ratio and its logarithm are shown to follow asymptotically normal distributions. Moreover, the ordinary goodness-of-fit test statistic for two-by-two contingency tables with dependent data is shown to be asymptotically chi-square distributed. We also discuss the consistency of maximum likelihood estimators of the unknown parameters for a wide class of adaptive designs.     

Covariate Selection for the Semiparametric Additive Risk Model

Abstract.  This paper considers covariate selection for the additive hazards model. This model is particularly simple to study theoretically and its practical implementation has several major advantages to the similar methodology for the proportional hazards model. One complication compared with the proportional model is, however, that there is no simple likelihood to work with. We here study a least squares criterion with desirable properties and show how this criterion can be interpreted as a prediction error. Given this criterion, we define ridge and Lasso estimators as well as an adaptive Lasso and study their large sample properties for the situation where the number of covariates p is smaller than the number of observations. We also show that the adaptive Lasso has the oracle property. In many practical situations, it is more relevant to tackle the situation with large p compared with the number of observations. We do this by studying the properties of the so-called Dantzig selector in the setting of the additive risk model. Specifically, we establish a bound on how close the solution is to a true sparse signal in the case where the number of covariates is large. In a simulation study, we also compare the Dantzig and adaptive Lasso for a moderate to small number of covariates. The methods are applied to a breast cancer data set with gene expression recordings and to the primary biliary cirrhosis clinical data.     

Sparse additive models

Summary.  We present a new class of methods for high dimensional non-parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non-parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin. They are also closely related to the COSSO model of Lin and Zhang but decouple smoothing and sparsity, enabling the use of arbitrary non-parametric smoothers. We give an analysis of the theoretical properties of sparse additive models and present empirical results on synthetic and real data, showing that they can be effective in fitting sparse non-parametric models in high dimensional data.     

A simple two-stage design for quantitative responses with application to a study in diabetic neuropathic pain

Two-stage designs offer substantial advantages for early phase II studies. The interim analysis following the first stage allows the study to be stopped for futility, or more positively, it might lead to early progression to the trials needed for late phase II and phase III. If the study is to continue to its second stage, then there is an opportunity for a revision of the total sample size. Two-stage designs have been implemented widely in oncology studies in which there is a single treatment arm and patient responses are binary. In this paper the case of two-arm comparative studies in which responses are quantitative is considered. This setting is common in therapeutic areas other than oncology. It will be assumed that observations are normally distributed, but that there is some doubt concerning their standard deviation, motivating the need for sample size review. The work reported has been motivated by a study in diabetic neuropathic pain, and the development of the design for that trial is described in detail.     

Doubly adaptive biased coin designs with heterogeneous responses

Doubly adaptive biased coin design (DBCD) is an important family of response-adaptive randomization procedures for clinical trials. It uses sequentially updated estimation to skew the allocation probability to favor the treatment that has performed better thus far. An important assumption for the DBCD is the homogeneity assumption for the patient responses. However, this assumption may be violated in many sequential experiments. Here we prove the robustness of the DBCD against certain time trends in patient responses. Strong consistency and asymptotic normality of the design are obtained under some widely satisfied conditions. Also, we propose a general weighted likelihood method to reduce the bias caused by the heterogeneity in the inference after a trial. Some numerical studies are also presented to illustrate the finite sample properties of DBCD.     

Approximating the risk score for disease diagnosis using MARS

In disease screening and diagnosis, often multiple markers are measured and combined to improve the accuracy of diagnosis. McIntosh and Pepe [Combining several screening tests: optimality of the risk score, Biometrics 58 (2002), pp. 657–664] showed that the risk score, defined as the probability of disease conditional on multiple markers, is the optimal function for classification based on the Neyman–Pearson lemma. They proposed a two-step procedure to approximate the risk score. However, the resulting receiver operating characteristic (ROC) curve is only defined in a subrange (L, h) of false-positive rates in (0,1) and the determination of the lower limit L needs extra prior information. In practice, most diagnostic tests are not perfect, and it is usually rare that a single marker is uniformly better than the other tests. Using simulation, I show that multivariate adaptive regression spline is a useful tool to approximate the risk score when combining multiple markers, especially when ROC curves from multiple tests cross. The resulting ROC is defined in the whole range of (0,1) and is easy to implement and has intuitive interpretation. The sample code of the application is shown in the appendix.     

论适应性表征及其方法论意义

适应性是主体适应其环境的特性,是物理演化系统和生物演化系统的本质属性,反映在认知上就是适应性表征。适应性表征也因此成为知识显现的方式和创造的核心,具有协调性、匹配性、互补性、模拟性和类比性,其表现方式有直接具象表征、间接具象表征、直接抽象表征和间接抽象表征,分别对应于经验主义、建构经验主义、理性主义和科学实在论。适应性表征的实现是通过科学理论核心概念的变化、定律的凝练、理论的更替、模型推理和世界观的改变展开的,具有经验上的适当性。这种经验适当性不仅是感知层次的体验,更是一种认知模式。在认知意义上,经验也是基于心理模型的,心理建模的路径有思想语言、心理表象、心理命题、思想实验和心理模拟推理。这种基于心理建模的适应性表征为科学认知和科学发现提供了有益的方法论。     

变革社会中的政治资源配置

政治资源配置是变革社会中的一个重要发展参数。社会转型时期 ,既有政治资源的配置结构有所松动 ,出现分散性发展的倾向 ,而其既有之高度集中性也愈益显示某些弊端。因应形势的变化 ,中国共产党有必要引导政治资源配置作出适应性调整 ,促使其向合理性和适度社会倾向性变革 ,并增强自身开发、掌用政治资源的能力 ,确保党在政治资源配置中的主导地位     

HF通信网中的突发自适应选频技术

介绍了突发自适应选频技术在ＨＦ通信网中的应用，分析了突发自适应选频的特点及网络应用的优势，设计了一套用于网络突发选频的协议，讨论了协议中的重要参数设定和协议性能，并提出了多种方案供优化