New estimation and feature selection methods in mixture‐of‐experts models

We study estimation and feature selection problems in mixture‐of‐experts models. An $l_2$ ‐penalized maximum likelihood estimator is proposed as an alternative to the ordinary maximum likelihood estimator. The estimator is particularly advantageous when fitting a mixture‐of‐experts model to data with many correlated features. It is shown that the proposed estimator is root‐$n$ consistent, and simulations show its superior finite sample behaviour compared to that of the maximum likelihood estimator. For feature selection, two extra penalty functions are applied to the $l_2$ ‐penalized log‐likelihood function. The proposed feature selection method is computationally much more efficient than the popular all‐subset selection methods. Theoretically it is shown that the method is consistent in feature selection, and simulations support our theoretical results. A real‐data example is presented to demonstrate the method. The Canadian Journal of Statistics 38: 519–539; 2010 © 2010 Statistical Society of Canada     

The Dantzig Selector in Cox's Proportional Hazards Model

Abstract. The Dantzig selector (DS) is a recent approach of estimation in high‐dimensional linear regression models with a large number of explanatory variables and a relatively small number of observations. As in the least absolute shrinkage and selection operator (LASSO), this approach sets certain regression coefficients exactly to zero, thus performing variable selection. However, such a framework, contrary to the LASSO, has never been used in regression models for survival data with censoring. A key motivation of this article is to study the estimation problem for Cox's proportional hazards (PH) function regression models using a framework that extends the theory, the computational advantages and the optimal asymptotic rate properties of the DS to the class of Cox's PH under appropriate sparsity scenarios. We perform a detailed simulation study to compare our approach with other methods and illustrate it on a well‐known microarray gene expression data set for predicting survival from gene expressions.     

Detecting the Dimensionality for Principal Components Model

A crucial issue for principal components analysis (PCA) is to determine the number of principal components to capture the variability of usually high-dimensional data. In this article the dimension detection for PCA is formulated as a variable selection problem for regressions. The adaptive LASSO is used for the variable selection in this application. Simulations demonstrate that this approach is more accurate than existing methods in some cases and competitive in some others. The performance of this model is also illustrated using a real example.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Adaptive LASSO for varying-coefficient partially linear measurement error models

This paper extends the adaptive LASSO (ALASSO) for simultaneous parameter estimation and variable selection to a varying-coefficient partially linear model where some of the covariates are subject to measurement errors of an additive form. We draw comparisons with the SCAD, and prove that both the ALASSO and the SCAD attain the oracle property under this setup. We further develop an algorithm in the spirit of LARS for finding the solution path of the ALASSO in practical applications. Finite sample properties of the proposed methods are examined in a simulation study, and a real data example based on the U.S. Department of Agriculture's Continuing Survey of Food Intakes by Individuals (CSFII) is considered.     

Edge selection for undirected graphs

This article explores an ‘Edge Selection’ procedure to fit an undirected graph to a given data set. Undirected graphs are routinely used to represent, model and analyse associative relationships among the entities on a social, biological or genetic network. Our proposed method combines the computational efficiency of least angle regression and at the same time ensures symmetry of the selected adjacency matrix. Various local and global properties of the edge selection path are explored analytically. In particular, a suitable parameter that controls the amount of shrinkage is identified and we consider several cross-validation techniques to choose an accurate predictive model on the path. The proposed method is illustrated with a detailed simulation study involving models with various levels of sparsity and variability in the nodal degree distributions. Finally, our method is used to select undirected graphs from various real data sets. We employ it for identifying the regulatory network of isoprenoid pathways from a gene-expression data and also to identify genetic network from a high-dimensional breast cancer study data.     

国民幸福感的指标体系构建与影响因素分析：基于LASSO的筛选方法

国民幸福感是经济社会发展和公共政策的终极目标。借鉴已有研究文献的幸福指数量表,本文构建一套适合于测度中国国民幸福感的指标体系,并通过全国性的问卷调查获取相关数据。为了有效地选择重要变量和消除估计偏差,本文采用新近发展的重要统计方法LASSO筛选法,先从6个个人特征变量和40个维度变量中筛选重要变量,然后再进行回归系数估计与显著性检验。回归结果表明：①性别、婚否和学历水平3个个人特征变量对幸福感有显著的影响;②9个维度变量通过显著性检验,其中家庭生活满意度、自我价值评价、社会福利保障满意度和生活方式健康度评价对幸福感的影响最为显著。在此基础上,本文又分男女、城乡、南北三个组别进行考察。最后,提出旨在提高国民幸福感的政策举措。     

基于状态空间图模型方法的投资组合优化决策

将lasso图理论合并到状态空间模型中,利用条件独立性且通过范数惩罚法对协方差阵进行估计。新方法兼具图模型和动态状态空间模型的优点。最后将该方法应用于欧洲股票市场进行投资组合优化决策,结果表明基于lasso图方法的状态空间模型的投资组合业绩要优于自回归和一般的状态空间模型。     

Shrinkage estimation in lognormal regression model for censored data

We introduce in this paper, the shrinkage estimation method in the lognormal regression model for censored data involving many predictors, some of which may not have any influence on the response of interest. We develop the asymptotic properties of the shrinkage estimators (SEs) using the notion of asymptotic distributional biases and risks. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the SEs is strictly less than the corresponding classical estimators. Furthermore, we study the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the SEs. A simulation study for various combinations of the inactive predictors and censoring percentages shows that the SEs perform better than the penalty estimators in certain parts of the parameter space, especially when there are many inactive predictors in the model. It also shows that the shrinkage and penalty estimators outperform the classical estimators. A real-life data example using Worcester heart attack study is used to illustrate the performance of the suggested estimators.