Dealing with big data: comparing dimension reduction and shrinkage regression methods

In the past decades, the number of variables explaining observations in different practical applications increased gradually. This has led to heavy computational tasks, despite of widely using provisional variable selection methods in data processing. Therefore, more methodological techniques have appeared to reduce the number of explanatory variables without losing much of the information. In these techniques, two distinct approaches are apparent: ‘shrinkage regression’ and ‘sufficient dimension reduction’. Surprisingly, there has not been any communication or comparison between these two methodological categories, and it is not clear when each of these two approaches are appropriate. In this paper, we fill some of this gap by first reviewing each category in brief, paying special attention to the most commonly used methods in each category. We then compare commonly used methods from both categories based on their accuracy, computation time, and their ability to select effective variables. A simulation study on the performance of the methods in each category is generated as well. The selected methods are concurrently tested on two sets of real data which allows us to recommend conditions under which one approach is more appropriate to be applied to high-dimensional data.     

Profiled adaptive Elastic-Net procedure for partially linear models with high-dimensional covariates

We study variable selection for partially linear models when the dimension of covariates diverges with the sample size. We combine the ideas of profiling and adaptive Elastic-Net. The resulting procedure has oracle properties and can handle collinearity well. A by-product is the uniform bound for the absolute difference between the profiled and original predictors. We further examine finite sample performance of the proposed procedure by simulation studies and analysis of a labor-market dataset for an illustration.