Adaptive elastic net-penalized quantile regression for variable selection

Abstract

There has been much attention on the high-dimensional linear regression models, which means the number of observations is much less than that of covariates. Considering the fact that the high dimensionality often induces the collinearity problem, in this article, we study the penalized quantile regression with the elastic net (EnetQR) that combines the strengths of the quadratic regularization and the lasso shrinkage. We investigate the weak oracle property of the EnetQR under mild conditions in the high dimensional setting. Moreover, we propose a two-step procedure, called adaptive elastic net quantile regression (AEnetQR), in which the weight vector in the second step is constructed from the EnetQR estimate in the first step. This two-step procedure is justified theoretically to possess the weak oracle property. The finite sample properties are performed through the Monte Carlo simulation and a real-data analysis.     

On variable selection in generalized linear and related regression models

This paper is concerned with selection of explanatory variables in generalized linear models (GLM). The class of GLM's is quite large and contains e.g. the ordinary linear regression, the binary logistic regression, the probit model and Poisson regression with linear or log-linear parameter structure. We show that, through an approximation of the log likelihood and a certain data transformation, the variable selection problem in a GLM can be converted into variable selection in an ordinary (unweighted) linear regression model. As a consequence no specific computer software for variable selection in GLM's is needed. Instead, some suitable variable selection program for linear regression can be used. We also present a simulation study which shows that the log likelihood approximation is very good in many practical situations. Finally, we mention briefly possible extensions to regression models outside the class of GLM's.     

Normal approximation in an urn model with indistinguishable balls

For the Bose-Einstein Statistics, where n indistinguishable balls are distributed in m urns such that all the arrangements are equally likely, define the random variables

M_k = number of urns containing exactly k balls each;

N_k = number of urns containing at least k balls each.

We consider the approximation of the distributions of M_k and N_k by suitable normal distributions, for large but finite m. Estimates are found for the error in the approximation to both the probability mass function and the distribution function in each case. These results apply also to the alternative model where no urn is allowed to be empty. The results are illustrated by some numerical examples.     

Dynamic Forecasts of Qualitative Variables

This article presents a new Qual VAR model for incorporating information from qualitative and/or discrete variables in vector autoregressions. With a Qual VAR, it is possible to create dynamic forecasts of the qualitative variable using standard VAR projections. Previous forecasting methods for qualitative variables, in contrast, produce only static forecasts. I apply the Qual VAR to forecasting the 2001 business recession out of sample and to analyzing the Romer and Romer narrative measure of monetary policy contractions as an endogenous variable in a VAR. Out of sample, the model predicts the timing of the 2001 recession quite well relative to the recession probabilities put forth at the time by professional forecasters. Qual VARs—which include information about the qualitative variable—can also enhance the quality of density forecasts of the other variables in the system.     

The Effect of Pseudo-Exogenous Instrumental Variables on Hausman Test

Hausman test is popularly used to examine the endogeneity of explanatory variables in a regression model. To derive a well-defined asymptotic distribution of Hausman test, the correlation between the instrumental variables and the error term needs to converge to zero. However, it is possible that there remains considerable correlation in finite samples between the instruments and the error, even though their correlation eventually converges to zero. This article investigates the potential problem that such “pseudo-exogenous” instruments may create. We show that the performance of Hausman test is deteriorated when the instruments are asymptotically exogenous but endogenous in finite samples, through Monte Carlo simulations.     

Computing Marginal Likelihoods via Posterior Sampling

This article presents a novel and simple approach to the estimation of a marginal likelihood, in a Bayesian context. The estimate is based on a Markov chain output which provides samples from the posterior distribution and an additional latent variable. It is the mean of this latent variable which provides the estimate for the value of the marginal likelihood.     

Group Variable Selection with Oracle Property by Weight-Fused Adaptive Elastic Net Model for Strongly Correlated Data

This paper is the generalization of weight-fused elastic net (Fu and Xu, 2012 Fu, G., Xu, Q. (2012). Grouping variable selection by weight fused elastic net for multi-collinear data. Communications in Statistics-Simulation and Computation 41(2):205–221.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), which performs group variable selection by combining weight-fused LASSO(wfLasso) and elastic net (Zou and Hastie, 2005 Zou, H., Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67(2):301–320.[Crossref], [Web of Science ®] , [Google Scholar]) penalties. In this study, the elastic net penalty is replaced by adaptive elastic net penalty (AdaEnet) (Zou and Zhang, 2009 Zou, H., Zhang, H. (2009). On the adaptive elastic-net with a diverging number of parameters. Annals of Statistics 37(4):1733–1751.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), and a new group variable selection algorithm with oracle property (Fan and Li, 2001 Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96(456):1348–1360.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]; Zou, 2006 Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101(476):1418–1429.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) is obtained.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

Design-based random permutation models with auxiliary information

We extend the random permutation model to obtain the best linear unbiased estimator of a finite population mean accounting for auxiliary variables under simple random sampling without replacement (SRS) or stratified SRS. The proposed method provides a systematic design-based justification for well-known results involving common estimators derived under minimal assumptions that do not require specification of a functional relationship between the response and the auxiliary variables.     

Lower posterior death probabilities from a quick medical response in road traffic accidents

Introduction: We use data from Spain on roads and motorways traffic accidents in May 2004 to quantify the statistical association between quick medical response time and mortality rate. Method: Probit and logit parameters are estimated by a Bayesian method in which samples from the posterior densities are obtained through an MCMC simulation scheme. We provide posterior credible intervals and posterior partial effects of a quick medical response at several time levels over which we express our prior beliefs. Results: A reduction of 5 min, from a 25-min response-time level, is associated with lower posterior probabilities of death in roads and motorways accidents of 24% and 30%, respectively.