Doubly sparse regression incorporating graphical structure among predictors

Recent research has demonstrated that information learned from building a graphical model on the predictor set of a regularized linear regression model can be leveraged to improve prediction of a continuous outcome. In this article, we present a new model that encourages sparsity at both the level of the regression coefficients and the level of individual contributions in a decomposed representation. This model provides parameter estimates with a finite sample error bound and exhibits robustness to errors in the input graph structure. Through a simulation study and the analysis of two real data sets, we demonstrate that our model provides a predictive benefit when compared to previously proposed models. Furthermore, it is a highly flexible model that provides a unified framework for the fitting of many commonly used regularized regression models. The Canadian Journal of Statistics 47: 729–747; 2019 © 2019 Statistical Society of Canada     

B-spline estimation for partially linear varying coefficient composite quantile regression models

Abstract

In this article, a new composite quantile regression estimation (CQR) approach is proposed for partially linear varying coefficient models (PLVCM) under composite quantile loss function with B-spline approximations. The major advantage of the proposed procedures over the existing ones is easy to implement using existing software, and it requires no specification of the error distributions. Under the regularity conditions, the consistency and asymptotic normality of the estimators are also derived. Finally, a simulation study and a real data application are undertaken to assess the finite sample performance of the proposed estimation procedure.     

Extreme Quantile Estimation for Autoregressive Models

ABSTRACT

A quantile autoregresive model is a useful extension of classical autoregresive models as it can capture the influences of conditioning variables on the location, scale, and shape of the response distribution. However, at the extreme tails, standard quantile autoregression estimator is often unstable due to data sparsity. In this article, assuming quantile autoregresive models, we develop a new estimator for extreme conditional quantiles of time series data based on extreme value theory. We build the connection between the second-order conditions for the autoregression coefficients and for the conditional quantile functions, and establish the asymptotic properties of the proposed estimator. The finite sample performance of the proposed method is illustrated through a simulation study and the analysis of U.S. retail gasoline price.     

Quantile regression in longitudinal studies with dropouts and measurement errors

ABSTRACT

Quantile regression models, as an important tool in practice, can describe effects of risk factors on the entire conditional distribution of the response variable with its estimates robust to outliers. However, there is few discussion on quantile regression for longitudinal data with both missing responses and measurement errors, which are commonly seen in practice. We develop a weighted and bias-corrected quantile loss function for the quantile regression with longitudinal data, which allows both missingness and measurement errors. Additionally, we establish the asymptotic properties of the proposed estimator. Simulation studies demonstrate the expected performance in correcting the bias resulted from missingness and measurement errors. Finally, we investigate the Lifestyle Education for Activity and Nutrition study and confirm the effective of intervention in producing weight loss after nine month at the high quantile.     

General composite quantile regression: Theory and methods

Abstract

In this article, we propose a new regression method called general composite quantile regression (GCQR) which releases the unrealistic finite error variance assumption being imposed by the traditional least squares (LS) method. Unlike the recently proposed composite quantile regression (CQR) method, our proposed GCQR allows any continuous non-uniform density/weight function. As a result, determination of the number of uniform quantile positions is not required. Most importantly, the proposed GCQR criterion can be readily transformed to a linear programing problem, which substantially reduces the computing time. Our theoretical and empirical results show that the GCQR is generally efficient than the CQR and LS if the weight function is appropriately chosen. The oracle properties of the penalized GCQR are also provided. Our simulation results are consistent with the derived theoretical findings. A real data example is analyzed to demonstrate our methodologies.     

Bayesian adaptive Lasso for quantile regression models with nonignorably missing response data

Abstract

Handling data with the nonignorably missing mechanism is still a challenging problem in statistics. In this paper, we develop a fully Bayesian adaptive Lasso approach for quantile regression models with nonignorably missing response data, where the nonignorable missingness mechanism is specified by a logistic regression model. The proposed method extends the Bayesian Lasso by allowing different penalization parameters for different regression coefficients. Furthermore, a hybrid algorithm that combined the Gibbs sampler and Metropolis-Hastings algorithm is implemented to simulate the parameters from posterior distributions, mainly including regression coefficients, shrinkage coefficients, parameters in the non-ignorable missing models. Finally, some simulation studies and a real example are used to illustrate the proposed methodology.     

Efficient Robust Estimation for Linear Models with Missing Response at Random

Coefficient estimation in linear regression models with missing data is routinely carried out in the mean regression framework. However, the mean regression theory breaks down if the error variance is infinite. In addition, correct specification of the likelihood function for existing imputation approach is often challenging in practice, especially for skewed data. In this paper, we develop a novel composite quantile regression and a weighted quantile average estimation procedure for parameter estimation in linear regression models when some responses are missing at random. Instead of imputing the missing response by randomly drawing from its conditional distribution, we propose to impute both missing and observed responses by their estimated conditional quantiles given the observed data and to use the parametrically estimated propensity scores to weigh check functions that define a regression parameter. Both estimation procedures are resistant to heavy‐tailed errors or outliers in the response and can achieve nice robustness and efficiency. Moreover, we propose adaptive penalization methods to simultaneously select significant variables and estimate unknown parameters. Asymptotic properties of the proposed estimators are carefully investigated. An efficient algorithm is developed for fast implementation of the proposed methodologies. We also discuss a model selection criterion, which is based on an IC_Q ‐type statistic, to select the penalty parameters. The performance of the proposed methods is illustrated via simulated and real data sets.     

Bayesian Tobit quantile regression using g-prior distribution with ridge parameter

A Bayesian approach is proposed for coefficient estimation in the Tobit quantile regression model. The proposed approach is based on placing a g-prior distribution depends on the quantile level on the regression coefficients. The prior is generalized by introducing a ridge parameter to address important challenges that may arise with censored data, such as multicollinearity and overfitting problems. Then, a stochastic search variable selection approach is proposed for Tobit quantile regression model based on g-prior. An expression for the hyperparameter g is proposed to calibrate the modified g-prior with a ridge parameter to the corresponding g-prior. Some possible extensions of the proposed approach are discussed, including the continuous and binary responses in quantile regression. The methods are illustrated using several simulation studies and a microarray study. The simulation studies and the microarray study indicate that the proposed approach performs well.     

Quantile regression methods for censored gap time data

ABSTRACT

In longitudinal studies, subjects may potentially undergo a series of sequentially ordered events. The gap times, which are the times between two serial events, are often the outcome variables of interest. This study considers quantile regression models of gap times for censored serial-event data and adapts a weighted version of the estimating equation for regression coefficients. The resulting estimators are uniformly consistent and asymptotically normal. Extensive simulation studies are presented to evaluate the finite-sample performance of the proposed methods. An analysis of the tumor recurrence data for bladder cancer patients is also provided to illustrate our proposed methods.     

Linear quantile mixed models

Dependent data arise in many studies. Frequently adopted sampling designs, such as cluster, multilevel, spatial, and repeated measures, may induce this dependence, which the analysis of the data needs to take into due account. In a previous publication (Geraci and Bottai in Biostatistics 8:140–154, 2007), we proposed a conditional quantile regression model for continuous responses where subject-specific random intercepts were included to account for within-subject dependence in the context of longitudinal data analysis. The approach hinged upon the link existing between the minimization of weighted absolute deviations, typically used in quantile regression, and the maximization of a Laplace likelihood. Here, we consider an extension of those models to more complex dependence structures in the data, which are modeled by including multiple random effects in the linear conditional quantile functions. We also discuss estimation strategies to reduce the computational burden and inefficiency associated with the Monte Carlo EM algorithm we have proposed previously. In particular, the estimation of the fixed regression coefficients and of the random effects’ covariance matrix is based on a combination of Gaussian quadrature approximations and non-smooth optimization algorithms. Finally, a simulation study and a number of applications of our models are presented.     

Adaptive penalized quantile regression for high dimensional data

We propose a new adaptive L₁ penalized quantile regression estimator for high-dimensional sparse regression models with heterogeneous error sequences. We show that under weaker conditions compared with alternative procedures, the adaptive L₁ quantile regression selects the true underlying model with probability converging to one, and the unique estimates of nonzero coefficients it provides have the same asymptotic normal distribution as the quantile estimator which uses only the covariates with non-zero impact on the response. Thus, the adaptive L₁ quantile regression enjoys oracle properties. We propose a completely data driven choice of the penalty level _λn${\lambda }_n$, which ensures good performance of the adaptive L₁ quantile regression. Extensive Monte Carlo simulation studies have been conducted to demonstrate the finite sample performance of the proposed method.     

Power-transformed linear regression on quantile residual life for censored competing risks data

ABSTRACT

This paper proposes a power-transformed linear quantile regression model for the residual lifetime of competing risks data. The proposed model can describe the association between any quantile of a time-to-event distribution among survivors beyond a specific time point and the covariates. Under covariate-dependent censoring, we develop an estimation procedure with two steps, including an unbiased monotone estimating equation for regression parameters and cumulative sum processes for the Box–Cox transformation parameter. The asymptotic properties of the estimators are also derived. We employ an efficient bootstrap method for the estimation of the variance–covariance matrix. The finite-sample performance of the proposed approaches are evaluated through simulation studies and a real example.     

Sparsistent and constansistent estimation of the varying-coefficient model with a diverging number of predictors

ABSTRACT

The varying-coefficient model is a flexible class of approaches that extends simple linear relationships between covariates and responses. Two related problems concerning these models are selecting relevant variables and determining non-varying coefficients among those relevant ones. In this paper we study the sparsistency and constansistency of the regularized estimation approach when the number of predictors diverges with the sample size. Here, constansistency refers to the desired property that the non-zero, non-varying coefficients are identified with probability tending to one.     

Estimation and variable selection for partially functional linear models

In this paper, a new estimation procedure based on composite quantile regression and functional principal component analysis (PCA) method is proposed for the partially functional linear regression models (PFLRMs). The proposed estimation method can simultaneously estimate both the parametric regression coefficients and functional coefficient components without specification of the error distributions. The proposed estimation method is shown to be more efficient empirically for non-normal random error, especially for Cauchy error, and almost as efficient for normal random errors. Furthermore, based on the proposed estimation procedure, we use the penalized composite quantile regression method to study variable selection for parametric part in the PFLRMs. Under certain regularity conditions, consistency, asymptotic normality, and Oracle property of the resulting estimators are derived. Simulation studies and a real data analysis are conducted to assess the finite sample performance of the proposed methods.     

A weighted quantile regression for left-truncated and right-censored data

Quantile regression methods have been used to estimate upper and lower quantile reference curves as the function of several covariates. In this article, it is demonstrated that the estimating equation of Zhou [A weighted quantile regression for randomly truncated data, Comput. Stat. Data Anal. 55 (2011), pp. 554–566.] can be extended to analyse left-truncated and right-censored data. We evaluate the finite sample performance of the proposed estimators through simulation studies. The proposed estimator β?(q) is applied to the Veteran's Administration lung cancer data reported by Prentice [Exponential survival with censoring and explanatory variables, Biometrika 60 (1973), pp. 279–288].     

Nonparametric Estimation of the Conditional Distribution at Regression Boundary Points

Abstract

Nonparametric regression is a standard statistical tool with increased importance in the Big Data era. Boundary points pose additional difficulties but local polynomial regression can be used to alleviate them. Local linear regression, for example, is easy to implement and performs quite well both at interior and boundary points. Estimating the conditional distribution function and/or the quantile function at a given regressor point is immediate via standard kernel methods but problems ensue if local linear methods are to be used. In particular, the distribution function estimator is not guaranteed to be monotone increasing, and the quantile curves can “cross.” In the article at hand, a simple method of correcting the local linear distribution estimator for monotonicity is proposed, and its good performance is demonstrated via simulations and real data examples. Supplementary materials for this article are available online.     

Quantile regression for panel data models with fixed effects under random censoring

Abstract

The locally weighted censored quantile regression approach is proposed for panel data models with fixed effects, which allows for random censoring. The resulting estimators are obtained by employing the fixed effects quantile regression method. The weights are selected either parametrically, semi-parametrically or non-parametrically. The large panel data asymptotics are used in an attempt to cope with the incidental parameter problem. The consistency and limiting distribution of the proposed estimator are also derived. The finite sample performance of the proposed estimators are examined via Monte Carlo simulations.     

Single-index composite quantile regression

In this paper, we extend the composite quantile regression (CQR) method to a single-index model. The unknown link function is estimated by local composite quantile regression and the parametric index is estimated through the linear composite quantile. It is shown that the proposed estimators are consistent and asymptotically normal. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

Sparsity identification in ultra-high dimensional quantile regression models with longitudinal data

Abstract

In this paper, we propose a variable selection method for quantile regression model in ultra-high dimensional longitudinal data called as the weighted adaptive robust lasso (WAR-Lasso) which is double-robustness. We derive the consistency and the model selection oracle property of WAR-Lasso. Simulation studies show the double-robustness of WAR-Lasso in both cases of heavy-tailed distribution of the errors and the heavy contaminations of the covariates. WAR-Lasso outperform other methods such as SCAD and etc. A real data analysis is carried out. It shows that WAR-Lasso tends to select fewer variables and the estimated coefficients are in line with economic significance.     

Local linear double and asymmetric kernel estimation of conditional quantiles

Abstract

In this work, we propose and investigate a family of non parametric quantile regression estimates. The proposed estimates combine local linear fitting and double kernel approaches. More precisely, we use a Beta kernel when covariate’s support is compact and Gamma kernel for left-bounded supports. Finite sample properties together with asymptotic behavior of the proposed estimators are presented. It is also shown that these estimates enjoy the property of having finite variance and resistance to sparse design.