Local quantile regression

Quantile regression is a technique to estimate conditional quantile curves. It provides a comprehensive picture of a response contingent on explanatory variables. In a flexible modeling framework, a specific form of the conditional quantile curve is not a priori fixed. This motivates a local parametric rather than a global fixed model fitting approach. A nonparametric smoothing estimator of the conditional quantile curve requires to balance between local curvature and stochastic variability. In this paper, we suggest a local model selection technique that provides an adaptive estimator of the conditional quantile regression curve at each design point. Theoretical results claim that the proposed adaptive procedure performs as good as an oracle which would minimize the local estimation risk for the problem at hand. We illustrate the performance of the procedure by an extensive simulation study and consider a couple of applications: to tail dependence analysis for the Hong Kong stock market and to analysis of the distributions of the risk factors of temperature dynamics.     

Regularized Bayesian quantile regression

A number of nonstationary models have been developed to estimate extreme events as function of covariates. A quantile regression (QR) model is a statistical approach intended to estimate and conduct inference about the conditional quantile functions. In this article, we focus on the simultaneous variable selection and parameter estimation through penalized quantile regression. We conducted a comparison of regularized Quantile Regression model with B-Splines in Bayesian framework. Regularization is based on penalty and aims to favor parsimonious model, especially in the case of large dimension space. The prior distributions related to the penalties are detailed. Five penalties (Lasso, Ridge, SCAD0, SCAD1 and SCAD2) are considered with their equivalent expressions in Bayesian framework. The regularized quantile estimates are then compared to the maximum likelihood estimates with respect to the sample size. A Markov Chain Monte Carlo (MCMC) algorithms are developed for each hierarchical model to simulate the conditional posterior distribution of the quantiles. Results indicate that the SCAD0 and Lasso have the best performance for quantile estimation according to Relative Mean Biais (RMB) and the Relative Mean-Error (RME) criteria, especially in the case of heavy distributed errors. A case study of the annual maximum precipitation at Charlo, Eastern Canada, with the Pacific North Atlantic climate index as covariate is presented.     

Multi-step quantile regression tree

Quantile regression (QR) proposed by Koenker and Bassett [Regression quantiles, Econometrica 46(1) (1978), pp. 33–50] is a statistical technique that estimates conditional quantiles. It has been widely studied and applied to economics. Meinshausen [Quantile regression forests, J. Mach. Learn. Res. 7 (2006), pp. 983–999] proposed quantile regression forests (QRF), a non-parametric way based on random forest. QRF performs well in terms of prediction accuracy, but it struggles with noisy data sets. This motivates us to propose a multi-step QR tree method using GUIDE (Generalized, Unbiased, Interaction Detection and Estimation) made by Loh [Regression trees with unbiased variable selection and interaction detection, Statist. Sinica 12 (2002), pp. 361–386]. Our simulation study shows that the multi-step QR tree performs better than a single tree or QRF especially when it deals with data sets having many irrelevant variables.     

Nonstationary nonlinear quantile regression

This study examines estimation and inference based on quantile regression for parametric nonlinear models with an integrated time series covariate. We first derive the limiting distribution of the nonlinear quantile regression estimator and then consider testing for parameter restrictions, when the regression function is specified as an asymptotically homogeneous function. We also study linear-in-parameter regression models when the regression function is given by integrable regression functions as well as asymptotically homogeneous regression functions. We, furthermore, propose a fully modified estimator to reduce the bias in the original estimator under a certain set of conditions. Finally, simulation studies show that the estimators behave well, especially when the regression error term has a fat-tailed distribution.     

Single-index composite quantile regression

In this paper, we extend the composite quantile regression (CQR) method to a single-index model. The unknown link function is estimated by local composite quantile regression and the parametric index is estimated through the linear composite quantile. It is shown that the proposed estimators are consistent and asymptotically normal. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

Bayesian Lasso-mixed quantile regression

In this paper, we discuss the regularization in linear-mixed quantile regression. A hierarchical Bayesian model is used to shrink the fixed and random effects towards the common population values by introducing an l₁ penalty in the mixed quantile regression check function. A Gibbs sampler is developed to simulate the parameters from the posterior distributions. Through simulation studies and analysis of an age-related macular degeneration (ARMD) data, we assess the performance of the proposed method. The simulation studies and the ARMD data analysis indicate that the proposed method performs well in comparison with the other approaches.     

Structured kernel quantile regression

Quantile regression can provide more useful information on the conditional distribution of a response variable given covariates while classical regression provides informations on the conditional mean alone. In this paper, we propose a structured quantile estimation methodology in a nonparametric function estimation setup. Through the functional analysis of variance decomposition, the optimization of the proposed method can be solved using a series of quadratic and linear programmings. Our method automatically selects relevant covariates by adopting a lasso-type penalty. The performance of the proposed methodology is illustrated through numerical examples on both simulated and real data.     

Composite kernel quantile regression

The composite quantile regression (CQR) has been developed for the robust and efficient estimation of regression coefficients in a liner regression model. By employing the idea of the CQR, we propose a new regression method, called composite kernel quantile regression (CKQR), which uses the sum of multiple check functions as a loss in reproducing kernel Hilbert spaces for the robust estimation of a nonlinear regression function. The numerical results demonstrate the usefulness of the proposed CKQR in estimating both conditional nonlinear mean and quantile functions.     

Bayesian composite quantile regression

One advantage of quantile regression, relative to the ordinary least-square (OLS) regression, is that the quantile regression estimates are more robust against outliers and non-normal errors in the response measurements. However, the relative efficiency of the quantile regression estimator with respect to the OLS estimator can be arbitrarily small. To overcome this problem, composite quantile regression methods have been proposed in the literature which are resistant to heavy-tailed errors or outliers in the response and at the same time are more efficient than the traditional single quantile-based quantile regression method. This paper studies the composite quantile regression from a Bayesian perspective. The advantage of the Bayesian hierarchical framework is that the weight of each component in the composite model can be treated as open parameter and automatically estimated through Markov chain Monte Carlo sampling procedure. Moreover, the lasso regularization can be naturally incorporated into the model to perform variable selection. The performance of the proposed method over the single quantile-based method was demonstrated via extensive simulations and real data analysis.     

On multivariate quantile regression

To detect the dependence on the covariates in the lower and upper tails of the response distribution, regression quantiles are very useful tools in linear model problems with univariate response. We consider here a notion of regression quantiles for problems with multivariate responses. The approach is based on minimizing a loss function equivalent to that in the case of univariate response. To construct an affine equivariant notion of multivariate regression quantiles, we have considered a transformation retransformation procedure based on ‘data-driven coordinate systems’. We indicate some algorithm to compute the proposed estimates and establish asymptotic normality for them. We also, suggest an adaptive procedure to select the optimal data-driven coordinate system. We discuss the performance of our estimates with the help of a finite sample simulation study and to illustrate our methodology, we analyzed an interesting data-set on blood pressures of a group of women and another one on the dependence of sales performances on creative test scores.     

Hierarchically penalized quantile regression

In many regression problems, predictors are naturally grouped. For example, when a set of dummy variables is used to represent categorical variables, or a set of basis functions of continuous variables is included in the predictor set, it is important to carry out a feature selection both at the group level and at individual variable levels within the group simultaneously. To incorporate the group and variables within-group information into a regularized model fitting, several regularization methods have been developed, including the Cox regression and the conditional mean regression. Complementary to earlier works, the simultaneous group and within-group variables selection method is examined in quantile regression. We propose a hierarchically penalized quantile regression, and show that the hierarchical penalty possesses the oracle property in quantile regression, as well as in the Cox regression. The proposed method is evaluated through simulation studies and a real data application.     

Bayesian bridge quantile regression

Regularization methods for simultaneous variable selection and coefficient estimation have been shown to be effective in quantile regression in improving the prediction accuracy. In this article, we propose the Bayesian bridge for variable selection and coefficient estimation in quantile regression. A simple and efficient Gibbs sampling algorithm was developed for posterior inference using a scale mixture of uniform representation of the Bayesian bridge prior. This is the first work to discuss regularized quantile regression with the bridge penalty. Both simulated and real data examples show that the proposed method often outperforms quantile regression without regularization, lasso quantile regression, and Bayesian lasso quantile regression.     

Partially linear censored quantile regression

Censored regression quantile (CRQ) methods provide a powerful and flexible approach to the analysis of censored survival data when standard linear models are felt to be appropriate. In many cases however, greater flexibility is desired to go beyond the usual multiple regression paradigm. One area of common interest is that of partially linear models: one (or more) of the explanatory covariates are assumed to act on the response through a non-linear function. Here the CRQ approach of Portnoy (J Am Stat Assoc 98:1001–1012, 2003) is extended to this partially linear setting. Basic consistency results are presented. A simulation experiment and unemployment example justify the value of the partially linear approach over methods based on the Cox proportional hazards model and on methods not permitting nonlinearity.     

Monotone support vector quantile regression

Quantile regression (QR) models have received a great deal of attention in both the theoretical and applied statistical literature. In this paper we propose support vector quantile regression (SVQR) with monotonicity restriction, which is easily obtained via the dual formulation of the optimization problem. We also provide the generalized approximate cross validation method for choosing the hyperparameters which affect the performance of the proposed SVQR. The experimental results for the synthetic and real data sets confirm the successful performance of the proposed model.     

Functional single-index quantile regression models

It is known that functional single-index regression models can achieve better prediction accuracy than functional linear models or fully nonparametric models, when the target is to predict a scalar response using a function-valued covariate. However, the performance of these models may be adversely affected by extremely large values or skewness in the response. In addition, they are not able to offer a full picture of the conditional distribution of the response. Motivated by using trajectories of $$hbox {PM}_{{10}}$$ concentrations of last day to predict the maximum $$hbox {PM}_{{10}}$$ concentration of the current day, a functional single-index quantile regression model is proposed to address those issues. A generalized profiling method is employed to estimate the model. Simulation studies are conducted to investigate the finite sample performance of the proposed estimator. We apply the proposed framework to predict the maximal value of $$hbox {PM}_{{10}}$$ concentrations based on the intraday $$hbox {PM}_{{10}}$$ concentrations of the previous day.     

Bayesian composite Tobit quantile regression

Composite quantile regression models have been shown to be effective techniques in improving the prediction accuracy [H. Zou and M. Yuan, Composite quantile regression and the oracle model selection theory, Ann. Statist. 36 (2008), pp. 1108–1126; J. Bradic, J. Fan, and W. Wang, Penalized composite quasi-likelihood for ultrahighdimensional variable selection, J. R. Stat. Soc. Ser. B 73 (2011), pp. 325–349; Z. Zhao and Z. Xiao, Efficient regressions via optimally combining quantile information, Econometric Theory 30(06) (2014), pp. 1272–1314]. This paper studies composite Tobit quantile regression (TQReg) from a Bayesian perspective. A simple and efficient MCMC-based computation method is derived for posterior inference using a mixture of an exponential and a scaled normal distribution of the skewed Laplace distribution. The approach is illustrated via simulation studies and a real data set. Results show that combine information across different quantiles can provide a useful method in efficient statistical estimation. This is the first work to discuss composite TQReg from a Bayesian perspective.     

On multivariate quantile regression analysis

This paper investigates the estimation of parameters in a multivariate quantile regression model when the investigator wants to evaluate the associated distribution function. It proposes a new directional quantile estimator with the following properties: (1) it applies to an arbitrary number of random variables; (2) it is equivalent to estimating the distribution function allowing for non-convex distribution contours; (3) it satisfies nice equivariance properties; (4) it has desirable statistical properties (i.e., consistency and asymptotic normality); and (5) its implementation involves a modest computational burden: our proposed estimator can be obtained by solving parametric linear programming problems. As such, this paper expands the range of applications of quantile estimation for multivariate regression models.     

General composite quantile regression: Theory and methods

Abstract

In this article, we propose a new regression method called general composite quantile regression (GCQR) which releases the unrealistic finite error variance assumption being imposed by the traditional least squares (LS) method. Unlike the recently proposed composite quantile regression (CQR) method, our proposed GCQR allows any continuous non-uniform density/weight function. As a result, determination of the number of uniform quantile positions is not required. Most importantly, the proposed GCQR criterion can be readily transformed to a linear programing problem, which substantially reduces the computing time. Our theoretical and empirical results show that the GCQR is generally efficient than the CQR and LS if the weight function is appropriately chosen. The oracle properties of the penalized GCQR are also provided. Our simulation results are consistent with the derived theoretical findings. A real data example is analyzed to demonstrate our methodologies.     

A generalized quantile regression model

A new class of probability distributions, the so-called connected double truncated gamma distribution, is introduced. We show that using this class as the error distribution of a linear model leads to a generalized quantile regression model that combines desirable properties of both least-squares and quantile regression methods: robustness to outliers and differentiable loss function.