Bayesian variable selection and estimation in maximum entropy quantile regression

Quantile regression has gained increasing popularity as it provides richer information than the regular mean regression, and variable selection plays an important role in the quantile regression model building process, as it improves the prediction accuracy by choosing an appropriate subset of regression predictors. Unlike the traditional quantile regression, we consider the quantile as an unknown parameter and estimate it jointly with other regression coefficients. In particular, we adopt the Bayesian adaptive Lasso for the maximum entropy quantile regression. A flat prior is chosen for the quantile parameter due to the lack of information on it. The proposed method not only addresses the problem about which quantile would be the most probable one among all the candidates, but also reflects the inner relationship of the data through the estimated quantile. We develop an efficient Gibbs sampler algorithm and show that the performance of our proposed method is superior than the Bayesian adaptive Lasso and Bayesian Lasso through simulation studies and a real data analysis.     

Bayesian analysis of dynamic panel data by penalized quantile regression

Existing literature on quantile regression for panel data models with individual effects advocates the application of penalization to reduce the dynamic panel bias and increase the efficiency of the estimators. In this paper, we consider penalized quantile regression for dynamic panel data with random effects from a Bayesian perspective, where the penalty involves an adaptive Lasso shrinkage of the random effects. We also address the role of initial conditions in dynamic panel data models, emphasizing joint modeling of start-up and subsequent responses. For posterior inference, an efficient Gibbs sampler is developed to simulate the parameters from the posterior distributions. Through simulation studies and analysis of a real data set, we assess the performance of the proposed Bayesian method.     

Bayesian adaptive Lasso for quantile regression models with nonignorably missing response data

Abstract

Handling data with the nonignorably missing mechanism is still a challenging problem in statistics. In this paper, we develop a fully Bayesian adaptive Lasso approach for quantile regression models with nonignorably missing response data, where the nonignorable missingness mechanism is specified by a logistic regression model. The proposed method extends the Bayesian Lasso by allowing different penalization parameters for different regression coefficients. Furthermore, a hybrid algorithm that combined the Gibbs sampler and Metropolis-Hastings algorithm is implemented to simulate the parameters from posterior distributions, mainly including regression coefficients, shrinkage coefficients, parameters in the non-ignorable missing models. Finally, some simulation studies and a real example are used to illustrate the proposed methodology.     

基于分位回归的风险保费预测

风险保费预测是非寿险费率厘定的重要组成部分。在传统的分位回归厘定风险保费中,通常假设分位数水平是事先给定的,缺乏一定的客观性。为此,提出了一种应用分位回归厘定风险保费的新方法。基于破产概率确定保单组合的总风险保费,建立个体保单的分位回归模型,并与总风险保费建立等式关系,通过数值方法求解出分位数水平,实现对个体保单风险保费的预测。通过一组实际数据分析表明,该方法具有良好的预测效果。     

Quantile regression for large-scale data via sparse exponential transform method

In recent decades, quantile regression has received much more attention from academics and practitioners. However, most of existing computational algorithms are only effective for small or moderate size problems. They cannot solve quantile regression with large-scale data reliably and efficiently. To this end, we propose a new algorithm to implement quantile regression on large-scale data using the sparse exponential transform (SET) method. This algorithm mainly constructs a well-conditioned basis and a sampling matrix to reduce the number of observations. It then solves a quantile regression problem on this reduced matrix and obtains an approximate solution. Through simulation studies and empirical analysis of a 5% sample of the US 2000 Census data, we demonstrate efficiency of the SET-based algorithm. Numerical results indicate that our new algorithm is effective in terms of computation time and performs well for large-scale quantile regression.     

Bayesian Semiparametric Modelling in Quantile Regression

Abstract. We propose a Bayesian semiparametric methodology for quantile regression modelling. In particular, working with parametric quantile regression functions, we develop Dirichlet process mixture models for the error distribution in an additive quantile regression formulation. The proposed non‐parametric prior probability models allow the shape of the error density to adapt to the data and thus provide more reliable predictive inference than models based on parametric error distributions. We consider extensions to quantile regression for data sets that include censored observations. Moreover, we employ dependent Dirichlet processes to develop quantile regression models that allow the error distribution to change non‐parametrically with the covariates. Posterior inference is implemented using Markov chain Monte Carlo methods. We assess and compare the performance of our models using both simulated and real data sets.     

Regularized Bayesian quantile regression

A number of nonstationary models have been developed to estimate extreme events as function of covariates. A quantile regression (QR) model is a statistical approach intended to estimate and conduct inference about the conditional quantile functions. In this article, we focus on the simultaneous variable selection and parameter estimation through penalized quantile regression. We conducted a comparison of regularized Quantile Regression model with B-Splines in Bayesian framework. Regularization is based on penalty and aims to favor parsimonious model, especially in the case of large dimension space. The prior distributions related to the penalties are detailed. Five penalties (Lasso, Ridge, SCAD0, SCAD1 and SCAD2) are considered with their equivalent expressions in Bayesian framework. The regularized quantile estimates are then compared to the maximum likelihood estimates with respect to the sample size. A Markov Chain Monte Carlo (MCMC) algorithms are developed for each hierarchical model to simulate the conditional posterior distribution of the quantiles. Results indicate that the SCAD0 and Lasso have the best performance for quantile estimation according to Relative Mean Biais (RMB) and the Relative Mean-Error (RME) criteria, especially in the case of heavy distributed errors. A case study of the annual maximum precipitation at Charlo, Eastern Canada, with the Pacific North Atlantic climate index as covariate is presented.     

Overlapping group lasso for high-dimensional generalized linear models

Abstract

Structured sparsity has recently been a very popular technique to deal with the high-dimensional data. In this paper, we mainly focus on the theoretical problems for the overlapping group structure of generalized linear models (GLMs). Although the overlapping group lasso method for GLMs has been widely applied in some applications, the theoretical properties about it are still unknown. Under some general conditions, we presents the oracle inequalities for the estimation and prediction error of overlapping group Lasso method in the generalized linear model setting. Then, we apply these results to the so-called Logistic and Poisson regression models. It is shown that the results of the Lasso and group Lasso procedures for GLMs can be recovered by specifying the group structures in our proposed method. The effect of overlap and the performance of variable selection of our proposed method are both studied by numerical simulations. Finally, we apply our proposed method to two gene expression data sets: the p53 data and the lung cancer data.     

Bayesian bridge quantile regression

Regularization methods for simultaneous variable selection and coefficient estimation have been shown to be effective in quantile regression in improving the prediction accuracy. In this article, we propose the Bayesian bridge for variable selection and coefficient estimation in quantile regression. A simple and efficient Gibbs sampling algorithm was developed for posterior inference using a scale mixture of uniform representation of the Bayesian bridge prior. This is the first work to discuss regularized quantile regression with the bridge penalty. Both simulated and real data examples show that the proposed method often outperforms quantile regression without regularization, lasso quantile regression, and Bayesian lasso quantile regression.     

Influence Measures in Quantile Regression Models

In this article, we use the asymmetric Laplace distribution to define a new method to determine the influence of a certain observation in the fit of quantile regression models. Our measure is based on the likelihood displacement function and we propose two types of measures in order to determine influential observations in a set of conditional quantiles conjointly or in each conditional quantile of interest. We verify the validity of our average measure in a simulated data set as well in an illustrative example with data about air pollution.     

面板数据的贝叶斯Elastic Net分位数回归方法及其应用研究

本文首次将Elastic Net这种用于高度相关变量的惩罚方法用于面板数据的贝叶斯分位数回归,并基于非对称Laplace先验分布推导所有参数的后验分布,进而构建Gibbs抽样。为了验证模型的有效性,本文将面板数据的贝叶斯Elastic Net分位数回归方法(BQR. EN)与面板数据的贝叶斯分位数回归方法(BQR)、面板数据的贝叶斯Lasso分位数回归方法(BLQR)、面板数据的贝叶斯自适应Lasso分位数回归方法(BALQR)进行了多种情形下的全方位比较,结果表明BQR. EN方法适用于具有高度相关性、数据维度很高和尖峰厚尾分布特征的数据。进一步地,本文就BQR. EN方法在不同扰动项假设、不同样本量的情形展开模拟比较,验证了新方法的稳健性和小样本特性。最后,本文选取互联网金融类上市公司经济增加值(EVA)作为实证研究对象,检验新方法在实际问题中的参数估计与变量选择能力,实证结果符合预期。     

A weighted linear quantile regression

In this article, we introduce a new weighted quantile regression method. Traditionally, the estimation of the parameters involved in quantile regression is obtained by minimizing a loss function based on absolute distances with weights independent of explanatory variables. Specifically, we study a new estimation method using a weighted loss function with the weights associated with explanatory variables so that the performance of the resulting estimation can be improved. In full generality, we derive the asymptotic distribution of the weighted quantile regression estimators for any uniformly bounded positive weight function independent of the response. Two practical weighting schemes are proposed, each for a certain type of data. Monte Carlo simulations are carried out for comparing our proposed methods with the classical approaches. We also demonstrate the proposed methods using two real-life data sets from the literature. Both our simulation study and the results from these examples show that our proposed method outperforms the classical approaches when the relative efficiency is measured by the mean-squared errors of the estimators.     

Bayesian non-crossing quantile regression for regularly varying distributions

Quantile regression is a very important statistical tool for predictive modelling and risk assessment. For many applications, conditional quantile at different levels are estimated separately. Consequently the monotonicity of conditional quantiles can be violated when quantile regression curves cross each other. In this paper, we propose a new Bayesian multiple quantile regression based on heavy tailed distribution for non-crossing. We consider a linear quantile regression model for simultaneous Bayesian estimation of multiple quantiles based on a regularly varying assumptions. The numerical and competitive performance of the proposed method is illustrated by simulation.     

Interquantile shrinkage in additive models

In this paper, we investigate the commonality of nonparametric component functions among different quantile levels in additive regression models. We propose two fused adaptive group Least Absolute Shrinkage and Selection Operator penalties to shrink the difference of functions between neighbouring quantile levels. The proposed methodology is able to simultaneously estimate the nonparametric functions and identify the quantile regions where functions are unvarying, and thus is expected to perform better than standard additive quantile regression when there exists a region of quantile levels on which the functions are unvarying. Under some regularity conditions, the proposed penalised estimators can theoretically achieve the optimal rate of convergence and identify the true varying/unvarying regions consistently. Simulation studies and a real data application show that the proposed methods yield good numerical results.     

Wavelet-based LASSO in functional linear quantile regression

ABSTRACT

In this paper, we develop an efficient wavelet-based regularized linear quantile regression framework for coefficient estimations, where the responses are scalars and the predictors include both scalars and function. The framework consists of two important parts: wavelet transformation and regularized linear quantile regression. Wavelet transform can be used to approximate functional data through representing it by finite wavelet coefficients and effectively capturing its local features. Quantile regression is robust for response outliers and heavy-tailed errors. In addition, comparing with other methods it provides a more complete picture of how responses change conditional on covariates. Meanwhile, regularization can remove small wavelet coefficients to achieve sparsity and efficiency. A novel algorithm, Alternating Direction Method of Multipliers (ADMM) is derived to solve the optimization problems. We conduct numerical studies to investigate the finite sample performance of our method and applied it on real data from ADHD studies.     

Bayesian composite quantile regression for linear mixed-effects models

Longitudinal data are commonly modeled with the normal mixed-effects models. Most modeling methods are based on traditional mean regression, which results in non robust estimation when suffering extreme values or outliers. Median regression is also not a best choice to estimation especially for non normal errors. Compared to conventional modeling methods, composite quantile regression can provide robust estimation results even for non normal errors. In this paper, based on a so-called pseudo composite asymmetric Laplace distribution (PCALD), we develop a Bayesian treatment to composite quantile regression for mixed-effects models. Furthermore, with the location-scale mixture representation of the PCALD, we establish a Bayesian hierarchical model and achieve the posterior inference of all unknown parameters and latent variables using Markov Chain Monte Carlo (MCMC) method. Finally, this newly developed procedure is illustrated by some Monte Carlo simulations and a case analysis of HIV/AIDS clinical data set.     

Randomized quantile regression estimation for heteroskedastic non parametric model

In this paper, we propose robust randomized quantile regression estimators for the mean and (condition) variance functions of the popular heteroskedastic non parametric regression model. Unlike classical approaches which consider quantile as a fixed quantity, our method treats quantile as a uniformly distributed random variable. Our proposed method can be employed to estimate the error distribution, which could significantly improve prediction results. An automatic bandwidth selection scheme will be discussed. Asymptotic properties and relative efficiencies of the proposed estimators are investigated. Our empirical results show that the proposed estimators work well even for random errors with infinite variances. Various numerical simulations and two real data examples are used to demonstrate our methodologies.     

Weighted composite quantile regression for partially linear varying coefficient models

Partially linear varying coefficient models (PLVCMs) with heteroscedasticity are considered in this article. Based on composite quantile regression, we develop a weighted composite quantile regression (WCQR) to estimate the non parametric varying coefficient functions and the parametric regression coefficients. The WCQR is augmented using a data-driven weighting scheme. Moreover, the asymptotic normality of proposed estimators for both the parametric and non parametric parts are studied explicitly. In addition, by comparing the asymptotic relative efficiency theoretically and numerically, WCQR method all outperforms the CQR method and some other estimate methods. To achieve sparsity with high-dimensional covariates, we develop a variable selection procedure to select significant parametric components for the PLVCM and prove the method possessing the oracle property. Both simulations and data analysis are conducted to illustrate the finite-sample performance of the proposed methods.     

Bayesian quantile regression for hierarchical linear models

The paper proposes a Bayesian quantile regression method for hierarchical linear models. Existing approaches of hierarchical linear quantile regression models are scarce and most of them were not from the perspective of Bayesian thoughts, which is important for hierarchical models. In this paper, based on Bayesian theories and Markov Chain Monte Carlo methods, we introduce Asymmetric Laplace distributed errors to simulate joint posterior distributions of population parameters and across-unit parameters and then derive their posterior quantile inferences. We run a simulation as the proposed method to examine the effects on parameters induced by units and quantile levels; the method is also applied to study the relationship between Chinese rural residents' family annual income and their cultivated areas. Both the simulation and real data analysis indicate that the method is effective and accurate.     

Single-index composite quantile regression

In this paper, we extend the composite quantile regression (CQR) method to a single-index model. The unknown link function is estimated by local composite quantile regression and the parametric index is estimated through the linear composite quantile. It is shown that the proposed estimators are consistent and asymptotically normal. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.