Reversible jump Markov chain Monte Carlo algorithms for Bayesian variable selection in logistic mixed models

In this article, to reduce computational load in performing Bayesian variable selection, we used a variant of reversible jump Markov chain Monte Carlo methods, and the Holmes and Held (HH) algorithm, to sample model index variables in logistic mixed models involving a large number of explanatory variables. Furthermore, we proposed a simple proposal distribution for model index variables, and used a simulation study and real example to compare the performance of the HH algorithm with our proposed and existing proposal distributions. The results show that the HH algorithm with our proposed proposal distribution is a computationally efficient and reliable selection method.     

Instrumental variable based variable selection for generalized linear models with endogenous covariates

We study the variable selection problem for a class of generalized linear models with endogenous covariates. Based on the instrumental variable adjustment technology and the smooth-threshold estimating equation (SEE) method, we propose an instrumental variable based variable selection procedure. The proposed variable selection method can attenuate the effect of endogeneity in covariates, and is easy for application in practice. Some theoretical results are also derived such as the consistency of the proposed variable selection procedure and the convergence rate of the resulting estimator. Further, some simulation studies and a real data analysis are conducted to evaluate the performance of the proposed method, and simulation results show that the proposed method is workable.     

Bayesian variable selection approach to a Bernstein polynomial regression model with stochastic constraints

This paper provides a Bayesian estimation procedure for monotone regression models incorporating the monotone trend constraint subject to uncertainty. For monotone regression modeling with stochastic restrictions, we propose a Bayesian Bernstein polynomial regression model using two-stage hierarchical prior distributions based on a family of rectangle-screened multivariate Gaussian distributions extended from the work of Gurtis and Ghosh [7 S.M. Curtis and S.K. Ghosh, A variable selection approach to monotonic regression with Bernstein polynomials, J. Appl. Stat. 38 (2011), pp. 961–976. doi: 10.1080/02664761003692423[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]]. This approach reflects the uncertainty about the prior constraint, and thus proposes a regression model subject to monotone restriction with uncertainty. Based on the proposed model, we derive the posterior distributions for unknown parameters and present numerical schemes to generate posterior samples. We show the empirical performance of the proposed model based on synthetic data and real data applications and compare the performance to the Bernstein polynomial regression model of Curtis and Ghosh [7 S.M. Curtis and S.K. Ghosh, A variable selection approach to monotonic regression with Bernstein polynomials, J. Appl. Stat. 38 (2011), pp. 961–976. doi: 10.1080/02664761003692423[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]] for the shape restriction with certainty. We illustrate the effectiveness of our proposed method that incorporates the uncertainty of the monotone trend and automatically adapts the regression function to the monotonicity, through empirical analysis with synthetic data and real data applications.     

Bayesian variable selection for the Cox regression model with missing covariates

In this paper, we develop Bayesian methodology and computational algorithms for variable subset selection in Cox proportional hazards models with missing covariate data. A new joint semi-conjugate prior for the piecewise exponential model is proposed in the presence of missing covariates and its properties are examined. The covariates are assumed to be missing at random (MAR). Under this new prior, a version of the Deviance Information Criterion (DIC) is proposed for Bayesian variable subset selection in the presence of missing covariates. Monte Carlo methods are developed for computing the DICs for all possible subset models in the model space. A Bone Marrow Transplant (BMT) dataset is used to illustrate the proposed methodology.     

Performances of Bayesian model selection criteria for generalized linear models with non-ignorably missing covariates

This article deals with model comparison as an essential part of generalized linear modelling in the presence of covariates missing not at random (MNAR). We provide an evaluation of the performances of some of the popular model selection criteria, particularly of deviance information criterion (DIC) and weighted L (WL) measure, for comparison among a set of candidate MNAR models. In addition, we seek to provide deviance and quadratic loss-based model selection criteria with alternative penalty terms targeting directly the MNAR models. This work is motivated by the need in the literature to understand the performances of these important model selection criteria for comparison among a set of MNAR models. A Monte Carlo simulation experiment is designed to assess the finite sample performances of these model selection criteria in the context of interest under different scenarios for missingness amounts. Some naturally driven DIC and WL extensions are also discussed and evaluated.     

Quantile regression and variable selection for the single-index model

In this paper, we propose a new full iteration estimation method for quantile regression (QR) of the single-index model (SIM). The asymptotic properties of the proposed estimator are derived. Furthermore, we propose a variable selection procedure for the QR of SIM by combining the estimation method with the adaptive LASSO penalized method to get sparse estimation of the index parameter. The oracle properties of the variable selection method are established. Simulations with various non-normal errors are conducted to demonstrate the finite sample performance of the estimation method and the variable selection procedure. Furthermore, we illustrate the proposed method by analyzing a real data set.     

Bayesian quantile regression and variable selection for partial linear single-index model: Using free knot spline

Partial linear single-index model (PLSIM) has both the flexibility of nonparametric treatment and interpretability of linear term, yet existing literatures about it mainly focused on mean regression, and quantile regression analysis is scarce. Based on free knot spline approximation, we apply asymmetric Laplace distribution to implement Bayesian quantile regression, and perform variable selection in linear term and index vector via binary indicators. Our approach is exempt from regularity conditions in frequentist method, and could execute variable selection and quantile regression under mutual posterior correction, which is also the first work to implement them jointly for PLSIM in fully Bayesian framework. The numerical simulation manifests the superiority of our approach to previous methods, which embodied in better efficiency of variable selection, index vector estimates and link function approximation with different error distributions. For illustration of its application, we build a power consumption model of A₂/O process in wastewater treatment and emphatically analyze the impact of water quality factors.     

Bayesian analysis of semiparametric Bernstein polynomial regression models for data with sample selection

In regression analysis, a sample selection scheme often applies to the response variable, which results in missing not at random observations on the variable. In this case, a regression analysis using only the selected cases would lead to biased results. This paper proposes a Bayesian methodology to correct this bias based on a semiparametric Bernstein polynomial regression model that incorporates the sample selection scheme into a stochastic monotone trend constraint, variable selection, and robustness against departures from the normality assumption. We present the basic theoretical properties of the proposed model that include its stochastic representation, sample selection bias quantification, and hierarchical model specification to deal with the stochastic monotone trend constraint in the nonparametric component, simple bias corrected estimation, and variable selection for the linear components. We then develop computationally feasible Markov chain Monte Carlo methods for semiparametric Bernstein polynomial functions with stochastically constrained parameter estimation and variable selection procedures. We demonstrate the finite-sample performance of the proposed model compared to existing methods using simulation studies and illustrate its use based on two real data applications.     

Sparse Bayesian variable selection in multinomial probit regression model with application to high-dimensional data classification

Here we consider a multinomial probit regression model where the number of variables substantially exceeds the sample size and only a subset of the available variables is associated with the response. Thus selecting a small number of relevant variables for classification has received a great deal of attention. Generally when the number of variables is substantial, sparsity-enforcing priors for the regression coefficients are called for on grounds of predictive generalization and computational ease. In this paper, we propose a sparse Bayesian variable selection method in multinomial probit regression model for multi-class classification. The performance of our proposed method is demonstrated with one simulated data and three well-known gene expression profiling data: breast cancer data, leukemia data, and small round blue-cell tumors. The results show that compared with other methods, our method is able to select the relevant variables and can obtain competitive classification accuracy with a small subset of relevant genes.     

WLAD-LASSO method for robust estimation and variable selection in partially linear models

This paper focuses on robust estimation and variable selection for partially linear models. We combine the weighted least absolute deviation (WLAD) regression with the adaptive least absolute shrinkage and selection operator (LASSO) to achieve simultaneous robust estimation and variable selection for partially linear models. Compared with the LAD-LASSO method, the WLAD-LASSO method will resist to the heavy-tailed errors and outliers in the parametric components. In addition, we estimate the unknown smooth function by a robust local linear regression. Under some regular conditions, the theoretical properties of the proposed estimators are established. We further examine finite-sample performance of the proposed procedure by simulation studies and a real data example.     

Robust variable selection in finite mixture of regression models using the t distribution

Abstract

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with heavy tails and outliers. In this paper, we introduce a robust variable selection procedure for FMR models using the t distribution. With appropriate selection of the tuning parameters, the consistency and the oracle property of the regularized estimators are established. To estimate the parameters of the model, we develop an EM algorithm for numerical computations and a method for selecting tuning parameters adaptively. The parameter estimation performance of the proposed model is evaluated through simulation studies. The application of the proposed model is illustrated by analyzing a real data set.     

Model selection in finite mixture of regression models: a Bayesian approach with innovative weighted g priors and reversible jump Markov chain Monte Carlo implementation

Finite mixture of regression (FMR) models are aimed at characterizing subpopulation heterogeneity stemming from different sets of covariates that impact different groups in a population. We address the contemporary problem of simultaneously conducting covariate selection and determining the number of mixture components from a Bayesian perspective that can incorporate prior information. We propose a Gibbs sampling algorithm with reversible jump Markov chain Monte Carlo implementation to accomplish concurrent covariate selection and mixture component determination in FMR models. Our Bayesian approach contains innovative features compared to previously developed reversible jump algorithms. In addition, we introduce component-adaptive weighted g priors for regression coefficients, and illustrate their improved performance in covariate selection. Numerical studies show that the Gibbs sampler with reversible jump implementation performs well, and that the proposed weighted priors can be superior to non-adaptive unweighted priors.     

Joint analysis of occurrence and time to stability after entrance into the Italian labour market: an approach based on a Bayesian cure model with structured stochastic search variable selection

Precarious employment is a serious social problem, especially in those countries, such as Italy, where there are limited benefits from social security. We investigate this phenomenon by analysing the initial part of the career of employees starting with unstable contracts for a panel of Italian workers. Our aim is to estimate the probability of getting a stable job and to detect factors influencing both this probability and the duration of precariousness. To answer these questions, we use an ad hoc mixture cure rate model in a Bayesian framework.