Bayesian MARS

A Bayesian approach to multivariate adaptive regression spline (MARS) fitting (Friedman, 1991) is proposed. This takes the form of a probability distribution over the space of possible MARS models which is explored using reversible jump Markov chain Monte Carlo methods (Green, 1995). The generated sample of MARS models produced is shown to have good predictive power when averaged and allows easy interpretation of the relative importance of predictors to the overall fit.     

Bayesian adaptive Lasso for quantile regression models with nonignorably missing response data

Abstract

Handling data with the nonignorably missing mechanism is still a challenging problem in statistics. In this paper, we develop a fully Bayesian adaptive Lasso approach for quantile regression models with nonignorably missing response data, where the nonignorable missingness mechanism is specified by a logistic regression model. The proposed method extends the Bayesian Lasso by allowing different penalization parameters for different regression coefficients. Furthermore, a hybrid algorithm that combined the Gibbs sampler and Metropolis-Hastings algorithm is implemented to simulate the parameters from posterior distributions, mainly including regression coefficients, shrinkage coefficients, parameters in the non-ignorable missing models. Finally, some simulation studies and a real example are used to illustrate the proposed methodology.     

Nonparametric regression using linear combinations of basis functions

This paper discusses a Bayesian approach to nonparametric regression initially proposed by Smith and Kohn (1996. Journal of Econometrics 75: 317–344). In this approach the regression function is represented as a linear combination of basis terms. The basis terms can be univariate or multivariate functions and can include polynomials, natural splines and radial basis functions. A Bayesian hierarchical model is used such that the coefficient of each basis term can be zero with positive prior probability. The presence of basis terms in the model is determined by latent indicator variables. The posterior mean is estimated by Markov chain Monte Carlo simulation because it is computationally intractable to compute the posterior mean analytically unless a small number of basis terms is used. The present article updates the work of Smith and Kohn (1996. Journal of Econometrics 75: 317–344) to take account of work by us and others over the last three years. A careful discussion is given to all aspects of the model specification, function estimation and the use of sampling schemes. In particular, new sampling schemes are introduced to carry out the variable selection methodology.     

A Bayesian Analysis of Autoregressive Models with Exogenous Variables and Power-Transformed and Threshold GARCH Errors

Consider a class of autoregressive models with exogenous variables and power transformed and threshold GARCH (ARX-PTTGARCH) errors, which is a natural generalization of the standard and special GARCH model. We propose a Bayesian method to show that combining Gibbs sampler and Metropolis-Hastings algorithm to give a Bayesian analysis can be applied to estimate parameters of ARX-PTTGARCH models with success.     

A semiparametric Bayesian approach to simplex regression model with heterogeneous dispersion

ABSTRACT

Simplex regression model is often employed to analyze continuous proportion data in many studies. In this paper, we extend the assumption of a constant dispersion parameter (homogeneity) to varying dispersion parameter (heterogeneity) in Simplex regression model, and present the B-spline to approximate the smoothing unknown function within the Bayesian framework. A hybrid algorithm combining the block Gibbs sampler and the Metropolis-Hastings algorithm is presented for sampling observations from the posterior distribution. The procedures for computing model comparison criteria such as conditional predictive ordinate statistic, deviance information criterion, and averaged mean squared error are presented. Also, we develop a computationally feasible Bayesian case-deletion influence measure based on the Kullback-Leibler divergence. Several simulation studies and a real example are employed to illustrate the proposed methodologies.     

Equation-solving estimator based on Metropolis-Hastings algorithm with delayed rejection

Numerous works have recently attempted to develop more efficient estimators for MCMC inference than classical ones. In this perspective and approximate nonstandard discrete distributions, Liang and Liu proposed the equation solving estimator as an alternative to the conventional frequency estimator. The specific MCMC method used is the Metropolis-Hastings (M-H) algorithm. In this work, we propose to adapt the equation-solving estimator to the context of simulation using the Metropolis-Hastings algorithm with delayed rejection (MHDR). Developed originally by Mira, this algorithm is considered an improved version of the standard M-H sampler which aims to reduce the variance of MCMC estimators. An application to a Bayesian hypothesis test problem shows the superiority of the equation-solving estimator, based on MHDR sampling, over the one introduced by Liang and Liu.     

Minimum average deviance estimation for sufficient dimension reduction

Sufficient dimension reduction methods aim to reduce the dimensionality of predictors while preserving regression information relevant to the response. In this article, we develop Minimum Average Deviance Estimation (MADE) methodology for sufficient dimension reduction. The purpose of MADE is to generalize Minimum Average Variance Estimation (MAVE) beyond its assumption of additive errors to settings where the outcome follows an exponential family distribution. As in MAVE, a local likelihood approach is used to learn the form of the regression function from the data and the main parameter of interest is a dimension reduction subspace. To estimate this parameter within its natural space, we propose an iterative algorithm where one step utilizes optimization on the Stiefel manifold. MAVE is seen to be a special case of MADE in the case of Gaussian outcomes with a common variance. Several procedures are considered to estimate the reduced dimension and to predict the outcome for an arbitrary covariate value. Initial simulations and data analysis examples yield encouraging results and invite further exploration of the methodology.     

Bayesian inference for pairwise interacting point processes

Pairwise interacting point processes are commonly used to model spatial point patterns. To perform inference, the established frequentist methods can produce good point estimates when the interaction in the data is moderate, but some methods may produce severely biased estimates when the interaction in strong. Furthermore, because the sampling distributions of the estimates are unclear, interval estimates are typically obtained by parametric bootstrap methods. In the current setting however, the behavior of such estimates is not well understood. In this article we propose Bayesian methods for obtaining inferences in pairwise interacting point processes. The requisite application of Markov chain Monte Carlo (MCMC) techniques is complicated by an intractable function of the parameters in the likelihood. The acceptance probability in a Metropolis-Hastings algorithm involves the ratio of two likelihoods evaluated at differing parameter values. The intractable functions do not cancel, and hence an intractable ratio r must be estimated within each iteration of a Metropolis-Hastings sampler. We propose the use of importance sampling techniques within MCMC to address this problem. While r may be estimated by other methods, these, in general, are not readily applied in a Bayesian setting. We demonstrate the validity of our importance sampling approach with a small simulation study. Finally, we analyze the Swedish pine sapling dataset (Strand 1972) and contrast the results with those in the literature.     

Modeling a mixture of ordinal and continuous repeated measures

We study the correlation structure for a mixture of ordinal and continuous repeated measures using a Bayesian approach. We assume a multivariate probit model for the ordinal variables and a normal linear regression for the continuous variables, where latent normal variables underlying the ordinal data are correlated with continuous variables in the model. Due to the probit model assumption, we are required to sample a covariance matrix with some of the diagonal elements equal to one. The key computational idea is to use parameter-extended data augmentation, which involves applying the Metropolis-Hastings algorithm to get a sample from the posterior distribution of the covariance matrix incorporating the relevant restrictions. The methodology is illustrated through a simulated example and through an application to data from the UCLA Brain Injury Research Center.     

Independent screening in high-dimensional exponential family predictors’ space

We present a methodology for screening predictors that, given the response, follow a one-parameter exponential family distributions. Screening predictors can be an important step in regressions when the number of predictors p is excessively large or larger than n the number of observations. We consider instances where a large number of predictors are suspected irrelevant for having no information about the response. The proposed methodology helps remove these irrelevant predictors while capturing those linearly or nonlinearly related to the response.     

Bayesian analysis of a linear mixed model with AR(p) errors via MCMC

We develop Bayesian procedures to make inference about parameters of a statistical design with autocorrelated error terms. Modelling treatment effects can be complex in the presence of other factors such as time; for example in longitudinal data. In this paper, Markov chain Monte Carlo methods (MCMC), the Metropolis-Hastings algorithm and Gibbs sampler are used to facilitate the Bayesian analysis of real life data when the error structure can be expressed as an autoregressive model of order p. We illustrate our analysis with real data.     

Bayesian regression analysis of data with censored initiating and terminating times: applications to aids

Data with censored initiating and terminating times arises quite frequently in acquired immunodeficiency syndrome (AIDS) epidemiologic studies. Analysis of such data involves a complicated bivariate likelihood, which is difficult to deal with computationally. Bayesian analysis, op the other hand, presents added complexities that have yet to be resolved. By exploiting the simple form of a complete data likelihood and utilizing the power of a Markov Chain Monte Carlo (MCMC) algorithm, this paper presents a methodology for fitting Bayesian regression models to such data. The proposed methods extend the work of Sinha (1997), who considered non-parametric Bayesian analysis of this type of data. The methodology is illustiated with an application to a cohort of HIV infected hemophiliac patients.     

Bayesian Analysis of Two-Regime Threshold Autoregressive Moving Average Model with Exogenous Inputs

We consider Bayesian analysis of threshold autoregressive moving average model with exogenous inputs (TARMAX). In order to obtain the desired marginal posterior distributions of all parameters including the threshold value of the two-regime TARMAX model, we use two different Markov chain Monte Carlo (MCMC) methods to apply Gibbs sampler with Metropolis-Hastings algorithm. The first one is used to obtain iterative least squares estimates of the parameters. The second one includes two MCMC stages for estimate the desired marginal posterior distributions and the parameters. Simulation experiments and a real data example show support to our approaches.     

Application of a predictive distribution formula to Bayesian computation for incomplete data models

We consider exact and approximate Bayesian computation in the presence of latent variables or missing data. Specifically we explore the application of a posterior predictive distribution formula derived in Sweeting And Kharroubi (2003), which is a particular form of Laplace approximation, both as an importance function and a proposal distribution. We show that this formula provides a stable importance function for use within poor man’s data augmentation schemes and that it can also be used as a proposal distribution within a Metropolis-Hastings algorithm for models that are not analytically tractable. We illustrate both uses in the case of a censored regression model and a normal hierarchical model, with both normal and Student t distributed random effects. Although the predictive distribution formula is motivated by regular asymptotic theory, it is not necessary that the likelihood has a closed form or that it possesses a local maximum.     

Robust Bayesian methodology with applications in credibility premium derivation and future claim size prediction

Robust Bayesian methodology deals with the problem of explaining uncertainty of the inputs (the prior, the model, and the loss function) and provides a breakthrough way to take into account the input’s variation. If the uncertainty is in terms of the prior knowledge, robust Bayesian analysis provides a way to consider the prior knowledge in terms of a class of priors \(\varGamma \) and derive some optimal rules. In this paper, we motivate utilizing robust Bayes methodology under the asymmetric general entropy loss function in insurance and pursue two main goals, namely (i) computing premiums and (ii) predicting a future claim size. To achieve the goals, we choose some classes of priors and deal with (i) Bayes and posterior regret gamma minimax premium computation, (ii) Bayes and posterior regret gamma minimax prediction of a future claim size under the general entropy loss. We also perform a prequential analysis and compare the performance of posterior regret gamma minimax predictors against the Bayes predictors.     

Equation-solving estimator based on the general n-step MHDR algorithm

Markov chain Monte Carlo (MCMC) methods provide an important means to simulate from almost any probability density. To approximate non-standard discrete distributions, the equation-solving MCMC estimator was developed as an alternative to the classical frequency estimator. The used simulation scheme is the Metropolis–Hastings (M–H) algorithm. Recently, this estimator has been extended to the specific context of 2-step Metropolis-Hastings with delayed rejection (MHDR) algorithm, which allowed a considerable reduction in asymptotic variance. In this paper, we propose an adaptation of equation-solving estimator to the case of general n-step MHDR sampler. The aim is to further improve the precision. An application to a Bayesian hypothesis test problem shows the high performance, in terms of accuracy, of the equation-solving estimator, based on a MHDR algorithm with more than two stages.     

A bayesian predictive approach to the selection of variables in multiple regression

The squared error loss function applied to Bayesian predictive distributions is investigated as a variable selection criterion in linear regression equations. It is illustrated that “cost-free” variables may be eliminated if they are poor predictors. Regression models where the predictors are fixed and where they are stochastic are both considered. An empirical examination of the criterion and a comparison with other techniques are presented.     

Bayesian variable selection with related predictors

In data sets with many predictors, algorithms for identifying a good subset of predictors are often used. Most such algorithms do not allow for any relationships between predictors. For example, stepwise regression might select a model containing an interaction AB but neither main effect A or B. This paper develops mathematical representations of this and other relations between predictors, which may then be incorporated in a model selection procedure. A Bayesian approach that goes beyond the standard independence prior for variable selection is adopted, and preference for certain models is interpreted as prior information. Priors relevant to arbitrary interactions and polynomials, dummy variables for categorical factors, competing predictors, and restrictions on the size of the models are developed. Since the relations developed are for priors, they may be incorporated in any Bayesian variable selection algorithm for any type of linear model. The application of the methods here is illustrated via the stochastic search variable selection algorithm of George and McCulloch (1993), which is modified to utilize the new priors. The performance of the approach is illustrated with two constructed examples and a computer performance dataset.     

Bayesian analysis of dynamic magnetic resonance breast images

Summary.  We describe an integrated methodology for analysing dynamic magnetic resonance images of the breast. The problems that motivate this methodology arise from a collaborative study with a tumour institute. The methods are developed within the Bayesian framework and comprise image restoration and classification steps. Two different approaches are proposed for the restoration. Bayesian inference is performed by means of Markov chain Monte Carlo algorithms. We make use of a Metropolis algorithm with a specially chosen proposal distribution that performs better than more commonly used proposals. The classification step is based on a few attribute images yielded by the restoration step that describe the essential features of the contrast agent variation over time. Procedures for hyperparameter estimation are provided, so making our method automatic. The results show the potential of the methodology to extract useful information from acquired dynamic magnetic resonance imaging data about tumour morphology and internal pathophysiological features.     

Parameter expansion for sampling a correlation matrix: an efficient GPX-RPMH algorithm

Sampling the correlation matrix (R) plays an important role in statistical inference for correlated models. There are two main constraints on a correlation matrix: positive definiteness and fixed diagonal elements. These constraints make sampling R difficult. In this paper, an efficient generalized parameter expanded re-parametrization and Metropolis-Hastings (GPX-RPMH) algorithm for sampling a correlation matrix is proposed. Drawing all components of R simultaneously from its full conditional distribution is realized by first drawing a covariance matrix from the derived parameter expanded candidate density (PXCD), and then translating it back to a correlation matrix and accepting it according to a Metropolis-Hastings (M-H) acceptance rate. The mixing rate in the M-H step can be adjusted through a class of tuning parameters embedded in the generalized candidate prior (GCP), which is chosen for R to derive the PXCD. This algorithm is illustrated using multivariate regression (MVR) models and a simulation study shows that the performance of the GPX-RPMH algorithm is more efficient than that of other methods.