A robust multivariate measurement error model with skew-normal/independent distributions and Bayesian MCMC implementation

Skew-normal/independent distributions are a class of asymmetric thick-tailed distributions that include the skew-normal distribution as a special case. In this paper, we explore the use of Markov Chain Monte Carlo (MCMC) methods to develop a Bayesian analysis in multivariate measurement errors models. We propose the use of skew-normal/independent distributions to model the unobserved value of the covariates (latent variable) and symmetric normal/independent distributions for the random errors term, providing an appealing robust alternative to the usual symmetric process in multivariate measurement errors models. Among the distributions that belong to this class of distributions, we examine univariate and multivariate versions of the skew-normal, skew-t, skew-slash and skew-contaminated normal distributions. The results and methods are applied to a real data set.     

Bayesian variable selection with related predictors

In data sets with many predictors, algorithms for identifying a good subset of predictors are often used. Most such algorithms do not allow for any relationships between predictors. For example, stepwise regression might select a model containing an interaction AB but neither main effect A or B. This paper develops mathematical representations of this and other relations between predictors, which may then be incorporated in a model selection procedure. A Bayesian approach that goes beyond the standard independence prior for variable selection is adopted, and preference for certain models is interpreted as prior information. Priors relevant to arbitrary interactions and polynomials, dummy variables for categorical factors, competing predictors, and restrictions on the size of the models are developed. Since the relations developed are for priors, they may be incorporated in any Bayesian variable selection algorithm for any type of linear model. The application of the methods here is illustrated via the stochastic search variable selection algorithm of George and McCulloch (1993), which is modified to utilize the new priors. The performance of the approach is illustrated with two constructed examples and a computer performance dataset.     

Multivariate partially linear single-index models: Bayesian analysis

Partially linear single-index models play important roles in advanced non-/semi-parametric statistics due to their generality and flexibility. We generalise these models from univariate response to multivariate responses. A Bayesian method with free-knot spline is used to analyse the proposed models, including the estimation and the prediction, and a Metropolis-within-Gibbs sampler is provided for posterior exploration. We also utilise the partially collapsed idea in our algorithm to speed up the convergence. The proposed models and methods of analysis are demonstrated by simulation studies and are applied to a real data set.     

Robust linear mixed models with skew-normal independent distributions from a Bayesian perspective

Linear mixed models were developed to handle clustered data and have been a topic of increasing interest in statistics for the past 50 years. Generally, the normality (or symmetry) of the random effects is a common assumption in linear mixed models but it may, sometimes, be unrealistic, obscuring important features of among-subjects variation. In this article, we utilize skew-normal/independent distributions as a tool for robust modeling of linear mixed models under a Bayesian paradigm. The skew-normal/independent distributions is an attractive class of asymmetric heavy-tailed distributions that includes the skew-normal distribution, skew-t, skew-slash and the skew-contaminated normal distributions as special cases, providing an appealing robust alternative to the routine use of symmetric distributions in this type of models. The methods developed are illustrated using a real data set from Framingham cholesterol study.     

Bayesian variable selection and estimation in maximum entropy quantile regression

Quantile regression has gained increasing popularity as it provides richer information than the regular mean regression, and variable selection plays an important role in the quantile regression model building process, as it improves the prediction accuracy by choosing an appropriate subset of regression predictors. Unlike the traditional quantile regression, we consider the quantile as an unknown parameter and estimate it jointly with other regression coefficients. In particular, we adopt the Bayesian adaptive Lasso for the maximum entropy quantile regression. A flat prior is chosen for the quantile parameter due to the lack of information on it. The proposed method not only addresses the problem about which quantile would be the most probable one among all the candidates, but also reflects the inner relationship of the data through the estimated quantile. We develop an efficient Gibbs sampler algorithm and show that the performance of our proposed method is superior than the Bayesian adaptive Lasso and Bayesian Lasso through simulation studies and a real data analysis.     

A simulation approach to convergence rates for Markov chain Monte Carlo algorithms

Markov chain Monte Carlo (MCMC) methods, including the Gibbs sampler and the Metropolis–Hastings algorithm, are very commonly used in Bayesian statistics for sampling from complicated, high-dimensional posterior distributions. A continuing source of uncertainty is how long such a sampler must be run in order to converge approximately to its target stationary distribution. A method has previously been developed to compute rigorous theoretical upper bounds on the number of iterations required to achieve a specified degree of convergence in total variation distance by verifying drift and minorization conditions. We propose the use of auxiliary simulations to estimate the numerical values needed in this theorem. Our simulation method makes it possible to compute quantitative convergence bounds for models for which the requisite analytical computations would be prohibitively difficult or impossible. On the other hand, although our method appears to perform well in our example problems, it cannot provide the guarantees offered by analytical proof.     

Testing a linear ARMA model against threshold-ARMA models: A Bayesian approach

We introduce a Bayesian approach to test linear autoregressive moving-average (ARMA) models against threshold autoregressive moving-average (TARMA) models. First, the marginal posterior densities of all parameters, including the threshold and delay, of a TARMA model are obtained by using Gibbs sampler with Metropolis–Hastings algorithm. Second, reversible-jump Markov chain Monte Carlo (RJMCMC) method is adopted to calculate the posterior probabilities for ARMA and TARMA models: Posterior evidence in favor of TARMA models indicates threshold nonlinearity. Finally, based on RJMCMC scheme and Akaike information criterion (AIC) or Bayesian information criterion (BIC), the procedure for modeling TARMA models is exploited. Simulation experiments and a real data example show that our method works well for distinguishing an ARMA from a TARMA model and for building TARMA models.     

Bayesian subset selection approach to ranking normal means

In this, article we consider a Bayesian approach to the problem of ranking the means of normal distributed populations, which is a common problem in the biological sciences. We use a decision-theoretic approach with a straightforward loss function to determine a set of candidate rankings. This loss function allows the researcher to balance the risk of not including the correct ranking with the risk of increasing the number of rankings selected. We apply our new procedure to an example regarding the effect of zinc on the diversity of diatom species.     

An MCMC model search algorithm for regression problems

In this paper, we improve upon the Carlin and Chib Markov chain Monte Carlo algorithm that searches in model and parameter spaces. Our proposed algorithm attempts non-uniformly chosen ‘local’ moves in the model space and avoids some pitfalls of other existing algorithms. In a series of examples with linear and logistic regression, we report evidence that our proposed algorithm performs better than the existing algorithms.     

基于逆跳MCMC的贝叶斯分位自回归模型研究

考虑到传统信息理论方法确定模型存在不足,在贝叶斯理论框架下提出了基于逆跳马尔可夫链蒙特卡罗法确定分位自回归模型阶次的方法。在时间序列服从非对称Laplace分布的条件下,设计了马尔可夫链蒙特卡罗数值计算程序,得到了不同分位数下模型参数的贝叶斯估计值。实证研究表明：基于逆跳马尔可夫链蒙特卡罗法的贝叶斯分位自回归模型能有效地揭示滞后变量对响应变量的位置、尺度和形状的影响。     

A MCMC algorithm to fit a general exchangeable model

Consider the exchangeable Bayesian hierarchical model where observations y_i are independently distributed from sampling densities with unknown means, the means µi, are a random sample from a distribution g, and the parameters of g are assigned a known distribution h. A simple algorithm is presented for summarizing the posterior distribution based on Gibbs sampling and the Metropolis algorithm. The software program Matlab is used to implement the algorithm and provide a graphical output analysis. An binomial example is used to illustrate the flexibility of modeling possible using this algorithm. Methods of model checking and extensions to hierarchical regression modeling are discussed.     

Clustering Gene Expression Data using a Posterior Split‐Merge‐Birth Procedure

Abstract. DNA array technology is an important tool for genomic research due to its capa‐city of measuring simultaneously the expression levels of a great number of genes or fragments of genes in different experimental conditions. An important point in gene expression data analysis is to identify clusters of genes which present similar expression levels. We propose a new procedure for estimating the mixture model for clustering of gene expression data. The proposed method is a posterior split‐merge‐birth MCMC procedure which does not require the specification of the number of components, since it is estimated jointly with component parameters. The strategy for splitting is based on data and on posterior distribution from the previously allocated observations. This procedure defines a quick split proposal in contrary to other split procedures, which require substantial computational effort. The performance of the method is verified using real and simulated datasets.     

Bayesian latent variable models for clustered mixed outcomes

A general framework is proposed for modelling clustered mixed outcomes. A mixture of generalized linear models is used to describe the joint distribution of a set of underlying variables, and an arbitrary function relates the underlying variables to be observed outcomes. The model accommodates multilevel data structures, general covariate effects and distinct link functions and error distributions for each underlying variable. Within the framework proposed, novel models are developed for clustered multiple binary, unordered categorical and joint discrete and continuous outcomes. A Markov chain Monte Carlo sampling algorithm is described for estimating the posterior distributions of the parameters and latent variables. Because of the flexibility of the modelling framework and estimation procedure, extensions to ordered categorical outcomes and more complex data structures are straightforward. The methods are illustrated by using data from a reproductive toxicity study.     

Bayesian variable selection in logistic regression: predicting company earnings direction

This paper presents a Bayesian technique for the estimation of a logistic regression model including variable selection. As in Ou & Penman (1989), the model is used to predict the direction of company earnings, one year ahead, from a large set of accounting variables from financial statements. To estimate the model, the paper presents a Markov chain Monte Carlo sampling scheme that includes the variable selection technique of Smith & Kohn (1996) and the non-Gaussian estimation method of Mira & Tierney (2001). The technique is applied to data for companies in the United States and Australia. The results obtained compare favourably to the technique used by Ou & Penman (1989) for both regions.     

Model selection in finite mixture of regression models: a Bayesian approach with innovative weighted g priors and reversible jump Markov chain Monte Carlo implementation

Finite mixture of regression (FMR) models are aimed at characterizing subpopulation heterogeneity stemming from different sets of covariates that impact different groups in a population. We address the contemporary problem of simultaneously conducting covariate selection and determining the number of mixture components from a Bayesian perspective that can incorporate prior information. We propose a Gibbs sampling algorithm with reversible jump Markov chain Monte Carlo implementation to accomplish concurrent covariate selection and mixture component determination in FMR models. Our Bayesian approach contains innovative features compared to previously developed reversible jump algorithms. In addition, we introduce component-adaptive weighted g priors for regression coefficients, and illustrate their improved performance in covariate selection. Numerical studies show that the Gibbs sampler with reversible jump implementation performs well, and that the proposed weighted priors can be superior to non-adaptive unweighted priors.     

On the Flexibility of Metropolis–Hastings Acceptance Probabilities in Auxiliary Variable Proposal Generation

Abstract. Use of auxiliary variables for generating proposal variables within a Metropolis–Hastings setting has been suggested in many different settings. This has in particular been of interest for simulation from complex distributions such as multimodal distributions or in transdimensional approaches. For many of these approaches, the acceptance probabilities that are used turn up somewhat magic and different proofs for their validity have been given in each case. In this article, we will present a general framework for construction of acceptance probabilities in auxiliary variable proposal generation. In addition to showing the similarities between many of the proposed algorithms in the literature, the framework also demonstrates that there is a great flexibility in how to construct acceptance probabilities. With this flexibility, alternative acceptance probabilities are suggested. Some numerical experiments are also reported.     

Bayesian analysis of joint mean and covariance models for longitudinal data

Efficient estimation of the regression coefficients in longitudinal data analysis requires a correct specification of the covariance structure. If misspecification occurs, it may lead to inefficient or biased estimators of parameters in the mean. One of the most commonly used methods for handling the covariance matrix is based on simultaneous modeling of the Cholesky decomposition. Therefore, in this paper, we reparameterize covariance structures in longitudinal data analysis through the modified Cholesky decomposition of itself. Based on this modified Cholesky decomposition, the within-subject covariance matrix is decomposed into a unit lower triangular matrix involving moving average coefficients and a diagonal matrix involving innovation variances, which are modeled as linear functions of covariates. Then, we propose a fully Bayesian inference for joint mean and covariance models based on this decomposition. A computational efficient Markov chain Monte Carlo method which combines the Gibbs sampler and Metropolis–Hastings algorithm is implemented to simultaneously obtain the Bayesian estimates of unknown parameters, as well as their standard deviation estimates. Finally, several simulation studies and a real example are presented to illustrate the proposed methodology.     

A guided walk Metropolis algorithm

The random walk Metropolis algorithm is a simple Markov chain Monte Carlo scheme which is frequently used in Bayesian statistical problems. We propose a guided walk Metropolis algorithm which suppresses some of the random walk behavior in the Markov chain. This alternative algorithm is no harder to implement than the random walk Metropolis algorithm, but empirical studies show that it performs better in terms of efficiency and convergence time.     

Bayesian inference for joint location and scale nonlinear models with skew-normal errors

A regression model with skew-normal errors provides a useful extension for ordinary normal regression models when the dataset under consideration involves asymmetric outcomes. In this article, we explore the use of Markov Chain Monte Carlo (MCMC) methods to develop a Bayesian analysis for joint location and scale nonlinear models with skew-normal errors, which relax the normality assumption and include the normal one as a special case. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of MCMC methods to simulate samples from the joint posterior distribution. Finally, simulation studies and a real example are used to illustrate the proposed methodology.     

Bayesian mixture models for complex high dimensional count data in phage display experiments

Summary.  Phage display is a biological process that is used to screen random peptide libraries for ligands that bind to a target of interest with high affinity. On the basis of a count data set from an innovative multistage phage display experiment, we propose a class of Bayesian mixture models to cluster peptide counts into three groups that exhibit different display patterns across stages. Among the three groups, the investigators are particularly interested in that with an ascending display pattern in the counts, which implies that the peptides are likely to bind to the target with strong affinity. We apply a Bayesian false discovery rate approach to identify the peptides with the strongest affinity within the group. A list of peptides is obtained, among which important ones with meaningful functions are further validated by biologists. To examine the performance of the Bayesian model, we conduct a simulation study and obtain desirable results.