VARIATIONAL BAYESIAN ANALYSIS FOR HIDDEN MARKOV MODELS

The variational approach to Bayesian inference enables simultaneous estimation of model parameters and model complexity. An interesting feature of this approach is that it also leads to an automatic choice of model complexity. Empirical results from the analysis of hidden Markov models with Gaussian observation densities illustrate this. If the variational algorithm is initialized with a large number of hidden states, redundant states are eliminated as the method converges to a solution, thereby leading to a selection of the number of hidden states. In addition, through the use of a variational approximation, the deviance information criterion for Bayesian model selection can be extended to the hidden Markov model framework. Calculation of the deviance information criterion provides a further tool for model selection, which can be used in conjunction with the variational approach.     

A semiparametric Bayesian approach to simplex regression model with heterogeneous dispersion

ABSTRACT

Simplex regression model is often employed to analyze continuous proportion data in many studies. In this paper, we extend the assumption of a constant dispersion parameter (homogeneity) to varying dispersion parameter (heterogeneity) in Simplex regression model, and present the B-spline to approximate the smoothing unknown function within the Bayesian framework. A hybrid algorithm combining the block Gibbs sampler and the Metropolis-Hastings algorithm is presented for sampling observations from the posterior distribution. The procedures for computing model comparison criteria such as conditional predictive ordinate statistic, deviance information criterion, and averaged mean squared error are presented. Also, we develop a computationally feasible Bayesian case-deletion influence measure based on the Kullback-Leibler divergence. Several simulation studies and a real example are employed to illustrate the proposed methodologies.     

Probit and Logit Model Selection

Monte Carlo experiments are conducted to compare the Bayesian and sample theory model selection criteria in choosing the univariate probit and logit models. We use five criteria: the deviance information criterion (DIC), predictive deviance information criterion (PDIC), Akaike information criterion (AIC), weighted, and unweighted sums of squared errors. The first two criteria are Bayesian while the others are sample theory criteria. The results show that if data are balanced none of the model selection criteria considered in this article can distinguish the probit and logit models. If data are unbalanced and the sample size is large the DIC and AIC choose the correct models better than the other criteria. We show that if unbalanced binary data are generated by a leptokurtic distribution the logit model is preferred over the probit model. The probit model is preferred if unbalanced data are generated by a platykurtic distribution. We apply the model selection criteria to the probit and logit models that link the ups and downs of the returns on S&P500 to the crude oil price.     

Bayesian measures of model complexity and fit

Summary. We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure p _D for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general p _D approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the 'hat' matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding p _D to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.     

Bayesian spatial quantile regression for areal count data,with application on substitute care placements in Texas

Quantile regression (QR) allows one to model the effect of covariates across the entire response distribution, rather than only at the mean, but QR methods have been almost exclusively applied to continuous response variables and without considering spatial effects. Of the few studies that have performed QR on count data, none have included random spatial effects, which is an integral facet of the Bayesian spatial QR model for areal counts that we propose. Additionally, we introduce a simplifying alternative to the response variable transformation currently employed in the QR for counts literature. The efficacy of the proposed model is demonstrated via simulation study and on a real data application from the Texas Department of Family and Protective Services (TDFPS). Our model outperforms a comparable non-spatial model in both instances, as evidenced by the deviance information criterion (DIC) and coverage probabilities. With the TDFPS data, we identify one of four covariates, along with the intercept, as having a nonconstant effect across the response distribution.     

Bayesian parametric accelerated failure time spatial model and its application to prostate cancer

Prostate cancer (PrCA) is the most common cancer diagnosed in American men and the second leading cause of death from malignancies. There are large geographical variation and racial disparities existing in the survival rate of PrCA. Much work on the spatial survival model is based on the proportional hazards (PH) model, but few focused on the accelerated failure time (AFT) model. In this paper, we investigate the PrCA data of Louisiana from the Surveillance, Epidemiology, and End Results program and the violation of the PH assumption suggests that the spatial survival model based on the AFT model is more appropriate for this data set. To account for the possible extra-variation, we consider spatially referenced independent or dependent spatial structures. The deviance information criterion is used to select a best-fitting model within the Bayesian frame work. The results from our study indicate that age, race, stage, and geographical distribution are significant in evaluating PrCA survival.     

A model of toxoplasmosis incidence in the UK: evidence synthesis and consistency of evidence

Summary.  We present a Bayesian evidence synthesis model combining data on seroprevalence, seroconversion and tests of recent infection, to produce estimates of current incidence of toxoplasmosis in the UK. The motivation for the study was the need for an estimate of current average incidence in the UK, with a realistic assessment of its uncertainty, to inform a decision model for a national screening programme to prevent congenital toxoplasmosis. The model has a hierarchical structure over geographic region, a random-walk model for temporal effects and a fixed age effect, with one or more types of data informing the regional estimates of incidence. Inference is obtained by using Markov chain Monte Carlo simulations. A key issue in the synthesis of evidence from multiple sources is model selection and the consistency of different types of evidence. Alternative models of incidence are compared by using the deviance information criterion, and we find that temporal effects are region specific. We assess the consistency of the various forms of evidence by using cross-validation where practical, and posterior and mixed prediction otherwise, and we discuss how these measures can be used to assess different aspects of consistency in a complex evidence structure. We discuss the contribution of the various forms of evidence to estimated current average incidence.     

A Bayesian approach of joint models for clustered zero-inflated count data with skewness and measurement errors

Count data with excess zeros are widely encountered in the fields of biomedical, medical, public health and social survey, etc. Zero-inflated Poisson (ZIP) regression models with mixed effects are useful tools for analyzing such data, in which covariates are usually incorporated in the model to explain inter-subject variation and normal distribution is assumed for both random effects and random errors. However, in many practical applications, such assumptions may be violated as the data often exhibit skewness and some covariates may be measured with measurement errors. In this paper, we deal with these issues simultaneously by developing a Bayesian joint hierarchical modeling approach. Specifically, by treating intercepts and slopes in logistic and Poisson regression as random, a flexible two-level ZIP regression model is proposed, where a covariate process with measurement errors is established and a skew-t-distribution is considered for both random errors and random effects. Under the Bayesian framework, model selection is carried out using deviance information criterion (DIC) and a goodness-of-fit statistics is also developed for assessing the plausibility of the posited model. The main advantage of our method is that it allows for more robustness and correctness for investigating heterogeneity from different levels, while accommodating the skewness and measurement errors simultaneously. An application to Shanghai Youth Fitness Survey is used as an illustrate example. Through this real example, it is showed that our approach is of interest and usefulness for applications.     

Non-mixture cure correlated frailty models in Bayesian approach

In this article, we develop a Bayesian approach for the estimation of two cure correlated frailty models that have been extended to the cure frailty models introduced by Yin [34]. We used the two different type of frailty with bivariate log-normal distribution instead of gamma distribution. A likelihood function was constructed based on a piecewise exponential distribution function. The model parameters were estimated by the Markov chain Monte Carlo method. The comparison of models is based on the Cox correlated frailty model with log-normal distribution. A real data set of bilateral corneal graft rejection was used to compare these models. The results of this data, based on deviance information criteria, showed the advantage of the proposed models.     

Estimation and inference for exponential smooth transition nonlinear volatility models

A family of threshold nonlinear generalised autoregressive conditionally heteroscedastic models is considered, that allows smooth transitions between regimes, capturing size asymmetry via an exponential smooth transition function. A Bayesian approach is taken and an efficient adaptive sampling scheme is employed for inference, including a novel extension to a recently proposed prior for the smoothing parameter that solves a likelihood identification problem. A simulation study illustrates that the sampling scheme performs well, with the chosen prior kept close to uninformative, while successfully ensuring identification of model parameters and accurate inference for the smoothing parameter. An empirical study confirms the potential suitability of the model, highlighting the presence of both mean and volatility (size) asymmetry; while the model is favoured over modern, popular model competitors, including those with sign asymmetry, via the deviance information criterion.     

Bayesian hierarchical models for linear networks

The purpose of this study is to highlight dangerous motorways via estimating the intensity of accidents and study its pattern across the UK motorway network. Two methods have been developed to achieve this aim. First, the motorway-specific intensity is estimated by using a homogeneous Poisson process. The heterogeneity across motorways is incorporated using two-level hierarchical models. The data structure is multilevel since each motorway consists of junctions that are joined by grouped segments. In the second method, the segment-specific intensity is estimated. The homogeneous Poisson process is used to model accident data within grouped segments but heterogeneity across grouped segments is incorporated using three-level hierarchical models. A Bayesian method via Markov Chain Monte Carlo is used to estimate the unknown parameters in the models and the sensitivity to the choice of priors is assessed. The performance of the proposed models is evaluated by a simulation study and an application to traffic accidents in 2016 on the UK motorway network. The deviance information criterion (DIC) and the widely applicable information criterion (WAIC) are employed to choose between models.     

Hierarchical Bayesian meta-analysis models for cross-platform microarray studies

The development of new technologies to measure gene expression has been calling for statistical methods to integrate findings across multiple-platform studies. A common goal of microarray analysis is to identify genes with differential expression between two conditions, such as treatment versus control. Here, we introduce a hierarchical Bayesian meta-analysis model to pool gene expression studies from different microarray platforms: spotted DNA arrays and short oligonucleotide arrays. The studies have different array design layouts, each with multiple sources of data replication, including repeated experiments, slides and probes. Our model produces the gene-specific posterior probability of differential expression, which is the basis for inference. In simulations combining two and five independent studies, our meta-analysis model outperformed separate analyses for three commonly used comparison measures; it also showed improved receiver operating characteristic curves. When combining spotted DNA and CombiMatrix short oligonucleotide array studies of Geobacter sulfurreducens, our meta-analysis model discovered more genes for fixed thresholds of posterior probability of differential expression and Bayesian false discovery than individual study analyses. We also examine an alternative model and compare models using the deviance information criterion.     

Bayesian misclassification and propensity score methods for clustered observational studies

Bayesian propensity score regression analysis with misclassified binary responses is proposed to analyse clustered observational data. This approach utilizes multilevel models and corrects for misclassification in the responses. Using the deviance information criterion (DIC), the performance of the approach is compared with approaches without correcting for misclassification, multilevel structure specification, or both in the study of the impact of female employment on the likelihood of physical violence. The smallest DIC confirms that our proposed model best fits the data. We conclude that female employment has an insignificant impact on the likelihood of physical spousal violence towards women. In addition, a simulation study confirms that the proposed approach performed best in terms of bias and coverage rate. Ignoring misclassification in response or multilevel structure of data would yield biased estimation of the exposure effect.     

Bayesian Inference in Dynamic Disequilibrium Models: An Application to the Polish Credit Market

We propose a Bayesian approach for inference in a dynamic disequilibrium model. To circumvent the difficulties raised by the Maddala and Nelson (1974) specification in the dynamic case, we analyze a dynamic extended version of the disequilibrium model of Ginsburgh et al. (1980). We develop a Gibbs sampler based on the simulation of the missing observations. The feasibility of the approach is illustrated by an empirical analysis of the Polish credit market, for which we conduct a specification search using the posterior deviance criterion of Spiegelhalter et al. (2002).     

On Variational Bayes Estimation and Variational Information Criteria for Linear Regression Models

Variational Bayes (VB) estimation is a fast alternative to Markov Chain Monte Carlo for performing approximate Baesian inference. This procedure can be an efficient and effective means of analyzing large datasets. However, VB estimation is often criticised, typically on empirical grounds, for being unable to produce valid statistical inferences. In this article we refute this criticism for one of the simplest models where Bayesian inference is not analytically tractable, that is, the Bayesian linear model (for a particular choice of priors). We prove that under mild regularity conditions, VB based estimators enjoy some desirable frequentist properties such as consistency and can be used to obtain asymptotically valid standard errors. In addition to these results we introduce two VB information criteria: the variational Akaike information criterion and the variational Bayesian information criterion. We show that variational Akaike information criterion is asymptotically equivalent to the frequentist Akaike information criterion and that the variational Bayesian information criterion is first order equivalent to the Bayesian information criterion in linear regression. These results motivate the potential use of the variational information criteria for more complex models. We support our theoretical results with numerical examples.     

Testing a linear ARMA model against threshold-ARMA models: A Bayesian approach

We introduce a Bayesian approach to test linear autoregressive moving-average (ARMA) models against threshold autoregressive moving-average (TARMA) models. First, the marginal posterior densities of all parameters, including the threshold and delay, of a TARMA model are obtained by using Gibbs sampler with Metropolis–Hastings algorithm. Second, reversible-jump Markov chain Monte Carlo (RJMCMC) method is adopted to calculate the posterior probabilities for ARMA and TARMA models: Posterior evidence in favor of TARMA models indicates threshold nonlinearity. Finally, based on RJMCMC scheme and Akaike information criterion (AIC) or Bayesian information criterion (BIC), the procedure for modeling TARMA models is exploited. Simulation experiments and a real data example show that our method works well for distinguishing an ARMA from a TARMA model and for building TARMA models.     

Saddlepoint condition on a predictor to reconfirm the need for the assumption of a prior distribution

Saddlepoint conditions on a predictor are introduced and developed to reconfirm the need for the assumption of a prior distribution in constructing a useful inferential procedure. A condition yields that the predictor induced from the maximum likelihood estimator is the worst under a loss, while the predictor induced from a suitable posterior mean is the best. This result indicates the promising role of Bayesian criteria, such as the deviance information criterion (DIC). As an implication, we critique the conventional empirical Bayes method because of its partial assumption of a prior distribution.     

A comparison of Monte Carlo methods for computing marginal likelihoods of item response theory models

Nowadays, Bayesian methods are routinely used for estimating parameters of item response theory (IRT) models. However, the marginal likelihoods are still rarely used for comparing IRT models due to their complexity and a relatively high dimension of the model parameters. In this paper, we review Monte Carlo (MC) methods developed in the literature in recent years and provide a detailed development of how these methods are applied to the IRT models. In particular, we focus on the “best possible” implementation of these MC methods for the IRT models. These MC methods are used to compute the marginal likelihoods under the one-parameter IRT model with the logistic link (1PL model) and the two-parameter logistic IRT model (2PL model) for a real English Examination dataset. We further use the widely applicable information criterion (WAIC) and deviance information criterion (DIC) to compare the 1PL model and the 2PL model. The 2PL model is favored by all of these three Bayesian model comparison criteria for the English Examination data.     

Bayesian gamma frailty models for survival data with semi-competing risks and treatment switching

Motivated from a colorectal cancer study, we propose a class of frailty semi-competing risks survival models to account for the dependence between disease progression time, survival time, and treatment switching. Properties of the proposed models are examined and an efficient Gibbs sampling algorithm using the collapsed Gibbs technique is developed. A Bayesian procedure for assessing the treatment effect is also proposed. The deviance information criterion (DIC) with an appropriate deviance function and Logarithm of the pseudomarginal likelihood (LPML) are constructed for model comparison. A simulation study is conducted to examine the empirical performance of DIC and LPML and as well as the posterior estimates. The proposed method is further applied to analyze data from a colorectal cancer study.     

Forecasting time series of economic processes by model averaging across data frames of various lengths

This paper presents an extension of mean-squared forecast error (MSFE) model averaging for integrating linear regression models computed on data frames of various lengths. Proposed method is considered to be a preferable alternative to best model selection by various efficiency criteria such as Bayesian information criterion (BIC), Akaike information criterion (AIC), F-statistics and mean-squared error (MSE) as well as to Bayesian model averaging (BMA) and naïve simple forecast average. The method is developed to deal with possibly non-nested models having different number of observations and selects forecast weights by minimizing the unbiased estimator of MSFE. Proposed method also yields forecast confidence intervals with a given significance level what is not possible when applying other model averaging methods. In addition, out-of-sample simulation and empirical testing proves efficiency of such kind of averaging when forecasting economic processes.