Bayesian Analysis of Instrumental Variable Models: Acceptance-Rejection within Direct Monte Carlo

We discuss Bayesian inferential procedures within the family of instrumental variables regression models and focus on two issues: existence conditions for posterior moments of the parameters of interest under a flat prior and the potential of Direct Monte Carlo (DMC) approaches for efficient evaluation of such possibly highly non-elliptical posteriors. We show that, for the general case of m endogenous variables under a flat prior, posterior moments of order r exist for the coefficients reflecting the endogenous regressors’ effect on the dependent variable, if the number of instruments is greater than m +r, even though there is an issue of local non-identification that causes non-elliptical shapes of the posterior. This stresses the need for efficient Monte Carlo integration methods. We introduce an extension of DMC that incorporates an acceptance-rejection sampling step within DMC. This Acceptance-Rejection within Direct Monte Carlo (ARDMC) method has the attractive property that the generated random drawings are independent, which greatly helps the fast convergence of simulation results, and which facilitates the evaluation of the numerical accuracy. The speed of ARDMC can be easily further improved by making use of parallelized computation using multiple core machines or computer clusters. We note that ARDMC is an analogue to the well-known “Metropolis-Hastings within Gibbs” sampling in the sense that one ‘more difficult’ step is used within an ‘easier’ simulation method. We compare the ARDMC approach with the Gibbs sampler using simulated data and two empirical data sets, involving the settler mortality instrument of Acemoglu et al. (2001 Acemoglu , D. , Johnson , S. , Robinson , J. A. ( 2001 ). The colonial origins of comparative development: An empirical investigation . American Economic Review 91 ( 5 ): 1369 – 1401 .[Crossref], [Web of Science ®] , [Google Scholar]) and father's education's instrument used by Hoogerheide et al. (2012a Hoogerheide , L. F. , Block , J. H. , Thurik , A. R. (2012a). Family background variables as instruments for education in income regressions: A Bayesian analysis. Economics of Education Review 31(5):515–523.[Crossref], [Web of Science ®] , [Google Scholar]). Even without making use of parallelized computation, an efficiency gain is observed both under strong and weak instruments, where the gain can be enormous in the latter case.     

Bayesian and robust Bayesian analysis in a general setting

In this paper we introduce a broad family of loss functions based on the concept of Bregman divergence. We deal with both Bayesian estimation and prediction problems and show that all Bayes solutions associated with loss functions belonging to the introduced family of losses satisfy the same equation. We further concentrate on the concept of robust Bayesian analysis and provide one equation that explicitly leads to robust Bayes solutions. The results are model-free and include many existing results in Bayesian and robust Bayesian contexts in the literature.     

Bayesian modelling of catch in a north-west Atlantic fishery

Summary. We model daily catches of fishing boats in the Grand Bank fishing grounds. We use data on catches per species for a number of vessels collected by the European Union in the context of the Northwest Atlantic Fisheries Organization. Many variables can be thought to influence the amount caught: a number of ship characteristics (such as the size of the ship, the fishing technique used and the mesh size of the nets) are obvious candidates, but one can also consider the season or the actual location of the catch. Our database leads to 28 possible regressors (arising from six continuous variables and four categorical variables, whose 22 levels are treated separately), resulting in a set of 177 million possible linear regression models for the log-catch. Zero observations are modelled separately through a probit model. Inference is based on Bayesian model averaging, using a Markov chain Monte Carlo approach. Particular attention is paid to the prediction of catches for single and aggregated ships.     

Bayesian Analysis of ARA Imperfect Repair Models

This article proposes a Bayesian analysis of a class of imperfect repair models, the ARA models. The choice of prior distributions and the computation of posterior distributions are discussed. The presentation is unified for all ARA models and many kinds of possible priors. A numerical study on the quality of the Bayesian estimators is presented, as well as a comparison with the maximum likelihood estimators. Finally, the approach is applied to a real data set.     

Bayesian Inference and Prediction Analysis of the Power Law Process Based on a Gamma Prior Distribution

Under a Gamma prior distribution, the importance sampling (IS) technique is applied to the Bayesian analysis of the Power Law Process (PLP). Samples of important parameters in the PLP are obtained from IS. Based on these samples, not only the posterior analyses of parameters and some functions of the parameter in the PLP can be performed conveniently, but also single-sample and two-sample predictions are constructed easily by the transformation formula of double integral. The sensitivity of the posterior mean of the parameter functions in the PLP is studied with respect to the prior moments in the Gamma prior distribution, and it can guide the selections of the prior moments. After some numerical experiments illustrate the rationality and feasibility of the proposed methods, an engineering example demonstrates its application.     

Pareto Regression: A Bayesian Analysis

Abstract

In this article, a new model is presented that is based on the Pareto distribution of the second kind, when the location parameter depends on covariates as well as unobserved heterogeneity. Bayesian analysis of the model can be performed using Markov Chain Monte Carlo techniques. The new procedures are illustrated in the context of artificial data as well as international output data.     

On the Robustness of Bayesian Modelling of Location and Scale Structures Using Heavy-Tailed Distributions

For solving conflicting information between data and prior distributions, Bayesian modelling with heavy-tailed distributions is applied. Exploiting properties of regularly varying functions and distribution functions as well as their relationship with the finiteness of the moments, we establish results for both location and shape parameter structures. And, as a side result, rates of convergence are derived.     

Interval-censored data with misclassification: a Bayesian approach

Survival data involving silent events are often subject to interval censoring (the event is known to occur within a time interval) and classification errors if a test with no perfect sensitivity and specificity is applied. Considering the nature of this data plays an important role in estimating the time distribution until the occurrence of the event. In this context, we incorporate validation subsets into the parametric proportional hazard model, and show that this additional data, combined with Bayesian inference, compensate the lack of knowledge about test sensitivity and specificity improving the parameter estimates. The proposed model is evaluated through simulation studies, and Bayesian analysis is conducted within a Gibbs sampling procedure. The posterior estimates obtained under validation subset models present lower bias and standard deviation compared to the scenario with no validation subset or the model that assumes perfect sensitivity and specificity. Finally, we illustrate the usefulness of the new methodology with an analysis of real data about HIV acquisition in female sex workers that have been discussed in the literature.     

Bayesian analysis of a multiple-recapture model

In the Bayesian analysis of a multiple-recapture census, different diffuse prior distributions can lead to markedly different inferences about the population size N. Through consideration of the Fisher information matrix it is shown that the number of captures in each sample typically provides little information about N. This suggests that if there is no prior information about capture probabilities, then knowledge of just the sample sizes and not the number of recaptures should leave the distribution of Nunchanged. A prior model that has this property is identified and the posterior distribution is examined. In particular, asymptotic estimates of the posterior mean and variance are derived. Differences between Bayesian and classical point and interval estimators are illustrated through examples.     

On a theorem of Stein relating Bayesian and classical inferences in group models

Let a group G act on the sample space. This paper gives another proof of a theorem of Stein relating a group invariant family of posterior Bayesian probability regions to classical confidence regions when an appropriate prior is used. The example of the central multivariate normal distribution is discussed.     

Bayesian inference for a flexible class of bivariate beta distributions

Several bivariate beta distributions have been proposed in the literature. In particular, Olkin and Liu [A bivariate beta distribution. Statist Probab Lett. 2003;62(4):407–412] proposed a 3 parameter bivariate beta model which Arnold and Ng [Flexible bivariate beta distributions. J Multivariate Anal. 2011;102(8):1194–1202] extend to 5 and 8 parameter models. The 3 parameter model allows for only positive correlation, while the latter models can accommodate both positive and negative correlation. However, these come at the expense of a density that is mathematically intractable. The focus of this research is on Bayesian estimation for the 5 and 8 parameter models. Since the likelihood does not exist in closed form, we apply approximate Bayesian computation, a likelihood free approach. Simulation studies have been carried out for the 5 and 8 parameter cases under various priors and tolerance levels. We apply the 5 parameter model to a real data set by allowing the model to serve as a prior to correlated proportions of a bivariate beta binomial model. Results and comparisons are then discussed.     

A Sequential Point Process Model and Bayesian Inference for Spatial Point Patterns with Linear Structures

Abstract. We introduce a flexible spatial point process model for spatial point patterns exhibiting linear structures, without incorporating a latent line process. The model is given by an underlying sequential point process model. Under this model, the points can be of one of three types: a ‘background point’ an ‘independent cluster point’ or a ‘dependent cluster point’. The background and independent cluster points are thought to exhibit ‘complete spatial randomness’, whereas the dependent cluster points are likely to occur close to previous cluster points. We demonstrate the flexibility of the model for producing point patterns with linear structures and propose to use the model as the likelihood in a Bayesian setting when analysing a spatial point pattern exhibiting linear structures. We illustrate this methodology by analysing two spatial point pattern datasets (locations of bronze age graves in Denmark and locations of mountain tops in Spain).     

Bayesian Inference for the Jump-Diffusion Model with M Jumps

In this article, we propose a new class of models—jump-diffusion models with M jumps (JD(M)J). These structures generalize the discretized arithmetic Brownian motion (for logarithmic rates of return) and the Bernoulli jump-diffusion model. The aim of this article is to present Bayesian tools for estimation and comparison of JD(M)J models. Presented methodology is illustrated with two empirical studies, employing both simulated and real-world data (the S&P100 Index).     

Numerical Bayesian inference with arbitrary prior

The purpose of the paper, is to explain how recent advances in Markov Chain Monte Carlo integration can facilitate the routine Bayesian analysis of the linear model when the prior distribution is completely user dependent. The method is based on a Metropolis-Hastings algorithm with a Student-t source distribution that can generate posterior moments as well as marginal posterior densities for model parameters. The method is illustrated with numerical examples where the combination of prior and likelihood information leads to multimodal posteriors due to prior-likelihood conflicts, and to cases where prior information can be summarized by symmetric stable Paretian distributions.     

Bayesian Analysis for a Redundant Repairable System with Imperfect Coverage

System characteristics of a redundant repairable system with two primary units and one standby are studied from a Bayesian viewpoint with different types of priors assumed for unknown parameters, in which the coverage factor is the same for an operating unit failure as that for a standby unit failure. Times to failure and times to repair of the operating and standby units are assumed to follow exponential distributions. When times to failure and times to repair with uncertain parameters, a Bayesian approach is adopted to evaluate system characteristics. Monte Carlo simulation is used to derive the posterior distribution for the mean time to system failure and the steady-state availability. Some numerical experiments are performed to illustrate the results derived in this paper.     

Non parametric Bayesian analysis of the two-sample problem with censoring

Testing for differences between two groups is a fundamental problem in statistics, and due to developments in Bayesian non parametrics and semiparametrics there has been renewed interest in approaches to this problem. Here we describe a new approach to developing such tests and introduce a class of such tests that take advantage of developments in Bayesian non parametric computing. This class of tests uses the connection between the Dirichlet process (DP) prior and the Wilcoxon rank sum test but extends this idea to the DP mixture prior. Here tests are developed that have appropriate frequentist sampling procedures for large samples but have the potential to outperform the usual frequentist tests. Extensions to interval and right censoring are considered and an application to a high-dimensional data set obtained from an RNA-Seq investigation demonstrates the practical utility of the method.     

Bayesian Inference in Dynamic Disequilibrium Models: An Application to the Polish Credit Market

We propose a Bayesian approach for inference in a dynamic disequilibrium model. To circumvent the difficulties raised by the Maddala and Nelson (1974 Maddala , G. , Nelson , F. ( 1974 ). Maximum likelihood methods for models of markets in disequilibrium . Econometrica 42 ( 6 ): 1013 – 1030 .[Crossref], [Web of Science ®] , [Google Scholar]) specification in the dynamic case, we analyze a dynamic extended version of the disequilibrium model of Ginsburgh et al. (1980 Ginsburgh , V. , Tishler , A. , Zang , I. ( 1980 ). Alternative estimation methods for two regime models . European Economic Review 13 : 207 – 228 .[Crossref], [Web of Science ®] , [Google Scholar]). We develop a Gibbs sampler based on the simulation of the missing observations. The feasibility of the approach is illustrated by an empirical analysis of the Polish credit market, for which we conduct a specification search using the posterior deviance criterion of Spiegelhalter et al. (2002 Spiegelhalter , D. J. , Best , N. G. , Carlin , B. P. , Van Der Linde , A. ( 2002 ). Bayesian measures of complexity and fit . Journal of the Royal Statistical Society, Ser. B 64 : 583 – 639 .[Crossref] , [Google Scholar]).     

Bayesian Inference in Dynamic Disequilibrium Models: An Application to the Polish Credit Market

We propose a Bayesian approach for inference in a dynamic disequilibrium model. To circumvent the difficulties raised by the Maddala and Nelson (1974) specification in the dynamic case, we analyze a dynamic extended version of the disequilibrium model of Ginsburgh et al. (1980). We develop a Gibbs sampler based on the simulation of the missing observations. The feasibility of the approach is illustrated by an empirical analysis of the Polish credit market, for which we conduct a specification search using the posterior deviance criterion of Spiegelhalter et al. (2002).     

Inference in hybrid Bayesian networks using dynamic discretization

We consider approximate inference in hybrid Bayesian Networks (BNs) and present a new iterative algorithm that efficiently combines dynamic discretization with robust propagation algorithms on junction trees. Our approach offers a significant extension to Bayesian Network theory and practice by offering a flexible way of modeling continuous nodes in BNs conditioned on complex configurations of evidence and intermixed with discrete nodes as both parents and children of continuous nodes. Our algorithm is implemented in a commercial Bayesian Network software package, AgenaRisk, which allows model construction and testing to be carried out easily. The results from the empirical trials clearly show how our software can deal effectively with different type of hybrid models containing elements of expert judgment as well as statistical inference. In particular, we show how the rapid convergence of the algorithm towards zones of high probability density, make robust inference analysis possible even in situations where, due to the lack of information in both prior and data, robust sampling becomes unfeasible.     

Estimation across multiple models with application to Bayesian computing and software development

Statistical models are sometimes incorporated into computer software for making predictions about future observations. When the computer model consists of a single statistical model this corresponds to estimation of a function of the model parameters. This paper is concerned with the case that the computer model implements multiple, individually-estimated statistical sub-models. This case frequently arises, for example, in models for medical decision making that derive parameter information from multiple clinical studies. We develop a method for calculating the posterior mean of a function of the parameter vectors of multiple statistical models that is easy to implement in computer software, has high asymptotic accuracy, and has a computational cost linear in the total number of model parameters. The formula is then used to derive a general result about posterior estimation across multiple models. The utility of the results is illustrated by application to clinical software that estimates the risk of fatal coronary disease in people with diabetes.