The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference

This paper introduces a new class of time-varying, measure-valued stochastic processes for Bayesian nonparametric inference. The class of priors is constructed by normalising a stochastic process derived from non-Gaussian Ornstein-Uhlenbeck processes and generalises the class of normalised random measures with independent increments from static problems. Some properties of the normalised measure are investigated. A particle filter and MCMC schemes are described for inference. The methods are applied to an example in the modelling of financial data.     

Posterior Analysis for Normalized Random Measures with Independent Increments

Abstract.  One of the main research areas in Bayesian Nonparametrics is the proposal and study of priors which generalize the Dirichlet process. In this paper, we provide a comprehensive Bayesian non-parametric analysis of random probabilities which are obtained by normalizing random measures with independent increments (NRMI). Special cases of these priors have already shown to be useful for statistical applications such as mixture models and species sampling problems. However, in order to fully exploit these priors, the derivation of the posterior distribution of NRMIs is crucial: here we achieve this goal and, indeed, provide explicit and tractable expressions suitable for practical implementation. The posterior distribution of an NRMI turns out to be a mixture with respect to the distribution of a specific latent variable. The analysis is completed by the derivation of the corresponding predictive distributions and by a thorough investigation of the marginal structure. These results allow to derive a generalized Blackwell–MacQueen sampling scheme, which is then adapted to cover also mixture models driven by general NRMIs.     

A class of neutral to the right priors induced by superposition of beta processes

A random distribution function on the positive real line which belongs to the class of neutral to the right priors is defined. It corresponds to the superposition of independent beta processes at the cumulative hazard level. The definition is constructive and starts with a discrete time process with random probability masses obtained from suitably defined products of independent beta random variables. The continuous time version is derived as the corresponding infinitesimal weak limit and is described in terms of completely random measures. It takes the interpretation of the survival distribution resulting from independent competing failure times. We discuss prior specification and illustrate posterior inference on a real data example.     

Nonparametric Bayesian modelling using skewed Dirichlet processes

We introduce a new class of discrete random probability measures that extend the definition of Dirichlet process (DP) by explicitly incorporating skewness. The asymmetry is controlled by a single parameter in such a way that symmetric DPs are obtained as a special case of the general construction. We review the main properties of skewed DPs and develop appropriate Polya urn schemes. We illustrate the modelling in the context of linear regression models of the capital asset pricing model (CAPM) type, where assessing symmetry for the error distribution is important to check validity of the model.     

On a Class of Random Probability Measures with General Predictive Structure

Abstract. In this study, we investigate a recently introduced class of non‐parametric priors, termed generalized Dirichlet process priors. Such priors induce (exchangeable random) partitions that are characterized by a more elaborate clustering structure than those arising from other widely used priors. A natural area of application of these random probability measures is represented by species sampling problems and, in particular, prediction problems in genomics. To this end, we study both the distribution of the number of distinct species present in a sample and the distribution of the number of new species conditionally on an observed sample. We also provide the Bayesian Non‐parametric estimator for the number of new species in an additional sample of given size and for the discovery probability as function of the size of the additional sample. Finally, the study of its conditional structure is completed by the determination of the posterior distribution.     

A Monte Carlo Markov chain algorithm for a class of mixture time series models

This article generalizes the Monte Carlo Markov Chain (MCMC) algorithm, based on the Gibbs weighted Chinese restaurant (gWCR) process algorithm, for a class of kernel mixture of time series models over the Dirichlet process. This class of models is an extension of Lo’s (Ann. Stat. 12:351–357, 1984) kernel mixture model for independent observations. The kernel represents a known distribution of time series conditional on past time series and both present and past latent variables. The latent variables are independent samples from a Dirichlet process, which is a random discrete (almost surely) distribution. This class of models includes an infinite mixture of autoregressive processes and an infinite mixture of generalized autoregressive conditional heteroskedasticity (GARCH) processes.     

Sequential Monte Carlo methods for mixtures with normalized random measures with independent increments priors

Normalized random measures with independent increments are a general, tractable class of nonparametric prior. This paper describes sequential Monte Carlo methods for both conjugate and non-conjugate nonparametric mixture models with these priors. A simulation study is used to compare the efficiency of the different algorithms for density estimation and comparisons made with Markov chain Monte Carlo methods. The SMC methods are further illustrated by applications to dynamically fitting a nonparametric stochastic volatility model and to estimation of the marginal likelihood in a goodness-of-fit testing example.     

Hybrid Dirichlet mixture models for functional data

Summary.  In functional data analysis, curves or surfaces are observed, up to measurement error, at a finite set of locations, for, say, a sample of n individuals. Often, the curves are homogeneous, except perhaps for individual-specific regions that provide heterogeneous behaviour (e.g. 'damaged' areas of irregular shape on an otherwise smooth surface). Motivated by applications with functional data of this nature, we propose a Bayesian mixture model, with the aim of dimension reduction, by representing the sample of n curves through a smaller set of canonical curves. We propose a novel prior on the space of probability measures for a random curve which extends the popular Dirichlet priors by allowing local clustering: non-homogeneous portions of a curve can be allocated to different clusters and the n individual curves can be represented as recombinations (hybrids) of a few canonical curves. More precisely, the prior proposed envisions a conceptual hidden factor with k -levels that acts locally on each curve. We discuss several models incorporating this prior and illustrate its performance with simulated and real data sets. We examine theoretical properties of the proposed finite hybrid Dirichlet mixtures, specifically, their behaviour as the number of the mixture components goes to ∞ and their connection with Dirichlet process mixtures.     

Modelling failures times with dependent renewal type models via exchangeability

Failure times of a machinery cannot always be assumed independent and identically distributed, eg, if after reparations the machinery is not restored to a same-as-new condition. Framed within the renewal processes approach, a generalization that considers exchangeable inter-arrival times is presented. The resulting model provides a more realistic approach to capture the dependence among events occurring at random times, while retaining much of the tractability of the classical renewal process. Extensions of some classical results and special cases of renewal functions are analysed, in particular, the one corresponding to an exchangeable sequence driven by a Dirichlet process. The proposal is tested through an estimation procedure using simulated data sets and with an application to the reliability of hydraulic subsystems in load-haul-dump machines.     

On Some Optimal Bayesian Nonparametric Rules for Estimating Distribution Functions

In this paper, we present a novel approach to estimating distribution functions, which combines ideas from Bayesian nonparametric inference, decision theory and robustness. Given a sample from a Dirichlet process on the space (𝒳, A), with parameter η in a class of measures, the sampling distribution function is estimated according to some optimality criteria (mainly minimax and regret), when a quadratic loss function is assumed. Estimates are then compared in two examples: one with simulated data and one with gas escapes data in a city network.     

Exact and approximate sum representations for the Dirichlet process

The Dirichlet process can be regarded as a random probability measure for which the authors examine various sum representations. They consider in particular the gamma process construction of Ferguson (1973) and the “stick‐breaking” construction of Sethuraman (1994). They propose a Dirichlet finite sum representation that strongly approximates the Dirichlet process. They assess the accuracy of this approximation and characterize the posterior that this new prior leads to in the context of Bayesian nonpara‐metric hierarchical models.     

Geometric stick-breaking processes for continuous-time Bayesian nonparametric modeling

We propose a new class of time dependent random probability measures and show how this can be used for Bayesian nonparametric inference in continuous time. By means of a nonparametric hierarchical model we define a random process with geometric stick-breaking representation and dependence structure induced via a one dimensional diffusion process of Wright-Fisher type. The sequence is shown to be a strongly stationary measure-valued process with continuous sample paths which, despite the simplicity of the weights structure, can be used for inferential purposes on the trajectory of a discretely observed continuous-time phenomenon. A simple estimation procedure is presented and illustrated with simulated and real financial data.     

Rubbery Polya Tree

Abstract. Polya trees (PT) are random probability measures which can assign probability 1 to the set of continuous distributions for certain specifications of the hyperparameters. This feature distinguishes the PT from the popular Dirichlet process (DP) model which assigns probability 1 to the set of discrete distributions. However, the PT is not nearly as widely used as the DP prior. Probably the main reason is an awkward dependence of posterior inference on the choice of the partitioning subsets in the definition of the PT. We propose a generalization of the PT prior that mitigates this undesirable dependence on the partition structure, by allowing the branching probabilities to be dependent within the same level. The proposed new process is not a PT anymore. However, it is still a tail‐free process and many of the prior properties remain the same as those for the PT.     

Flexible clustering via hidden hierarchical Dirichlet priors

The Bayesian approach to inference stands out for naturally allowing borrowing information across heterogeneous populations, with different samples possibly sharing the same distribution. A popular Bayesian nonparametric model for clustering probability distributions is the nested Dirichlet process, which however has the drawback of grouping distributions in a single cluster when ties are observed across samples. With the goal of achieving a flexible and effective clustering method for both samples and observations, we investigate a nonparametric prior that arises as the composition of two different discrete random structures and derive a closed-form expression for the induced distribution of the random partition, the fundamental tool regulating the clustering behavior of the model. On the one hand, this allows to gain a deeper insight into the theoretical properties of the model and, on the other hand, it yields an MCMC algorithm for evaluating Bayesian inferences of interest. Moreover, we single out limitations of this algorithm when working with more than two populations and, consequently, devise an alternative more efficient sampling scheme, which as a by-product, allows testing homogeneity between different populations. Finally, we perform a comparison with the nested Dirichlet process and provide illustrative examples of both synthetic and real data.     

Semiparametric Bayesian classification with longitudinal markers

Summary.  We analyse data from a study involving 173 pregnant women. The data are observed values of the β human chorionic gonadotropin hormone measured during the first 80 days of gestational age, including from one up to six longitudinal responses for each woman. The main objective in this study is to predict normal versus abnormal pregnancy outcomes from data that are available at the early stages of pregnancy. We achieve the desired classification with a semiparametric hierarchical model. Specifically, we consider a Dirichlet process mixture prior for the distribution of the random effects in each group. The unknown random-effects distributions are allowed to vary across groups but are made dependent by using a design vector to select different features of a single underlying random probability measure. The resulting model is an extension of the dependent Dirichlet process model, with an additional probability model for group classification. The model is shown to perform better than an alternative model which is based on independent Dirichlet processes for the groups. Relevant posterior distributions are summarized by using Markov chain Monte Carlo methods.     

Estimation of regression vectors in linear mixed models with Dirichlet process random effects

The Dirichlet process has been used extensively in Bayesian non parametric modeling, and has proven to be very useful. In particular, mixed models with Dirichlet process random effects have been used in modeling many types of data and can often outperform their normal random effect counterparts. Here we examine the linear mixed model with Dirichlet process random effects from a classical view, and derive the best linear unbiased estimator (BLUE) of the fixed effects. We are also able to calculate the resulting covariance matrix and find that the covariance is directly related to the precision parameter of the Dirichlet process, giving a new interpretation of this parameter. We also characterize the relationship between the BLUE and the ordinary least-squares (OLS) estimator and show how confidence intervals can be approximated.     

Bayes and empirical bayes estimation of the probability that z > x + y

Let X, Y and Z be independent random variables with common unknown distribution F. Using the Dirichlet process prior for F and squared erro loss function, the Bayes and empirical Bayes estimators of the parameters λ(F). the probability that Z > X + Y, are derived. The limiting Bayes estimator of λ(F) under some conditions on the parameter of the process is shown to be asymptotically normal. The aysmptotic optimality of the empirical Bayes estimator of λ(F) is established. When X, Y and Z have support on the positive real line, these results are derived for randomly right censored data. This problem relates to testing whether than used discussed by Hollander and Proshcan (1972) and Chen, Hollander and Langberg (1983).     

Convergence and Divergence of Random Series

This paper amplifies Daley's (1981) criteria for absolute convergence of certain random series by providing a sufficient condition which also is necessary if the summands are independent. Conditions for unconditional and conditional convergence are also given. These results are used to obtain a substantially complete picture of the behaviour of random Dirichlet series of a fairly general type. Behaviour of the partial sums of divergent series is discussed, with particular attention to Dirichlet series.     

On a score test for pure birth processes and a poisson dispersion test

We consider a generalization of a standard test for overdispersion (underdispersion) of possibly Poison data. Under the null hypothesis observed counts are increments of Poisson processes. Particular applications are toa random sample of identically distributed processes and a single observed process. The test has intuitive appeal beyond the specific alternatives considered.