A nonparametric Bayesian model for inference in related longitudinal studies

Summary.  We discuss a method for combining different but related longitudinal studies to improve predictive precision. The motivation is to borrow strength across clinical studies in which the same measurements are collected at different frequencies. Key features of the data are heterogeneous populations and an unbalanced design across three studies of interest. The first two studies are phase I studies with very detailed observations on a relatively small number of patients. The third study is a large phase III study with over 1500 enrolled patients, but with relatively few measurements on each patient. Patients receive different doses of several drugs in the studies, with the phase III study containing significantly less toxic treatments. Thus, the main challenges for the analysis are to accommodate heterogeneous population distributions and to formalize borrowing strength across the studies and across the various treatment levels. We describe a hierarchical extension over suitable semiparametric longitudinal data models to achieve the inferential goal. A nonparametric random-effects model accommodates the heterogeneity of the population of patients. A hierarchical extension allows borrowing strength across different studies and different levels of treatment by introducing dependence across these nonparametric random-effects distributions. Dependence is introduced by building an analysis of variance (ANOVA) like structure over the random-effects distributions for different studies and treatment combinations. Model structure and parameter interpretation are similar to standard ANOVA models. Instead of the unknown normal means as in standard ANOVA models, however, the basic objects of inference are random distributions, namely the unknown population distributions under each study. The analysis is based on a mixture of Dirichlet processes model as the underlying semiparametric model.     

Bayesian neural tree models for nonparametric regression

Frequentist and Bayesian methods differ in many aspects but share some basic optimal properties. In real-life prediction problems, situations exist in which a model based on one of the above paradigms is preferable depending on some subjective criteria. Nonparametric classification and regression techniques, such as decision trees and neural networks, have both frequentist (classification and regression trees (CARTs) and artificial neural networks) as well as Bayesian counterparts (Bayesian CART and Bayesian neural networks) to learning from data. In this paper, we present two hybrid models combining the Bayesian and frequentist versions of CART and neural networks, which we call the Bayesian neural tree (BNT) models. BNT models can simultaneously perform feature selection and prediction, are highly flexible, and generalise well in settings with limited training observations. We study the statistical consistency of the proposed approaches and derive the optimal value of a vital model parameter. The excellent performance of the newly proposed BNT models is shown using simulation studies. We also provide some illustrative examples using a wide variety of standard regression datasets from a public available machine learning repository to show the superiority of the proposed models in comparison to popularly used Bayesian CART and Bayesian neural network models.     

Inference and prediction with arma processes

An essential ingredient of any time series analysis is the estimation of the model parameters and the forecasting of future observations. This investigation takes a Bayesian approach to the analysis of time series by making inferences of the model parameters from the posterior distribution and forecasting from the predictive distribution.

The foundation of the approach is to approximate the condi-tional likelihood by a normal-gamma distribution on the parameter space. The techniques illustrated with many examples of ARMA processes.     

Bayesian optimal sequential design for nonparametric regression via inhomogeneous evolutionary MCMC

We develop a novel computational methodology for Bayesian optimal sequential design for nonparametric regression. This computational methodology, that we call inhomogeneous evolutionary Markov chain Monte Carlo, combines ideas of simulated annealing, genetic or evolutionary algorithms, and Markov chain Monte Carlo. Our framework allows optimality criteria with general utility functions and general classes of priors for the underlying regression function. We illustrate the usefulness of our novel methodology with applications to experimental design for nonparametric function estimation using Gaussian process priors and free-knot cubic splines priors.     

Geometric stick-breaking processes for continuous-time Bayesian nonparametric modeling

We propose a new class of time dependent random probability measures and show how this can be used for Bayesian nonparametric inference in continuous time. By means of a nonparametric hierarchical model we define a random process with geometric stick-breaking representation and dependence structure induced via a one dimensional diffusion process of Wright-Fisher type. The sequence is shown to be a strongly stationary measure-valued process with continuous sample paths which, despite the simplicity of the weights structure, can be used for inferential purposes on the trajectory of a discretely observed continuous-time phenomenon. A simple estimation procedure is presented and illustrated with simulated and real financial data.     

A note on Bayesian nonparametric regression function estimation

In this note the problem of nonparametric regression function estimation in a random design regression model with Gaussian errors is considered from the Bayesian perspective. It is assumed that the regression function belongs to a class of functions with a known degree of smoothness. A prior distribution on the given class can be induced by a prior on the coefficients in a series expansion of the regression function through an orthonormal system. The rate of convergence of the resulting posterior distribution is employed to provide a measure of the accuracy of the Bayesian estimation procedure defined by the posterior expected regression function. We show that the Bayes’ estimator achieves the optimal minimax rate of convergence under mean integrated squared error over the involved class of regression functions, thus being comparable to other popular frequentist regression estimators.     

Bandwidth selection for local linear regression smoothers

Summary. The paper presents a general strategy for selecting the bandwidth of nonparametric regression estimators and specializes it to local linear regression smoothers. The procedure requires the sample to be divided into a training sample and a testing sample. Using the training sample we first compute a family of regression smoothers indexed by their bandwidths. Next we select the bandwidth by minimizing the empirical quadratic prediction error on the testing sample. The resulting bandwidth satisfies a finite sample oracle inequality which holds for all bounded regression functions. This permits asymptotically optimal estimation for nearly any regression function. The practical performance of the method is illustrated by a simulation study which shows good finite sample behaviour of our method compared with other bandwidth selection procedures.     

Bayesian inference for bisexual galton-watson processes

In this work, an approach to the Bayesian estimation in a bisexual Galton-Watson process is considered. First we study an important parametric case assuming offspring distribution belonging to the bivariate series power family of distributions and then, we continue to investigate the nonparametric case. In both situations, Bayes estimators under weighted squared error loss function, for means, variances and covariance of the off spring distribution are obtained. For the superadditive case, the Bayes estimation of the asymptotic growth rate is also considered. Illustrative examples are given.     

A new Bayesian wavelet thresholding estimator of nonparametric regression

The methods of estimation of nonparametric regression function are quite common in statistical application. In this paper, the new Bayesian wavelet thresholding estimation is considered. The new mixture prior distributions for the estimation of nonparametric regression function by applying wavelet transformation are investigated. The reversible jump algorithm to obtain the appropriate prior distributions and value of thresholding is used. The performance of the proposed estimator is assessed with simulated data from well-known test functions by comparing the convergence rate of the proposed estimator with respect to another by evaluating the average mean square error and standard deviations. Finally by applying the developed method, density function of galaxy data is estimated.     

Decompounding discrete distributions: A nonparametric Bayesian approach

Suppose that a compound Poisson process is observed discretely in time and assume that its jump distribution is supported on the set of natural numbers. In this paper we propose a nonparametric Bayesian approach to estimate the intensity of the underlying Poisson process and the distribution of the jumps. We provide a Markov chain Monte Carlo scheme for obtaining samples from the posterior. We apply our method on both simulated and real data examples, and compare its performance with the frequentist plug-in estimator proposed by Buchmann and Grübel. On a theoretical side, we study the posterior from the frequentist point of view and prove that as the sample size n→∞, it contracts around the “true,” data-generating parameters at rate $1/\sqrt{n}$, up to a $\log n$ factor.     

Bayesian neural networks for nonlinear time series forecasting

In this article, we apply Bayesian neural networks (BNNs) to time series analysis, and propose a Monte Carlo algorithm for BNN training. In addition, we go a step further in BNN model selection by putting a prior on network connections instead of hidden units as done by other authors. This allows us to treat the selection of hidden units and the selection of input variables uniformly. The BNN model is compared to a number of competitors, such as the Box-Jenkins model, bilinear model, threshold autoregressive model, and traditional neural network model, on a number of popular and challenging data sets. Numerical results show that the BNN model has achieved a consistent improvement over the competitors in forecasting future values. Insights on how to improve the generalization ability of BNNs are revealed in many respects of our implementation, such as the selection of input variables, the specification of prior distributions, and the treatment of outliers.     

Some Properties and Generalizations of Non-negative Bayesian Time Series Models

We study the most basic Bayesian forecasting model for exponential family time series, the power steady model (PSM) of Smith, in terms of observable properties of one-step forecast distributions and sample paths. The PSM implies a constraint between location and spread of the forecast distribution. Including a scale parameter in the models does not always give an exact solution free of this problem, but it does suggest how to define related models free of the constraint. We define such a class of models which contains the PSM. We concentrate on the case where observations are non-negative. Probability theory and simulation show that under very mild conditions almost all sample paths of these models converge to some constant, making them unsuitable for modelling in many situations. The results apply more generally to non-negative models defined in terms of exponentially weighted moving averages. We use these and related results to motivate, define and apply very simple models based on directly specifying the forecast distributions.     

Adaptive lifting for nonparametric regression

Many wavelet shrinkage methods assume that the data are observed on an equally spaced grid of length of the form 2^J for some J. These methods require serious modification or preprocessed data to cope with irregularly spaced data. The lifting scheme is a recent mathematical innovation that obtains a multiscale analysis for irregularly spaced data. A key lifting component is the “predict” step where a prediction of a data point is made. The residual from the prediction is stored and can be thought of as a wavelet coefficient. This article exploits the flexibility of lifting by adaptively choosing the kind of prediction according to a criterion. In this way the smoothness of the underlying ‘wavelet’ can be adapted to the local properties of the function. Multiple observations at a point can readily be handled by lifting through a suitable choice of prediction. We adapt existing shrinkage rules to work with our adaptive lifting methods. We use simulation to demonstrate the improved sparsity of our techniques and improved regression performance when compared to both wavelet and non-wavelet methods suitable for irregular data. We also exhibit the benefits of our adaptive lifting on the real inductance plethysmography and motorcycle data.     

The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference

This paper introduces a new class of time-varying, measure-valued stochastic processes for Bayesian nonparametric inference. The class of priors is constructed by normalising a stochastic process derived from non-Gaussian Ornstein-Uhlenbeck processes and generalises the class of normalised random measures with independent increments from static problems. Some properties of the normalised measure are investigated. A particle filter and MCMC schemes are described for inference. The methods are applied to an example in the modelling of financial data.     

On nonparametric density estimation for multivariate linear long-memory processes

We consider nonparametric estimation of the density function and its derivatives for multivariate linear processes with long-range dependence. In a first step, the asymptotic distribution of the multivariate empirical process is derived. In a second step, the asymptotic distribution of kernel density estimators and their derivatives is obtained.     

Testing heteroscedasticity in nonparametric regression

The importance of being able to detect heteroscedasticity in regression is widely recognized because efficient inference for the regression function requires that heteroscedasticity is taken into account. In this paper a simple consistent test for heteroscedasticity is proposed in a nonparametric regression set-up. The test is based on an estimator for the best L ²-approximation of the variance function by a constant. Under mild assumptions asymptotic normality of the corresponding test statistic is established even under arbitrary fixed alternatives. Confidence intervals are obtained for a corresponding measure of heteroscedasticity. The finite sample performance and robustness of these procedures are investigated in a simulation study and Box-type corrections are suggested for small sample sizes.     

A Bayesian nonparametric causal model

Typically, in the practice of causal inference from observational studies, a parametric model is assumed for the joint population density of potential outcomes and treatment assignments, and possibly this is accompanied by the assumption of no hidden bias. However, both assumptions are questionable for real data, the accuracy of causal inference is compromised when the data violates either assumption, and the parametric assumption precludes capturing a more general range of density shapes (e.g., heavier tail behavior and possible multi-modalities). We introduce a flexible, Bayesian nonparametric causal model to provide more accurate causal inferences. The model makes use of a stick-breaking prior, which has the flexibility to capture any multi-modalities, skewness and heavier tail behavior in this joint population density, while accounting for hidden bias. We prove the asymptotic consistency of the posterior distribution of the model, and illustrate our causal model through the analysis of small and large observational data sets.     

MOSUM monitoring for variance change in nonparametric regression models

ABSTRACT

This article considers the monitoring for variance change in nonparametric regression models. First, the local linear estimator of the regression function is given. A moving square cumulative sum procedure is proposed based on residuals of the estimator. And the asymptotic results of the statistic under the null hypothesis and the alternative hypothesis are obtained. Simulations and Application support our procedure.     

Bayesian prediction bounds for the pareto lifetime model

Suppose that the length of time in years for which a business operates until failure has a Pareto distribution. Let x₁ ≤ x₂ x₃ ≤…≤z_k denote the survival lifetimes of the first k of a random sample of n businesses. Bayesian predictions are to be made on the ordered failure times of t h e remaining (n-k) businesses, using the conditional probability density function. Examples are given to illustrate our results.     

Optimal U-type design for Bayesian nonparametric multiresponse prediction

This paper presents an extension of the work of Yue and Chatterjee (2010) about U-type designs for Bayesian nonparametric response prediction. We consider nonparametric Bayesian regression model with p responses. We use U-type designs with n runs, m factors and q levels for the nonparametric multiresponse prediction based on the asymptotic Bayesian criterion. A lower bound for the proposed criterion is established, and some optimal and nearly optimal designs for the illustrative models are given.