Bayesian nonparametric estimation of a copula

A copula can fully characterize the dependence of multiple variables. The purpose of this paper is to provide a Bayesian nonparametric approach to the estimation of a copula, and we do this by mixing over a class of parametric copulas. In particular, we show that any bivariate copula density can be arbitrarily accurately approximated by an infinite mixture of Gaussian copula density functions. The model can be estimated by Markov Chain Monte Carlo methods and the model is demonstrated on both simulated and real data sets.     

Nonparametric hierarchical Bayes analysis of binomial data via Bernstein polynomial priors

For binomial data analysis, many methods based on empirical Bayes interpretations have been developed, in which a variance‐stabilizing transformation and a normality assumption are usually required. To achieve the greatest model flexibility, we conduct nonparametric Bayesian inference for binomial data and employ a special nonparametric Bayesian prior—the Bernstein–Dirichlet process (BDP)—in the hierarchical Bayes model for the data. The BDP is a special Dirichlet process (DP) mixture based on beta distributions, and the posterior distribution resulting from it has a smooth density defined on [0, 1]. We examine two Markov chain Monte Carlo procedures for simulating from the resulting posterior distribution, and compare their convergence rates and computational efficiency. In contrast to existing results for posterior consistency based on direct observations, the posterior consistency of the BDP, given indirect binomial data, is established. We study shrinkage effects and the robustness of the BDP‐based posterior estimators in comparison with several other empirical and hierarchical Bayes estimators, and we illustrate through examples that the BDP‐based nonparametric Bayesian estimate is more robust to the sample variation and tends to have a smaller estimation error than those based on the DP prior. In certain settings, the new estimator can also beat Stein's estimator, Efron and Morris's limited‐translation estimator, and many other existing empirical Bayes estimators. The Canadian Journal of Statistics 40: 328–344; 2012 © 2012 Statistical Society of Canada     

Adaptive Bayesian Procedures Using Random Series Priors

We consider a general class of prior distributions for nonparametric Bayesian estimation which uses finite random series with a random number of terms. A prior is constructed through distributions on the number of basis functions and the associated coefficients. We derive a general result on adaptive posterior contraction rates for all smoothness levels of the target function in the true model by constructing an appropriate ‘sieve’ and applying the general theory of posterior contraction rates. We apply this general result on several statistical problems such as density estimation, various nonparametric regressions, classification, spectral density estimation and functional regression. The prior can be viewed as an alternative to the commonly used Gaussian process prior, but properties of the posterior distribution can be analysed by relatively simpler techniques. An interesting approximation property of B‐spline basis expansion established in this paper allows a canonical choice of prior on coefficients in a random series and allows a simple computational approach without using Markov chain Monte Carlo methods. A simulation study is conducted to show that the accuracy of the Bayesian estimators based on the random series prior and the Gaussian process prior are comparable. We apply the method on Tecator data using functional regression models.     

A bayesian estimator for the dependence function of a bivariate extreme‐value distribution

Any continuous bivariate distribution can be expressed in terms of its margins and a unique copula. In the case of extreme‐value distributions, the copula is characterized by a dependence function while each margin depends on three parameters. The authors propose a Bayesian approach for the simultaneous estimation of the dependence function and the parameters defining the margins. They describe a nonparametric model for the dependence function and a reversible jump Markov chain Monte Carlo algorithm for the computation of the Bayesian estimator. They show through simulations that their estimator has a smaller mean integrated squared error than classical nonparametric estimators, especially in small samples. They illustrate their approach on a hydrological data set.     

A note on Bayesian nonparametric regression function estimation

In this note the problem of nonparametric regression function estimation in a random design regression model with Gaussian errors is considered from the Bayesian perspective. It is assumed that the regression function belongs to a class of functions with a known degree of smoothness. A prior distribution on the given class can be induced by a prior on the coefficients in a series expansion of the regression function through an orthonormal system. The rate of convergence of the resulting posterior distribution is employed to provide a measure of the accuracy of the Bayesian estimation procedure defined by the posterior expected regression function. We show that the Bayes’ estimator achieves the optimal minimax rate of convergence under mean integrated squared error over the involved class of regression functions, thus being comparable to other popular frequentist regression estimators.     

Modelling udder infection data using copula models for quadruples

We study copula models for correlated infection times in the four udder quarters of dairy cows. Both a semi-parametric and a nonparametric approach are considered to estimate the marginal survival functions, taking into account the effect of a binary udder quarter level covariate. We use a two-stage estimation approach and we briefly discuss the asymptotic behaviour of the estimators obtained in the first and the second stage of the estimation. A pseudo-likelihood ratio test is used to select an appropriate copula from the power variance copula family that describes the association between the outcomes in a cluster. We propose a new bootstrap algorithm to obtain the p-value for this test. This bootstrap algorithm also provides estimates for the standard errors of the estimated parameters in the copula. The proposed methods are applied to the udder infection data. A small simulation study for a setting similar to the setting of the udder infection data gives evidence that the proposed method provides a valid approach to select an appropriate copula within the power variance copula family.     

Bayesian nonparametric modelling of the link function in the single-index model using a Bernstein–Dirichlet process prior

This paper proposes the use of the Bernstein–Dirichlet process prior for a new nonparametric approach to estimating the link function in the single-index model (SIM). The Bernstein–Dirichlet process prior has so far mainly been used for nonparametric density estimation. Here we modify this approach to allow for an approximation of the unknown link function. Instead of the usual Gaussian distribution, the error term is assumed to be asymmetric Laplace distributed which increases the flexibility and robustness of the SIM. To automatically identify truly active predictors, spike-and-slab priors are used for Bayesian variable selection. Posterior computations are performed via a Metropolis-Hastings-within-Gibbs sampler using a truncation-based algorithm for stick-breaking priors. We compare the efficiency of the proposed approach with well-established techniques in an extensive simulation study and illustrate its practical performance by an application to nonparametric modelling of the power consumption in a sewage treatment plant.     

Bayesian clustering and product partition models

Summary. We present a decision theoretic formulation of product partition models (PPMs) that allows a formal treatment of different decision problems such as estimation or hypothesis testing and clustering methods simultaneously. A key observation in our construction is the fact that PPMs can be formulated in the context of model selection. The underlying partition structure in these models is closely related to that arising in connection with Dirichlet processes. This allows a straightforward adaptation of some computational strategies—originally devised for nonparametric Bayesian problems—to our framework. The resulting algorithms are more flexible than other competing alternatives that are used for problems involving PPMs. We propose an algorithm that yields Bayes estimates of the quantities of interest and the groups of experimental units. We explore the application of our methods to the detection of outliers in normal and Student t regression models, with clustering structure equivalent to that induced by a Dirichlet process prior. We also discuss the sensitivity of the results considering different prior distributions for the partitions.     

Bayesian nonparametric survival analysis using mixture of Burr XII distributions

ABSTRACT

Recently, the Bayesian nonparametric approaches in survival studies attract much more attentions. Because of multimodality in survival data, the mixture models are very common. We introduce a Bayesian nonparametric mixture model with Burr distribution (Burr type XII) as the kernel. Since the Burr distribution shares good properties of common distributions on survival analysis, it has more flexibility than other distributions. By applying this model to simulated and real failure time datasets, we show the preference of this model and compare it with Dirichlet process mixture models with different kernels. The Markov chain Monte Carlo (MCMC) simulation methods to calculate the posterior distribution are used.     

Positive quadrant dependence testing and constrained copula estimation

Positive quadrant dependence is a specific dependence structure that is of practical importance in for example modelling dependencies in insurance and actuarial sciences. This dependence structure imposes a constraint on the copula function. The interest in this paper is to test for positive quadrant dependence. One way to assess the distribution of the test statistics under the null hypothesis of positive quadrant dependence is to resample from a constrained copula. This requires constrained estimation of a copula function. We show that this use of resampling under a constrained copula improves considerably the power performance of existing testing procedures. We propose two resampling procedures, one based on a parametric constrained copula estimation and one relying on nonparametric estimation of a positive quadrant dependence copula, and discuss their properties. The finite‐sample performances of the resulting testing procedures are evaluated via a simulation study that also includes comparisons with existing tests. Finally, a data set of Danish fire insurance claims is tested for positive quadrant dependence. The Canadian Journal of Statistics 41: 36–64; 2013 © 2012 Statistical Society of Canada     

Mutual information as a measure of multivariate association: analytical properties and statistical estimation

Mutual information (also known as Kullback–Leibler divergence) can be viewed as a measure of multivariate association in a random vector. The definition incorporates the joint density as well as the marginal densities. We will focus on a representation of mutual information in terms of copula densities that is thus independent of the marginal distributions. This representation yields a different approach to estimating mutual information than the original definition does, as only the copula density has to be estimated. We review analytical properties and examples for selected distributions and discuss methods of nonparametric estimation of copula densities and hence of the mutual information from a sample. Based on a simulation study, we compare the performance of these estimators with respect to bias, standard deviation, and the root mean squared error. The Gauss and the Frank copula are considered as examples.     

Positive quadrant dependence tests for copulas

In this paper the interest is in testing the null hypothesis of positive quadrant dependence (PQD) between two random variables. Such a testing problem is important since prior knowledge of PQD is a qualitative restriction that should be taken into account in further statistical analysis, for example, when choosing an appropriate copula function to model the dependence structure. The key methodology of the proposed testing procedures consists of evaluating a “distance” between a nonparametric estimator of a copula and the independence copula, which serves as a reference case in the whole set of copulas having the PQD property. Choices of appropriate distances and nonparametric estimators of copula are discussed, and the proposed methods are compared with testing procedures based on bootstrap and multiplier techniques. The consistency of the testing procedures is established. In a simulation study the authors investigate the finite sample size and power performances of three types of test statistics, Kolmogorov–Smirnov, Cramér–von‐Mises, and Anderson–Darling statistics, together with several nonparametric estimators of a copula, including recently developed kernel type estimators. Finally, they apply the testing procedures on some real data. The Canadian Journal of Statistics 38: 555–581; 2010 © 2010 Statistical Society of Canada     

Bootstrapping the conditional copula

This paper is concerned with inference about the dependence or association between two random variables conditionally upon the given value of a covariate. A way to describe such a conditional dependence is via a conditional copula function. Nonparametric estimators for a conditional copula then lead to nonparametric estimates of conditional association measures such as a conditional Kendall's tau. The limiting distributions of nonparametric conditional copula estimators are rather involved. In this paper we propose a bootstrap procedure for approximating these distributions and their characteristics, and establish its consistency. We apply the proposed bootstrap procedure for constructing confidence intervals for conditional association measures, such as a conditional Blomqvist beta and a conditional Kendall's tau. The performances of the proposed methods are investigated via a simulation study involving a variety of models, ranging from models in which the dependence (weak or strong) on the covariate is only through the copula and not through the marginals, to models in which this dependence appears in both the copula and the marginal distributions. As a conclusion we provide practical recommendations for constructing bootstrap-based confidence intervals for the discussed conditional association measures.     

Estimating curves and derivatives with parametric penalized spline smoothing

Accurate estimation of an underlying function and its derivatives is one of the central problems in statistics. Parametric forms are often proposed based on the expert opinion or prior knowledge of the underlying function. However, these strict parametric assumptions may result in biased estimates when they are not completely accurate. Meanwhile, nonparametric smoothing methods, which do not impose any parametric form, are quite flexible. We propose a parametric penalized spline smoothing method, which has the same flexibility as the nonparametric smoothing methods. It also uses the prior knowledge of the underlying function by defining an additional penalty term using the distance of the fitted function to the assumed parametric function. Our simulation studies show that the parametric penalized spline smoothing method can obtain more accurate estimates of the function and its derivatives than the penalized spline smoothing method. The parametric penalized spline smoothing method is also demonstrated by estimating the human height function and its derivatives from the real data.     

Nonparametric Bayesian survival analysis using mixtures of Weibull distributions

Bayesian nonparametric methods have been applied to survival analysis problems since the emergence of the area of Bayesian nonparametrics. However, the use of the flexible class of Dirichlet process mixture models has been rather limited in this context. This is, arguably, to a large extent, due to the standard way of fitting such models that precludes full posterior inference for many functionals of interest in survival analysis applications. To overcome this difficulty, we provide a computational approach to obtain the posterior distribution of general functionals of a Dirichlet process mixture. We model the survival distribution employing a flexible Dirichlet process mixture, with a Weibull kernel, that yields rich inference for several important functionals. In the process, a method for hazard function estimation emerges. Methods for simulation-based model fitting, in the presence of censoring, and for prior specification are provided. We illustrate the modeling approach with simulated and real data.     

A new Bayesian wavelet thresholding estimator of nonparametric regression

The methods of estimation of nonparametric regression function are quite common in statistical application. In this paper, the new Bayesian wavelet thresholding estimation is considered. The new mixture prior distributions for the estimation of nonparametric regression function by applying wavelet transformation are investigated. The reversible jump algorithm to obtain the appropriate prior distributions and value of thresholding is used. The performance of the proposed estimator is assessed with simulated data from well-known test functions by comparing the convergence rate of the proposed estimator with respect to another by evaluating the average mean square error and standard deviations. Finally by applying the developed method, density function of galaxy data is estimated.     

Noninformative nonparametric quantile estimation for simple random samples

For noninformative nonparametric estimation of finite population quantiles under simple random sampling, estimation based on the Polya posterior is similar to estimation based on the Bayesian approach developed by Ericson (J. Roy. Statist. Soc. Ser. B 31 (1969) 195) in that the Polya posterior distribution is the limit of Ericson's posterior distributions as the weight placed on the prior distribution diminishes. Furthermore, Polya posterior quantile estimates can be shown to be admissible under certain conditions. We demonstrate the admissibility of the sample median as an estimate of the population median under such a set of conditions. As with Ericson's Bayesian approach, Polya posterior-based interval estimates for population quantiles are asymptotically equivalent to the interval estimates obtained from standard frequentist approaches. In addition, for small to moderate sized populations, Polya posterior-based interval estimates for quantiles of a continuous characteristic of interest tend to agree with the standard frequentist interval estimates.     

Simultaneous inference in structured additive conditional copula regression models: a unifying Bayesian approach

While most regression models focus on explaining distributional aspects of one single response variable alone, interest in modern statistical applications has recently shifted towards simultaneously studying multiple response variables as well as their dependence structure. A particularly useful tool for pursuing such an analysis are copula-based regression models since they enable the separation of the marginal response distributions and the dependence structure summarised in a specific copula model. However, so far copula-based regression models have mostly been relying on two-step approaches where the marginal distributions are determined first whereas the copula structure is studied in a second step after plugging in the estimated marginal distributions. Moreover, the parameters of the copula are mostly treated as a constant not related to covariates and most regression specifications for the marginals are restricted to purely linear predictors. We therefore propose simultaneous Bayesian inference for both the marginal distributions and the copula using computationally efficient Markov chain Monte Carlo simulation techniques. In addition, we replace the commonly used linear predictor by a generic structured additive predictor comprising for example nonlinear effects of continuous covariates, spatial effects or random effects and furthermore allow to make the copula parameters covariate-dependent. To facilitate Bayesian inference, we construct proposal densities for a Metropolis–Hastings algorithm relying on quadratic approximations to the full conditionals of regression coefficients avoiding manual tuning. The performance of the resulting Bayesian estimates is evaluated in simulations comparing our approach with penalised likelihood inference, studying the choice of a specific copula model based on the deviance information criterion, and comparing a simultaneous approach with a two-step procedure. Furthermore, the flexibility of Bayesian conditional copula regression models is illustrated in two applications on childhood undernutrition and macroecology.     

Empirical Bayes approach to wavelet regression using ϵ-contaminated priors

We consider an empirical Bayes approach to standard nonparametric regression estimation using a nonlinear wavelet methodology. Instead of specifying a single prior distribution on the parameter space of wavelet coefficients, which is usually the case in the existing literature, we elicit the ?-contamination class of prior distributions that is particularly attractive to work with when one seeks robust priors in Bayesian analysis. The type II maximum likelihood approach to prior selection is used by maximizing the predictive distribution for the data in the wavelet domain over a suitable subclass of the ?-contamination class of prior distributions. For the prior selected, the posterior mean yields a thresholding procedure which depends on one free prior parameter and it is level- and amplitude-dependent, thus allowing better adaptation in function estimation. We consider an automatic choice of the free prior parameter, guided by considerations on an exact risk analysis and on the shape of the thresholding rule, enabling the resulting estimator to be fully automated in practice. We also compute pointwise Bayesian credible intervals for the resulting function estimate using a simulation-based approach. We use several simulated examples to illustrate the performance of the proposed empirical Bayes term-by-term wavelet scheme, and we make comparisons with other classical and empirical Bayes term-by-term wavelet schemes. As a practical illustration, we present an application to a real-life data set that was collected in an atomic force microscopy study.     

Large sample properties of nonparametric copula estimators under bivariate censoring

A new nonparametric estimator is proposed for the copula function of a bivariate survival function for data subject to random right-censoring. We consider two censoring models: univariate and copula censoring. We show strong consistency and we obtain an i.i.d. representation for the copula estimator. In a simulation study we compare the new estimator to the one of Gribkova and Lopez [Nonparametric copula estimation under bivariate censoring; doi:10.1111/sjos.12144].