A Non‐parametric Test of Exchangeability for Extreme‐Value and Left‐Tail Decreasing Bivariate Copulas

Abstract. A non‐parametric rank‐based test of exchangeability for bivariate extreme‐value copulas is first proposed. The two key ingredients of the suggested approach are the non‐parametric rank‐based estimators of the Pickands dependence function recently studied by Genest and Segers, and a multiplier technique for obtaining approximate p‐values for the derived statistics. The proposed approach is then extended to left‐tail decreasing dependence structures that are not necessarily extreme‐value copulas. Large‐scale Monte Carlo experiments are used to investigate the level and power of the various versions of the test and show that the proposed procedure can be substantially more powerful than tests of exchangeability derived directly from the empirical copula. The approach is illustrated on well‐known financial data.     

Sequential Monte Carlo on large binary sampling spaces

A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric family for adaptive sampling on high dimensional binary spaces. A practical motivation for this problem is variable selection in a linear regression context. We want to sample from a Bayesian posterior distribution on the model space using an appropriate version of Sequential Monte Carlo. Raw versions of Sequential Monte Carlo are easily implemented using binary vectors with independent components. For high dimensional problems, however, these simple proposals do not yield satisfactory results. The key to an efficient adaptive algorithm are binary parametric families which take correlations into account, analogously to the multivariate normal distribution on continuous spaces. We provide a review of models for binary data and make one of them work in the context of Sequential Monte Carlo sampling. Computational studies on real life data with about a hundred covariates suggest that, on difficult instances, our Sequential Monte Carlo approach clearly outperforms standard techniques based on Markov chain exploration.     

Global sensitivity analysis using complex linear models

A global sensitivity analysis of complex computer codes is usually performed by calculating the Sobol indices. The indices are estimated using Monte Carlo methods. The Monte Carlo simulations are time-consuming even if the computer response is replaced by a metamodel. This paper proposes a new method for calculating sensitivity indices that overcomes the Monte Carlo estimation. The method assumes a discretization of the domain of simulation and uses the expansion of the computer response on an orthogonal basis of complex functions to built a metamodel. This metamodel is then used to derive an analytical estimation of the Sobol indices. This approach is successfully tested on analytical functions and is compared with two alternative methods.     

Power analysis of independence testing for three-way contingency tables of small sizes

The first aim of this paper is to introduce a modular test for the three-way contingency table (TT). The second aim is to describe the procedure of generating TT using the bar method. The third aim is on the one hand to suggest the measure of untruthfulness of H₀ and on the other hand to compare the quality of independence tests by using their power. Critical values for analyzed statistics were determined by simulating the Monte Carlo method.     

Geodesic Monte Carlo on Embedded Manifolds

Markov chain Monte Carlo methods explicitly defined on the manifold of probability distributions have recently been established. These methods are constructed from diffusions across the manifold and the solution of the equations describing geodesic flows in the Hamilton–Jacobi representation. This paper takes the differential geometric basis of Markov chain Monte Carlo further by considering methods to simulate from probability distributions that themselves are defined on a manifold, with common examples being classes of distributions describing directional statistics. Proposal mechanisms are developed based on the geodesic flows over the manifolds of support for the distributions, and illustrative examples are provided for the hypersphere and Stiefel manifold of orthonormal matrices.     

Bayesian validation assessment of multivariate computational models

Multivariate model validation is a complex decision-making problem involving comparison of multiple correlated quantities, based upon the available information and prior knowledge. This paper presents a Bayesian risk-based decision method for validation assessment of multivariate predictive models under uncertainty. A generalized likelihood ratio is derived as a quantitative validation metric based on Bayes’ theorem and Gaussian distribution assumption of errors between validation data and model prediction. The multivariate model is then assessed based on the comparison of the likelihood ratio with a Bayesian decision threshold, a function of the decision costs and prior of each hypothesis. The probability density function of the likelihood ratio is constructed using the statistics of multiple response quantities and Monte Carlo simulation. The proposed methodology is implemented in the validation of a transient heat conduction model, using a multivariate data set from experiments. The Bayesian methodology provides a quantitative approach to facilitate rational decisions in multivariate model assessment under uncertainty.     

A permutation procedure for testing the equality of pattern hypotheses across groups involving correlation or covariance matrices

This paper describes a permutation procedure to test for the equality of selected elements of a covariance or correlation matrix across groups. It involves either centring or standardising each variable within each group before randomly permuting observations between groups. Since the assumption of exchangeability of observations between groups does not strictly hold following such transformations, Monte Carlo simulations were used to compare expected and empirical rejection levels as a function of group size, the number of groups and distribution type (Normal, mixtures of Normals and Gamma with various values of the shape parameter). The Monte Carlo study showed that the estimated probability levels are close to those that would be obtained with an exact test except at very small sample sizes (5 or 10 observations per group). The test appears robust against non-normal data, different numbers of groups or variables per group and unequal sample sizes per group. Power was increased with increasing sample size, effect size and the number of elements in the matrix and power was decreased with increasingly unequal numbers of observations per group.     

Bayesian inference method for model validation and confidence extrapolation

This paper presents a Bayesian-hypothesis-testing-based methodology for model validation and confidence extrapolation under uncertainty, using limited test data. An explicit expression of the Bayes factor is derived for the interval hypothesis testing. The interval method is compared with the Bayesian point null hypothesis testing approach. The Bayesian network with Markov Chain Monte Carlo simulation and Gibbs sampling is explored for extrapolating the inference from the validated domain at the component level to the untested domain at the system level. The effect of the number of experiments on the confidence in the model validation decision is investigated. The probabilities of Type I and Type II errors in decision-making during the model validation and confidence extrapolation are quantified. The proposed methodologies are applied to a structural mechanics problem. Numerical results demonstrate that the Bayesian methodology provides a quantitative approach to facilitate rational decisions in model validation and confidence extrapolation under uncertainty.     

Sequential Monte Carlo with transformations

This paper examines methodology for performing Bayesian inference sequentially on a sequence of posteriors on spaces of different dimensions. For this, we use sequential Monte Carlo samplers, introducing the innovation of using deterministic transformations to move particles effectively between target distributions with different dimensions. This approach, combined with adaptive methods, yields an extremely flexible and general algorithm for Bayesian model comparison that is suitable for use in applications where the acceptance rate in reversible jump Markov chain Monte Carlo is low. We use this approach on model comparison for mixture models, and for inferring coalescent trees sequentially, as data arrives.     

Layered adaptive importance sampling

Monte Carlo methods represent the de facto standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performance of any such method is strictly related to the specification of the proposal distribution, such that unfortunate choices easily wreak havoc on the resulting estimators. In this work, we introduce a layered (i.e., hierarchical) procedure to generate samples employed within a Monte Carlo scheme. This approach ensures that an appropriate equivalent proposal density is always obtained automatically (thus eliminating the risk of a catastrophic performance), although at the expense of a moderate increase in the complexity. Furthermore, we provide a general unified importance sampling (IS) framework, where multiple proposal densities are employed and several IS schemes are introduced by applying the so-called deterministic mixture approach. Finally, given these schemes, we also propose a novel class of adaptive importance samplers using a population of proposals, where the adaptation is driven by independent parallel or interacting Markov chain Monte Carlo (MCMC) chains. The resulting algorithms efficiently combine the benefits of both IS and MCMC methods.     

A fractional order statistic towards defining a smooth quantile function for discrete data

This work is motivated in part by a recent publication by Ma et al. (2011) who resolved the asymptotic non-normality problem of the classical sample quantiles for discrete data through defining a new mid-distribution based quantile function. This work is the motivation for defining a new and improved smooth population quantile function given discrete data. Our definition is based on the theory of fractional order statistics. The main advantage of our definition as compared to its competitors is the capability to distinguish the uth quantile across different discrete distributions over the whole interval, u∈(0,1). In addition, we define the corresponding estimator of the smooth population quantiles and demonstrate the convergence and asymptotic normal distribution of the corresponding sample quantiles. We verify our theoretical results through a Monte Carlo simulation, and illustrate the utilization of our quantile function in a Q-Q plot for discrete data.     

An EM-type algorithm for multivariate mixture models

This paper introduces a new approach, based on dependent univariate GLMs, for fitting multivariate mixture models. This approach is a multivariate generalization of the method for univariate mixtures presented by Hinde (1982). Its accuracy and efficiency are compared with direct maximization of the log-likelihood. Using a simulation study, we also compare the efficiency of Monte Carlo and Gaussian quadrature methods for approximating the mixture distribution. The new approach with Gaussian quadrature outperforms the alternative methods considered. The work is motivated by the multivariate mixture models which have been proposed for modelling changes of employment states at an individual level. Similar formulations are of interest for modelling movement between other social and economic states and multivariate mixture models also occur in biostatistics and epidemiology.     

Profile Hellinger distance estimation

The successful application of the Hellinger distance approach to fully parametric models is well known. The corresponding optimal estimators, known as minimum Hellinger distance (MHD) estimators, are efficient and have excellent robustness properties [Beran R. Minimum Hellinger distance estimators for parametric models. Ann Statist. 1977;5:445–463]. This combination of efficiency and robustness makes MHD estimators appealing in practice. However, their application to semiparametric statistical models, which have a nuisance parameter (typically of infinite dimension), has not been fully studied. In this paper, we investigate a methodology to extend the MHD approach to general semiparametric models. We introduce the profile Hellinger distance and use it to construct a minimum profile Hellinger distance estimator of the finite-dimensional parameter of interest. This approach is analogous in some sense to the profile likelihood approach. We investigate the asymptotic properties such as the asymptotic normality, efficiency, and adaptivity of the proposed estimator. We also investigate its robustness properties. We present its small-sample properties using a Monte Carlo study.     

Empirical Bayes models of Poisson clinical trials and sample size determination

Bayesian methods are often used to reduce the sample sizes and/or increase the power of clinical trials. The right choice of the prior distribution is a critical step in Bayesian modeling. If the prior not completely specified, historical data may be used to estimate it. In the empirical Bayesian analysis, the resulting prior can be used to produce the posterior distribution. In this paper, we describe a Bayesian Poisson model with a conjugate Gamma prior. The parameters of Gamma distribution are estimated in the empirical Bayesian framework under two estimation schemes. The straightforward numerical search for the maximum likelihood (ML) solution using the marginal negative binomial distribution is unfeasible occasionally. We propose a simplification to the maximization procedure. The Markov Chain Monte Carlo method is used to create a set of Poisson parameters from the historical count data. These Poisson parameters are used to uniquely define the Gamma likelihood function. Easily computable approximation formulae may be used to find the ML estimations for the parameters of gamma distribution. For the sample size calculations, the ML solution is replaced by its upper confidence limit to reflect an incomplete exchangeability of historical trials as opposed to current studies. The exchangeability is measured by the confidence interval for the historical rate of the events. With this prior, the formula for the sample size calculation is completely defined. Published in 2009 by John Wiley & Sons, Ltd.     

Bayesian inference for generalized extreme value distributions via Hamiltonian Monte Carlo

In this article, we propose to evaluate and compare Markov chain Monte Carlo (MCMC) methods to estimate the parameters in a generalized extreme value model. We employed the Bayesian approach using traditional Metropolis-Hastings methods, Hamiltonian Monte Carlo (HMC), and Riemann manifold HMC (RMHMC) methods to obtain the approximations to the posterior marginal distributions of interest. Applications to real datasets and simulation studies provide evidence that the extra analytical work involved in Hamiltonian Monte Carlo algorithms is compensated by a more efficient exploration of the parameter space.     

A functional connectivity approach for modeling cross-sectional dependence with an application to the estimation of hedonic housing prices in Paris

This paper proposes a functional connectivity approach, inspired by brain imaging literature, to model cross-sectional dependence. Using a varying parameter framework, the model allows correlation patterns to arise from complex economic or social relations rather than being simply functions of economic or geographic distances between locations. It nests the conventional spatial and factor model approaches as special cases. A Bayesian Markov Chain Monte Carlo method implements this approach. A small scale Monte Carlo study is conducted to evaluate the performance of this approach in finite samples, which outperforms both a spatial model and a factor model. We apply the functional connectivity approach to estimate a hedonic housing price model for Paris using housing transactions over the period 1990–2003. It allows us to get more information about complex spatial connections and appears more suitable to capture the cross-sectional dependence than the conventional methods.     

Fitting finite mixture models using iterative Monte Carlo classification

Parameters of a finite mixture model are often estimated by the expectation–maximization (EM) algorithm where the observed data log-likelihood function is maximized. This paper proposes an alternative approach for fitting finite mixture models. Our method, called the iterative Monte Carlo classification (IMCC), is also an iterative fitting procedure. Within each iteration, it first estimates the membership probabilities for each data point, namely the conditional probability of a data point belonging to a particular mixing component given that the data point value is obtained, it then classifies each data point into a component distribution using the estimated conditional probabilities and the Monte Carlo method. It finally updates the parameters of each component distribution based on the classified data. Simulation studies were conducted to compare IMCC with some other algorithms for fitting mixture normal, and mixture t, densities.     

Particle methods for maximum likelihood estimation in latent variable models

Standard methods for maximum likelihood parameter estimation in latent variable models rely on the Expectation-Maximization algorithm and its Monte Carlo variants. Our approach is different and motivated by similar considerations to simulated annealing; that is we build a sequence of artificial distributions whose support concentrates itself on the set of maximum likelihood estimates. We sample from these distributions using a sequential Monte Carlo approach. We demonstrate state-of-the-art performance for several applications of the proposed approach.     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

Bayesian assessment of times to diagnosis in breast cancer screening

Breast cancer is one of the diseases with the most profound impact on health in developed countries and mammography is the most popular method for detecting breast cancer at a very early stage. This paper focuses on the waiting period from a positive mammogram until a confirmatory diagnosis is carried out in hospital. Generalized linear mixed models are used to perform the statistical analysis, always within the Bayesian reasoning. Markov chain Monte Carlo algorithms are applied for estimation by simulating the posterior distribution of the parameters and hyperparameters of the model through the free software WinBUGS.