Collapsibility for Directed Acyclic Graphs

Abstract.  Collapsibility means that the same statistical result of interest can be obtained before and after marginalization over some variables. In this paper, we discuss three kinds of collapsibility for directed acyclic graphs (DAGs): estimate collapsibility, conditional independence collapsibility and model collapsibility. Related to collapsibility, we discuss removability of variables from a DAG. We present conditions for these three different kinds of collapsibility and relationships among them. We give algorithms to find a minimum variable set containing a variable subset of interest onto which a statistical result is collapsible.     

Graphs for Margins of Bayesian Networks

Directed acyclic graph (DAG) models—also called Bayesian networks—are widely used in probabilistic reasoning, machine learning and causal inference. If latent variables are present, then the set of possible marginal distributions over the remaining (observed) variables is generally not represented by any DAG. Larger classes of mixed graphical models have been introduced to overcome this; however, as we show, these classes are not sufficiently rich to capture all the marginal models that can arise. We introduce a new class of hyper‐graphs, called mDAGs, and a latent projection operation to obtain an mDAG from the margin of a DAG. We show that each distinct marginal of a DAG model is represented by at least one mDAG and provide graphical results towards characterizing equivalence of these models. Finally, we show that mDAGs correctly capture the marginal structure of causally interpreted DAGs under interventions on the observed variables.     

ELICITING A DIRECTED ACYCLIC GRAPH FOR A MULTIVARIATE TIME SERIES OF VEHICLE COUNTS IN A TRAFFIC NETWORK

The problem of modelling multivariate time series of vehicle counts in traffic networks is considered. It is proposed to use a model called the linear multiregression dynamic model (LMDM). The LMDM is a multivariate Bayesian dynamic model which uses any conditional independence and causal structure across the time series to break down the complex multivariate model into simpler univariate dynamic linear models. The conditional independence and causal structure in the time series can be represented by a directed acyclic graph (DAG). The DAG not only gives a useful pictorial representation of the multivariate structure, but it is also used to build the LMDM. Therefore, eliciting a DAG which gives a realistic representation of the series is a crucial part of the modelling process. A DAG is elicited for the multivariate time series of hourly vehicle counts at the junction of three major roads in the UK. A flow diagram is introduced to give a pictorial representation of the possible vehicle routes through the network. It is shown how this flow diagram, together with a map of the network, can suggest a DAG for the time series suitable for use with an LMDM.     

Collapsibility of Conditional Graphical Models

Abstract. In this paper, we consider two kinds of collapsibility, that is, the model‐collapsibility and the estimate‐collapsibility, of conditional graphical models for multidimensional contingency tables. We show that these two definitions are equivalent, and propose a sufficient and necessary condition for them in terms of the interaction graph, which allows the collapsibility to be characterized and judged intuitively and conveniently.     

Objective Bayes Covariate‐Adjusted Sparse Graphical Model Selection

We present an objective Bayes method for covariance selection in Gaussian multivariate regression models having a sparse regression and covariance structure, the latter being Markov with respect to a directed acyclic graph (DAG). Our procedure can be easily complemented with a variable selection step, so that variable and graphical model selection can be performed jointly. In this way, we offer a solution to a problem of growing importance especially in the area of genetical genomics (eQTL analysis). The input of our method is a single default prior, essentially involving no subjective elicitation, while its output is a closed form marginal likelihood for every covariate‐adjusted DAG model, which is constant over each class of Markov equivalent DAGs; our procedure thus naturally encompasses covariate‐adjusted decomposable graphical models. In realistic experimental studies, our method is highly competitive, especially when the number of responses is large relative to the sample size.     

Whittemore's notion of collapsibility in multidimensional contingency tables

We develop simple necessary and sufficient conditions for a hierarchical log linear model to be strictly collapsible in the sense defined by Whittemore (1978). We then show that collapsibility as defined by Asmussen & Edwards (1983) can be viewed as equivalent to collapsibility as defined by Whittemore (1978) and illustrate why Bishop, Fienberg, & Holland's (1975, p.47) conditions for collapsibility are sufficient but not necessary. Finally, we discuss how collapsibility facilitates interpretation of certain hierarchical log linear models and formulation of hypotheses concerning marginal distributions associated with multidimensional contingency tables.     

Collapsibility of Graphical CG-Regression Models

Abstract.  CG-regressions are multivariate regression models for mixed continuous and discrete responses that result from conditioning in the class of conditional Gaussian (CG) models. Their conditional independence structure can be read off a marked graph. The property of collapsibility, in this context, means that the multivariate CG-regression can be decomposed into lower dimensional regressions that are still CG and are consistent with the corresponding subgraphs. We derive conditions for this property that can easily be checked on the graph, and indicate computational advantages of this kind of collapsibility. Further, a simple graphical condition is given for checking whether a decomposition into univariate regressions is possible.     

Collapsibility of contingency tables based on conditional models

Strict collapsibility and model collapsibility are two important concepts associated with the dimension reduction of a multidimensional contingency table, without losing the relevant information. In this paper, we obtain some necessary and sufficient conditions for the strict collapsibility of the full model, with respect to an interaction factor or a set of interaction factors, based on the interaction parameters of the conditional/layer log-linear models. For hierarchical log-linear models, we present also necessary and sufficient conditions for the full model to be model collapsible, based on the conditional interaction parameters. We discuss both the cases where one variable or a set of variables is conditioned. The connections between the strict collapsibility and the model collapsibility are also pointed out. Our results are illustrated through suitable examples, including a real life application.     

Identifiability and estimation of recursive max‐linear models

We address the identifiability and estimation of recursive max‐linear structural equation models represented by an edge‐weighted directed acyclic graph (DAG). Such models are generally unidentifiable and we identify the whole class of DAG s and edge weights corresponding to a given observational distribution. For estimation, standard likelihood theory cannot be applied because the corresponding families of distributions are not dominated. Given the underlying DAG, we present an estimator for the class of edge weights and show that it can be considered a generalized maximum likelihood estimator. In addition, we develop a simple method for identifying the structure of the DAG. With probability tending to one at an exponential rate with the number of observations, this method correctly identifies the class of DAGs and, similarly, exactly identifies the possible edge weights.     

Exact estimation of multiple directed acyclic graphs

This paper considers structure learning for multiple related directed acyclic graph (DAG) models. Building on recent developments in exact estimation of DAGs using integer linear programming (ILP), we present an ILP approach for joint estimation over multiple DAGs. Unlike previous work, we do not require that the vertices in each DAG share a common ordering. Furthermore, we allow for (potentially unknown) dependency structure between the DAGs. Results are presented on both simulated data and fMRI data obtained from multiple subjects.     

Conditions for Non-confounding and Collapsibility without Knowledge of Completely Constructed Causal Diagrams

In this paper, we discuss several concepts in causal inference in terms of causal diagrams proposed by Pearl (1993 , 1995a , b ), and we give conditions for non-confounding, homogeneity and collapsibility for causal effects without knowledge of a completely constructed causal diagram. We first introduce the concepts of non-confounding, conditional non-confounding, uniform non-confounding, homogeneity, collapsibility and strong collapsibility for causal effects, then we present necessary and sufficient conditions for uniform non-confounding, homegeneity and collapsibilities, and finally we show sufficient conditions for non-confounding, conditional non-confounding and uniform non-confounding.     

Average Collapsibility of Distribution Dependence and Quantile Regression Coefficients

Abstract. The Yule–Simpson paradox notes that an association between random variables can be reversed when averaged over a background variable. Cox and Wermuth introduced the concept of distribution dependence between two random variables X and Y, and gave two dependence conditions, each of which guarantees that reversal of qualitatively similar conditional dependences cannot occur after marginalizing over the background variable. Ma, Xie and Geng studied the uniform collapsibility of distribution dependence over a background variable W, under stronger homogeneity condition. Collapsibility ensures that associations are the same for conditional and marginal models. In this article, we use the notion of average collapsibility, which requires only the conditional effects average over the background variable to the corresponding marginal effect and investigate its conditions for distribution dependence and for quantile regression coefficients.     

Collapsibility of regression coefficients and its extensions

Collapsibility with respect to a measure of association implies that the measure of association can be obtained from the marginal model. We first discuss model collapsibility and collapsibility with respect to regression coefficients for linear regression models. For parallel regression models, we give simple and different proofs of some of the known results and obtain also certain new results. For random coefficient regression models, we define (average) A$A$-collapsibility and obtain conditions under which it holds. We consider Poisson regression and logistic regression models also, and derive conditions for collapsibility and A$A$-collapsibility, respectively. These results generalize some of the results available in the literature. Some suitable examples are also discussed.     

Information theoretic criterion approach to dimensionality reduction in multinomial logistic regression models. part i: theory

We discuss the issue of dimensionality reduction in multinomial logistic models as problems arising in variable selection, collapsibility of responses and linear restrictions in the parameter matrix. A method using the information theoretic criterion suggested by Bai, Krishnaiah and Zhao, a variation of Akaike information criterion, is used to estimate the rank of the parameter matrix. The same procedure is used for the selection of variables and collapsibility of response categories. The strong consistency of this procedure is established in all the problems.     

Sensitivity analysis of violations of the faithfulness assumption

We study the implication of violations of the faithfulness condition due to parameter cancellations on estimation of the directed acyclic graph (DAG) skeleton. Three settings are investigated: when (i) faithfulness is guaranteed (ii) faithfulness is not guaranteed and (iii) the parameter distributions are concentrated around unfaithfulness (near-unfaithfulness). In a simulation study, the effects of the different settings are compared using the parents and children (PC) and max–min parents and children (MMPC) algorithms. The results show that the performance in the faithful case is almost unchanged compared with the unrestricted case, whereas there is a general decrease in performance under the near-unfaithful case as compared with the unrestricted case. The response to near-unfaithful parameterizations is similar between the two algorithms, with the MMPC algorithm having higher true positive rates and the PC algorithm having lower false positive rates.     

Multiple time scales in survival analysis

In some problems in survival analysis there may be more than one plausible measure of time for each individual. For example mileage may be a better indication of the age of a car than months. This paper considers the possibility of combining two (or more) time scales measured on each individual into a single scale. A collapsibility condition is proposed for regarding the combined scale as fully informative regarding survival. The resulting model may be regarded as a generalization of the usual accelerated life model that allows time-dependent covariates. Parametric methods for the choice of time scale, for testing the validity of the collapsibility assumption and for parametric inference about the failure distribution along the new scale are discussed. Two examples are used to illustrate the methods, namely Hyde's (1980) Channing House data and a large cohort mortality study of asbestos workers in Quebec.     

Bayesian covariance matrix estimation using a mixture of decomposable graphical models

We present a Bayesian approach to estimating a covariance matrix by using a prior that is a mixture over all decomposable graphs, with the probability of each graph size specified by the user and graphs of equal size assigned equal probability. Most previous approaches assume that all graphs are equally probable. We show empirically that the prior that assigns equal probability over graph sizes outperforms the prior that assigns equal probability over all graphs in more efficiently estimating the covariance matrix. The prior requires knowing the number of decomposable graphs for each graph size and we give a simulation method for estimating these counts. We also present a Markov chain Monte Carlo method for estimating the posterior distribution of the covariance matrix that is much more efficient than current methods. Both the prior and the simulation method to evaluate the prior apply generally to any decomposable graphical model.     

Tractable Bayesian learning of tree belief networks

In this paper we present decomposable priors, a family of priors over structure and parameters of tree belief nets for which Bayesian learning with complete observations is tractable, in the sense that the posterior is also decomposable and can be completely determined analytically in polynomial time. Our result is the first where computing the normalization constant and averaging over a super-exponential number of graph structures can be performed in polynomial time. This follows from two main results: First, we show that factored distributions over spanning trees in a graph can be integrated in closed form. Second, we examine priors over tree parameters and show that a set of assumptions similar to Heckerman, Geiger and Chickering (1995) constrain the tree parameter priors to be a compactly parametrized product of Dirichlet distributions. Besides allowing for exact Bayesian learning, these results permit us to formulate a new class of tractable latent variable models in which the likelihood of a data point is computed through an ensemble average over tree structures.     

Modeling the density of US yield curve using Bayesian semiparametric dynamic Nelson-Siegel model

Abstract

This paper proposes the Bayesian semiparametric dynamic Nelson-Siegel model for estimating the density of bond yields. Specifically, we model the distribution of the yield curve factors according to an infinite Markov mixture (iMM). The model allows for time variation in the mean and covariance matrix of factors in a discrete manner, as opposed to continuous changes in these parameters such as the Time Varying Parameter (TVP) models. Estimating the number of regimes using the iMM structure endogenously leads to an adaptive process that can generate newly emerging regimes over time in response to changing economic conditions in addition to existing regimes. The potential of the proposed framework is examined using US bond yields data. The semiparametric structure of the factors can handle various forms of non-normalities including fat tails and nonlinear dependence between factors using a unified approach by generating new clusters capturing these specific characteristics. We document that modeling parameter changes in a discrete manner increases the model fit as well as forecasting performance at both short and long horizons relative to models with fixed parameters as well as the TVP model with continuous parameter changes. This is mainly due to fact that the discrete changes in parameters suit the typical low frequency monthly bond yields data characteristics better.     

Joint response graphs and separation induced by triangular systems

Summary.  We consider joint probability distributions generated recursively in terms of univariate conditional distributions satisfying conditional independence restrictions. The independences are captured by missing edges in a directed graph. A matrix form of such a graph, called the generating edge matrix, is triangular so the distributions that are generated over such graphs are called triangular systems. We study consequences of triangular systems after grouping or reordering of the variables for analyses as chain graph models, i.e. for alternative recursive factorizations of the given density using joint conditional distributions. For this we introduce families of linear triangular equations which do not require assumptions of distributional form. The strength of the associations that are implied by such linear families for chain graph models is derived. The edge matrices of chain graphs that are implied by any triangular system are obtained by appropriately transforming the generating edge matrix. It is shown how induced independences and dependences can be studied by graphs, by edge matrix calculations and via the properties of densities. Some ways of using the results are illustrated.