Pair‐copula constructions for non‐Gaussian DAG models

We propose a new type of multivariate statistical model that permits non‐Gaussian distributions as well as the inclusion of conditional independence assumptions specified by a directed acyclic graph. These models feature a specific factorisation of the likelihood that is based on pair‐copula constructions and hence involves only univariate distributions and bivariate copulas, of which some may be conditional. We demonstrate maximum‐likelihood estimation of the parameters of such models and compare them to various competing models from the literature. A simulation study investigates the effects of model misspecification and highlights the need for non‐Gaussian conditional independence models. The proposed methods are finally applied to modeling financial return data. The Canadian Journal of Statistics 40: 86–109; 2012 © 2012 Statistical Society of Canada     

Score-based causal learning in additive noise models

Given data sampled from a number of variables, one is often interested in the underlying causal relationships in the form of a directed acyclic graph. In the general case, without interventions on some of the variables it is only possible to identify the graph up to its Markov equivalence class. However, in some situations one can find the true causal graph just from observational data, for example, in structural equation models with additive noise and nonlinear edge functions. Most current methods for achieving this rely on nonparametric independence tests. One of the problems there is that the null hypothesis is independence, which is what one would like to get evidence for. We take a different approach in our work by using a penalized likelihood as a score for model selection. This is practically feasible in many settings and has the advantage of yielding a natural ranking of the candidate models. When making smoothness assumptions on the probability density space, we prove consistency of the penalized maximum likelihood estimator. We also present empirical results for simulated scenarios and real two-dimensional data sets (cause–effect pairs) where we obtain similar results as other state-of-the-art methods.     

Temporal aggregation in chain graph models

The dependence structure of an observed process induced by temporal aggregation of a time evolving hidden spatial phenomenon is addressed. Data are described by means of chain graph models and an algorithm to compute the chain graph resulting from the temporal aggregation of a directed acyclic graph is provided. This chain graph is the best graph which covers the independencies of the resulting process within the chain graph class. A sufficient condition that produces a memory loss of the observed process with respect to its hidden origin is analyzed. Some examples are used for illustrating algorithms and results.     

Identification of vector AR models with recursive structural errors using conditional independence graphs

In canonical vector time series autoregressions, which permit dependence only on past values, the errors generally show contemporaneous correlation. By contrast structural vector autoregressions allow contemporaneous series dependence and assume errors with no contemporaneous correlation. Such models having a recursive structure can be described by a directed acyclic graph. We show, with the use of a real example, how the identification of these models may be assisted by examination of the conditional independence graph of contemporaneous and lagged variables. In this example we identify the causal dependence of monthly Italian bank loan interest rates on government bond and repurchase agreement rates. When the number of series is larger, the structural modelling of the canonical errors alone is a useful initial step, and we first present such an example to demonstrate the general approach to identifying a directed graphical model.     

Objective Bayes Factors for Gaussian Directed Acyclic Graphical Models

Abstract. We propose an objective Bayesian method for the comparison of all Gaussian directed acyclic graphical models defined on a given set of variables. The method, which is based on the notion of fractional Bayes factor (BF), requires a single default (typically improper) prior on the space of unconstrained covariance matrices, together with a prior sample size hyper‐parameter, which can be set to its minimal value. We show that our approach produces genuine BFs. The implied prior on the concentration matrix of any complete graph is a data‐dependent Wishart distribution, and this in turn guarantees that Markov equivalent graphs are scored with the same marginal likelihood. We specialize our results to the smaller class of Gaussian decomposable undirected graphical models and show that in this case they coincide with those recently obtained using limiting versions of hyper‐inverse Wishart distributions as priors on the graph‐constrained covariance matrices.     

Objective Bayes Covariate‐Adjusted Sparse Graphical Model Selection

We present an objective Bayes method for covariance selection in Gaussian multivariate regression models having a sparse regression and covariance structure, the latter being Markov with respect to a directed acyclic graph (DAG). Our procedure can be easily complemented with a variable selection step, so that variable and graphical model selection can be performed jointly. In this way, we offer a solution to a problem of growing importance especially in the area of genetical genomics (eQTL analysis). The input of our method is a single default prior, essentially involving no subjective elicitation, while its output is a closed form marginal likelihood for every covariate‐adjusted DAG model, which is constant over each class of Markov equivalent DAGs; our procedure thus naturally encompasses covariate‐adjusted decomposable graphical models. In realistic experimental studies, our method is highly competitive, especially when the number of responses is large relative to the sample size.     

Identifiability and estimation of recursive max‐linear models

We address the identifiability and estimation of recursive max‐linear structural equation models represented by an edge‐weighted directed acyclic graph (DAG). Such models are generally unidentifiable and we identify the whole class of DAG s and edge weights corresponding to a given observational distribution. For estimation, standard likelihood theory cannot be applied because the corresponding families of distributions are not dominated. Given the underlying DAG, we present an estimator for the class of edge weights and show that it can be considered a generalized maximum likelihood estimator. In addition, we develop a simple method for identifying the structure of the DAG. With probability tending to one at an exponential rate with the number of observations, this method correctly identifies the class of DAGs and, similarly, exactly identifies the possible edge weights.     

Copula directed acyclic graphs

A new methodology for selecting a Bayesian network for continuous data outside the widely used class of multivariate normal distributions is developed. The ‘copula DAGs’ combine directed acyclic graphs and their associated probability models with copula C/D-vines. Bivariate copula densities introduce flexibility in the joint distributions of pairs of nodes in the network. An information criterion is studied for graph selection tailored to the joint modeling of data based on graphs and copulas. Examples and simulation studies show the flexibility and properties of the method.     

Inferring large graphs using $$\ell _1$$-penalized likelihood

We address the issue of recovering the structure of large sparse directed acyclic graphs from noisy observations of the system. We propose a novel procedure based on a specific formulation of the \(\ell _1\)-norm regularized maximum likelihood, which decomposes the graph estimation into two optimization sub-problems: topological structure and node order learning. We provide convergence inequalities for the graph estimator, as well as an algorithm to solve the induced optimization problem, in the form of a convex program embedded in a genetic algorithm. We apply our method to various data sets (including data from the DREAM4 challenge) and show that it compares favorably to state-of-the-art methods. This algorithm is available on CRAN as the R package GADAG.     

A Note on Collapsibility in DAG Models of Contingency Tables

Abstract.  Necessary and sufficient conditions for collapsibility of a directed acyclic graph (DAG) model for a contingency table are derived. By applying the conditions, we can easily check collapsibility over any variable in a given model either by using the joint probability distribution or by using the graph of the model structure. It is shown that collapsibility over a set of variables can be checked in a sequential manner. Furthermore, a DAG is compared with its moral graph in the context of collapsibility.     

The evaluation of DNA evidence in pedigrees requiring population inference

Summary. The evaluation of nuclear DNA evidence for identification purposes is performed here taking account of the uncertainty about population parameters. Graphical models are used to detail the hypotheses being debated in a trial with the aim of obtaining a directed acyclic graph. Graphs also clarify the set of evidence that contributes to population inferences and they also describe the conditional independence structure of DNA evidence. Numerical illustrations are provided by re-examining three case-studies taken from the literature. Our calculations of the weight of evidence differ from those given by the authors of case-studies in that they reveal more conservative values.     

Bayesian model comparison based on expected posterior priors for discrete decomposable graphical models

The implementation of the Bayesian paradigm to model comparison can be problematic. In particular, prior distributions on the parameter space of each candidate model require special care. While it is well known that improper priors cannot be routinely used for Bayesian model comparison, we claim that also the use of proper conventional priors under each model should be regarded as suspicious, especially when comparing models having different dimensions. The basic idea is that priors should not be assigned separately under each model; rather they should be related across models, in order to acquire some degree of compatibility, and thus allow fairer and more robust comparisons. In this connection, the intrinsic prior as well as the expected posterior prior (EPP) methodology represent a useful tool. In this paper we develop a procedure based on EPP to perform Bayesian model comparison for discrete undirected decomposable graphical models, although our method could be adapted to deal also with directed acyclic graph models. We present two possible approaches. One based on imaginary data, and one which makes use of a limited number of actual data. The methodology is illustrated through the analysis of a 2×3×4 contingency table.     

Formulas for counting acyclic digraph Markov equivalence classes

Multivariate Markov dependencies between different variables often can be represented graphically using acyclic digraphs (ADGs). In certain cases, though, different ADGs represent the same statistical model, thus leading to a set of equivalence classes of ADGs that constitute the true universe of available graphical models. Building upon the previously known formulas for counting the number of acyclic digraphs and the number of equivalence classes of size 1, formulas are developed to count ADG equivalence classes of arbitrary size, based on the chordal graph configurations that produce a class of that size. Theorems to validate the formulas as well as to aid in determining the appropriate chordal graphs to use for a given class size are included.     

Uniformity of Node Level Conflict Measures in Bayesian Hierarchical Models Based on Directed Acyclic Graphs

Hierarchical models defined by means of directed, acyclic graphs are a powerful and widely used tool for Bayesian analysis of problems of varying degrees of complexity. A simulation‐based method for model criticism in such models has been suggested by O'Hagan in the form of a conflict measure based on contrasting separate local information sources about each node in the graph. This measure is however not well calibrated. In order to rectify this, alternative mutually similar tail probability‐based measures have been proposed independently and have been proved to be uniformly distributed under the assumed model in quite general normal models with known covariance matrices. In the present paper, we extend this result to a variety of models. An advantage of this is that computationally costly pre‐calibration schemes needed for some other suggested methods can be avoided. Another advantage is that non‐informative prior distributions can be used when performing model criticism.     

Markov Chain Monte Carlo model selection for DAG models

We present a methodology for Bayesian model choice and averaging in Gaussian directed acyclic graphs (dags). The dimension-changing move involves adding or dropping a (directed) edge from the graph. The methodology employs the results in Geiger and Heckerman and searches directly in the space of all dags. Model determination is carried out by implementing a reversible jump Markov Chain Monte Carlo sampler. To achieve this aim we rely on the concept of adjacency matrices, which provides a relatively inexpensive check for acyclicity. The performance of our procedure is illustrated by means of two simulated datasets, as well as one real dataset.     

On Block Ordering of Variables in Graphical Modelling

Abstract.  In graphical modelling, the existence of substantive background knowledge on block ordering of variables is used to perform structural learning within the family of chain graphs (CGs) in which every block corresponds to an undirected graph and edges joining vertices in different blocks are directed in accordance with the ordering. We show that this practice may lead to an inappropriate restriction of the search space and introduce the concept of labelled block ordering B corresponding to a family of B - consistent CGs in which every block may be either an undirected graph or a directed acyclic graph or, more generally, a CG. In this way we provide a flexible tool for specifying subsets of chain graphs, and we observe that the most relevant subsets of CGs considered in the literature are families of B -consistent CGs for the appropriate choice of B . Structural learning within a family of B -consistent CGs requires to deal with Markov equivalence. We provide a graphical characterization of equivalence classes of B -consistent CGs, namely the B - essential graphs , as well as a procedure to construct the B -essential graph for any given equivalence class of B -consistent chain graphs. Both largest CGs and essential graphs turn out to be special cases of B -essential graphs.     

Graphs for Margins of Bayesian Networks

Directed acyclic graph (DAG) models—also called Bayesian networks—are widely used in probabilistic reasoning, machine learning and causal inference. If latent variables are present, then the set of possible marginal distributions over the remaining (observed) variables is generally not represented by any DAG. Larger classes of mixed graphical models have been introduced to overcome this; however, as we show, these classes are not sufficiently rich to capture all the marginal models that can arise. We introduce a new class of hyper‐graphs, called mDAGs, and a latent projection operation to obtain an mDAG from the margin of a DAG. We show that each distinct marginal of a DAG model is represented by at least one mDAG and provide graphical results towards characterizing equivalence of these models. Finally, we show that mDAGs correctly capture the marginal structure of causally interpreted DAGs under interventions on the observed variables.     

Exact estimation of multiple directed acyclic graphs

This paper considers structure learning for multiple related directed acyclic graph (DAG) models. Building on recent developments in exact estimation of DAGs using integer linear programming (ILP), we present an ILP approach for joint estimation over multiple DAGs. Unlike previous work, we do not require that the vertices in each DAG share a common ordering. Furthermore, we allow for (potentially unknown) dependency structure between the DAGs. Results are presented on both simulated data and fMRI data obtained from multiple subjects.     

Collapsibility of Graphical CG-Regression Models

Abstract.  CG-regressions are multivariate regression models for mixed continuous and discrete responses that result from conditioning in the class of conditional Gaussian (CG) models. Their conditional independence structure can be read off a marked graph. The property of collapsibility, in this context, means that the multivariate CG-regression can be decomposed into lower dimensional regressions that are still CG and are consistent with the corresponding subgraphs. We derive conditions for this property that can easily be checked on the graph, and indicate computational advantages of this kind of collapsibility. Further, a simple graphical condition is given for checking whether a decomposition into univariate regressions is possible.     

On the Markov Equivalence of Chain Graphs, Undirected Graphs, and Acyclic Digraphs

Graphical Markov models use undirected graphs (UDGs), acyclic directed graphs (ADGs), or (mixed) chain graphs to represent possible dependencies among random variables in a multivariate distribution. Whereas a UDG is uniquely determined by its associated Markov model, this is not true for ADGs or for general chain graphs (which include both UDGs and ADGs as special cases). This paper addresses three questions regarding the equivalence of graphical Markov models: when is a given chain graph Markov equivalent (1) to some UDG? (2) to some (at least one) ADG? (3) to some decomposable UDG? The answers are obtained by means of an extension of Frydenberg’s (1990) elegant graph-theoretic characterization of the Markov equivalence of chain graphs.