Alternative Markov Properties for Chain Graphs

Graphical Markov models use graphs to represent possible dependences among statistical variables. Lauritzen, Wermuth, and Frydenberg (LWF) introduced a Markov property for chain graphs (CG): graphs that can be used to represent both structural and associative dependences simultaneously and that include both undirected graphs (UG) and acyclic directed graphs (ADG) as special cases. Here an alternative Markov property (AMP) for CGs is introduced and shown to be the Markov property satisfied by a block-recursive linear system with multivariate normal errors. This model can be decomposed into a collection of conditional normal models, each of which combines the features of multivariate linear regression models and covariance selection models, facilitating the estimation of its parameters. In the general case, necessary and sufficient conditions are given for the equivalence of the LWF and AMP Markov properties of a CG, for the AMP Markov equivalence of two CGs, for the AMP Markov equivalence of a CG to some ADG or decomposable UG, and for other equivalences. For CGs, in some ways the AMP property is a more direct extension of the ADG Markov property than is the LWF property.     

A Unified Approach to the Characterization of Equivalence Classes of DAGs, Chain Graphs with no Flags and Chain Graphs

Abstract.  A Markov property associates a set of conditional independencies to a graph. Two alternative Markov properties are available for chain graphs (CGs), the Lauritzen–Wermuth–Frydenberg (LWF) and the Andersson–Madigan– Perlman (AMP) Markov properties, which are different in general but coincide for the subclass of CGs with no flags . Markov equivalence induces a partition of the class of CGs into equivalence classes and every equivalence class contains a, possibly empty, subclass of CGs with no flags itself containing a, possibly empty, subclass of directed acyclic graphs (DAGs). LWF-Markov equivalence classes of CGs can be naturally characterized by means of the so-called largest CGs , whereas a graphical characterization of equivalence classes of DAGs is provided by the essential graphs . In this paper, we show the existence of largest CGs with no flags that provide a natural characterization of equivalence classes of CGs of this kind, with respect to both the LWF- and the AMP-Markov properties. We propose a procedure for the construction of the largest CGs, the largest CGs with no flags and the essential graphs, thereby providing a unified approach to the problem. As by-products we obtain a characterization of graphs that are largest CGs with no flags and an alternative characterization of graphs which are largest CGs. Furthermore, a known characterization of the essential graphs is shown to be a special case of our more general framework. The three graphical characterizations have a common structure: they use two versions of a locally verifiable graphical rule. Moreover, in case of DAGs, an immediate comparison of three characterizing graphs is possible.     

Formulas for counting acyclic digraph Markov equivalence classes

Multivariate Markov dependencies between different variables often can be represented graphically using acyclic digraphs (ADGs). In certain cases, though, different ADGs represent the same statistical model, thus leading to a set of equivalence classes of ADGs that constitute the true universe of available graphical models. Building upon the previously known formulas for counting the number of acyclic digraphs and the number of equivalence classes of size 1, formulas are developed to count ADG equivalence classes of arbitrary size, based on the chordal graph configurations that produce a class of that size. Theorems to validate the formulas as well as to aid in determining the appropriate chordal graphs to use for a given class size are included.     

On Block Ordering of Variables in Graphical Modelling

Abstract.  In graphical modelling, the existence of substantive background knowledge on block ordering of variables is used to perform structural learning within the family of chain graphs (CGs) in which every block corresponds to an undirected graph and edges joining vertices in different blocks are directed in accordance with the ordering. We show that this practice may lead to an inappropriate restriction of the search space and introduce the concept of labelled block ordering B corresponding to a family of B - consistent CGs in which every block may be either an undirected graph or a directed acyclic graph or, more generally, a CG. In this way we provide a flexible tool for specifying subsets of chain graphs, and we observe that the most relevant subsets of CGs considered in the literature are families of B -consistent CGs for the appropriate choice of B . Structural learning within a family of B -consistent CGs requires to deal with Markov equivalence. We provide a graphical characterization of equivalence classes of B -consistent CGs, namely the B - essential graphs , as well as a procedure to construct the B -essential graph for any given equivalence class of B -consistent chain graphs. Both largest CGs and essential graphs turn out to be special cases of B -essential graphs.     

Markov Properties for Acyclic Directed Mixed Graphs

We consider acyclic directed mixed graphs, in which directed edges ( x → y ) and bi-directed edges ( x ↔ y ) may occur. A simple extension of Pearl's d -separation criterion, called m -separation, is applied to these graphs. We introduce a local Markov property which is equivalent to the global property resulting from the m -separation criterion for arbitrary distributions.     

A SINful approach to Gaussian graphical model selection

Multivariate Gaussian graphical models are defined in terms of Markov properties, i.e., conditional independences, corresponding to missing edges in the graph. Thus model selection can be accomplished by testing these independences, which are equivalent to zero values of corresponding partial correlation coefficients. For concentration graphs, acyclic directed graphs, and chain graphs (both LWF and AMP classes), we apply Fisher's z-transform, Šidák's correlation inequality, and Holm's step-down procedure to simultaneously test the multiple hypotheses specified by these zero values. This simple method for model selection controls the overall error rate for incorrect edge inclusion. Prior information about the presence and/or absence of particular edges can be readily incorporated.     

Graphs for Margins of Bayesian Networks

Directed acyclic graph (DAG) models—also called Bayesian networks—are widely used in probabilistic reasoning, machine learning and causal inference. If latent variables are present, then the set of possible marginal distributions over the remaining (observed) variables is generally not represented by any DAG. Larger classes of mixed graphical models have been introduced to overcome this; however, as we show, these classes are not sufficiently rich to capture all the marginal models that can arise. We introduce a new class of hyper‐graphs, called mDAGs, and a latent projection operation to obtain an mDAG from the margin of a DAG. We show that each distinct marginal of a DAG model is represented by at least one mDAG and provide graphical results towards characterizing equivalence of these models. Finally, we show that mDAGs correctly capture the marginal structure of causally interpreted DAGs under interventions on the observed variables.     

Maximum Likelihood Estimation in Gaussian Chain Graph Models under the Alternative Markov Property

Abstract.  The Andersson–Madigan–Perlman (AMP) Markov property is a recently proposed alternative Markov property (AMP) for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced Lauritzen–Wermuth–Frydenberg Markov property that is coherent with data-generation by natural block-recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP chain graph models can be obtained by combining generalized least squares and iterative proportional fitting to an iterative algorithm. In an appendix, we give useful convergence results for iterative partial maximization algorithms that apply in particular to the described algorithm.     

时间序列的图模型及其在股市相关性中的应用

图模型方法是高维数据统计分析的重要工具,时间序列的图模型方法有链图、因果图和偏相关图,将基于VAR模型的时间序列链图和因果图应用于国际股票市场,研究主要股指的动态相关性,结果表明：美国股市对周边股市的影响较大。将偏相关图应用于亚洲股票市场,研究亚洲主要股指的交互作用,结果表明：中国内地是相对独立的市场,中国香港、台湾以及新加坡、日本股票市场之间存在显著的信息流动。     

Collapsibility for Directed Acyclic Graphs

Abstract.  Collapsibility means that the same statistical result of interest can be obtained before and after marginalization over some variables. In this paper, we discuss three kinds of collapsibility for directed acyclic graphs (DAGs): estimate collapsibility, conditional independence collapsibility and model collapsibility. Related to collapsibility, we discuss removability of variables from a DAG. We present conditions for these three different kinds of collapsibility and relationships among them. We give algorithms to find a minimum variable set containing a variable subset of interest onto which a statistical result is collapsible.     

Testing lumpability for marginal discrete hidden Markov models

When some states of a Markov chain are aggregated (or lumped) and the new process, with lumped states, inherits the Markov property, the original chain is said to be lumpable. We discuss the notion of lumpability for discrete hidden Markov models (DHMMs) and we explain why, in general, testing this hypothesis leads to non-standard problems. Nevertheless, we present a case where lumpability in DHMMs is a regular problem of comparing nested models. Finally, some simulation results assessing the performance of the proposed test and an application to two real data sets are given.     

Longitudinal network models and permutation-uniform Markov chains

Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the same parameter, improving data reduction. Further we show that the permutation-uniform subclass of these chains permit interpretation as an independent, identically distributed sequence on the same state space. We then apply these ideas to temporal exponential random graph models, for which permutation uniformity is well suited, and discuss mean-parameter convergence, dyadic independence, and exchangeability. Our framework facilitates our introducing a new network model; simplifies analysis of some network and autoregressive models from the literature, including by permitting closed-form expressions for maximum likelihood estimates for some models; and facilitates applying standard tools to longitudinal-network Markov chains from either asymptotics or single-observation exponential random graph models.     

Instability, Sensitivity, and Degeneracy of Discrete Exponential Families

In applications to dependent data, first and foremost relational data, a number of discrete exponential family models has turned out to be near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We introduce the notion of instability with an eye to characterize, detect, and penalize discrete exponential family models that are near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We show that unstable discrete exponential family models are characterized by excessive sensitivity and near-degeneracy. In special cases, the subset of the natural parameter space corresponding to non-degenerate distributions and mean-value parameters far from the boundary of the mean-value parameter space turns out to be a lower-dimensional subspace of the natural parameter space. These characteristics of unstable discrete exponential family models tend to obstruct Markov chain Monte Carlo simulation and statistical inference. In applications to relational data, we show that discrete exponential family models with Markov dependence tend to be unstable and that the parameter space of some curved exponential families contains unstable subsets.     

Analyses of the Markov modulated fluid flow with one-sided ph-type jumps using coupled queues and the completed graphs

In this paper, we analyze Markov modulated fluid flow processes with one-sided ph-type jumps using the completed graph and also through the limits of coupled queueing processes to be constructed. For the models, we derive various results on time-dependent distributions and distributions of first passage times, and present the Riccati equations that transform matrices of the first return times to 0 satisfy. The Riccati equations enable us to compute the transform matrices using Newton’s method which is known very fast and stable. Finally, we present some duality results between the model with ph-type downward jumps and the model with ph-type upward jumps. This paper contains extended results of Ahn (2009) and probabilistic interpretations given by the completed graphs.     

An empirical Bayes procedure for the selection of Gaussian graphical models

A new methodology for model determination in decomposable graphical Gaussian models (Dawid and Lauritzen in Ann. Stat. 21(3), 1272?C1317, 1993) is developed. The Bayesian paradigm is used and, for each given graph, a hyper-inverse Wishart prior distribution on the covariance matrix is considered. This prior distribution depends on hyper-parameters. It is well-known that the models??s posterior distribution is sensitive to the specification of these hyper-parameters and no completely satisfactory method is registered. In order to avoid this problem, we suggest adopting an empirical Bayes strategy, that is a strategy for which the values of the hyper-parameters are determined using the data. Typically, the hyper-parameters are fixed to their maximum likelihood estimations. In order to calculate these maximum likelihood estimations, we suggest a Markov chain Monte Carlo version of the Stochastic Approximation EM algorithm. Moreover, we introduce a new sampling scheme in the space of graphs that improves the add and delete proposal of Armstrong et al. (Stat. Comput. 19(3), 303?C316, 2009). We illustrate the efficiency of this new scheme on simulated and real datasets.     

Approximate Bayesian Computation for Exponential Random Graph Models for Large Social Networks

We consider the issue of sampling from the posterior distribution of exponential random graph (ERG) models and other statistical models with intractable normalizing constants. Existing methods based on exact sampling are either infeasible or require very long computing time. We study a class of approximate Markov chain Monte Carlo (MCMC) sampling schemes that deal with this issue. We also develop a new Metropolis–Hastings kernel to sample sparse large networks from ERG models. We illustrate the proposed methods on several examples.     

The double chain markov model

In the class of discrete time Markovian processes, two models are widely used, the Markov chain and the hidden Markov model. A major difference between these two models lies in the relation between successive outputs of the observed variable. In a visible Markov chain, these are directly correlated while in hidden models they are not. However, in some situations it is possible to observe both a hidden Markov chain and a direct relation between successive observed outputs. Unfortunately, the use of either a visible or a hidden model implies the suppression of one of these hypothesis. This paper prsents a Markovian model under random environment called the Double Chain Markov Model which takes into account the maijn features of both visible and hidden models. Its main purpose is the modeling of non-homogeneous time-series. It is very flexible and can be estimated with traditional methods. The model is applied on a sequence of wind speeds and it appears to model data more successfully than both the usual Markov chains and hidden Markov models.     

ML, PL, QL in Markov Chain Models

Abstract.  In many spatial and spatial-temporal models, and more generally in models with complex dependencies, it may be too difficult to carry out full maximum-likelihood (ML) analysis. Remedies include the use of pseudo-likelihood (PL) and quasi-likelihood (QL) (also called the composite likelihood). The present paper studies the ML, PL and QL methods for general Markov chain models, partly motivated by the desire to understand the precise behaviour of the PL and QL methods in settings where this can be analysed. We present limiting normality results and compare performances in different settings. For Markov chain models, the PL and QL methods can be seen as maximum penalized likelihood methods. We find that QL is typically preferable to PL, and that it loses very little to ML, while sometimes earning in model robustness. It has also appeal and potential as a modelling tool. Our methods are illustrated for consonant-vowel transitions in poetry and for analysis of DNA sequence evolution-type models.     

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Summary.  Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models , where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.