Kernel partial correlation: a novel approach to capturing conditional independence in graphical models for noisy data

Graphical models capture the conditional independence structure among random variables via existence of edges among vertices. One way of inferring a graph is to identify zero partial correlation coefficients, which is an effective way of finding conditional independence under a multivariate Gaussian setting. For more general settings, we propose kernel partial correlation which extends partial correlation with a combination of two kernel methods. First, a nonparametric function estimation is employed to remove effects from other variables, and then the dependence between remaining random components is assessed through a nonparametric association measure. The proposed approach is not only flexible but also robust under high levels of noise owing to the robustness of the nonparametric approaches.     

ELICITING A DIRECTED ACYCLIC GRAPH FOR A MULTIVARIATE TIME SERIES OF VEHICLE COUNTS IN A TRAFFIC NETWORK

The problem of modelling multivariate time series of vehicle counts in traffic networks is considered. It is proposed to use a model called the linear multiregression dynamic model (LMDM). The LMDM is a multivariate Bayesian dynamic model which uses any conditional independence and causal structure across the time series to break down the complex multivariate model into simpler univariate dynamic linear models. The conditional independence and causal structure in the time series can be represented by a directed acyclic graph (DAG). The DAG not only gives a useful pictorial representation of the multivariate structure, but it is also used to build the LMDM. Therefore, eliciting a DAG which gives a realistic representation of the series is a crucial part of the modelling process. A DAG is elicited for the multivariate time series of hourly vehicle counts at the junction of three major roads in the UK. A flow diagram is introduced to give a pictorial representation of the possible vehicle routes through the network. It is shown how this flow diagram, together with a map of the network, can suggest a DAG for the time series suitable for use with an LMDM.     

Collapsibility of Graphical CG-Regression Models

Abstract.  CG-regressions are multivariate regression models for mixed continuous and discrete responses that result from conditioning in the class of conditional Gaussian (CG) models. Their conditional independence structure can be read off a marked graph. The property of collapsibility, in this context, means that the multivariate CG-regression can be decomposed into lower dimensional regressions that are still CG and are consistent with the corresponding subgraphs. We derive conditions for this property that can easily be checked on the graph, and indicate computational advantages of this kind of collapsibility. Further, a simple graphical condition is given for checking whether a decomposition into univariate regressions is possible.     

Singular Gaussian graphical models: Structure learning

ABSTRACT

The goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.     

Graphical models for skew-normal variates

This paper explores the usefulness of the multivariate skew-normal distribution in the context of graphical models. A slight extension of the family recently discussed by Azzalini & Dalla Valle (1996 ) and Azzalini & Capitanio (1999 ) is described, the main motivation being the additional property of closure under conditioning. After considerations of the main probabilistic features, the focus of the paper is on the construction of conditional independence graphs for skew-normal variables. Necessary and sufficient conditions for conditional independence are stated, and the admissible structures of a graph under restriction on univariate marginal distribution are studied. Finally, parameter estimation is considered. It is shown how the factorization of the likelihood function according to a graph can be rearranged in order to obtain a parameter based factorization.     

Joint response graphs and separation induced by triangular systems

Summary.  We consider joint probability distributions generated recursively in terms of univariate conditional distributions satisfying conditional independence restrictions. The independences are captured by missing edges in a directed graph. A matrix form of such a graph, called the generating edge matrix, is triangular so the distributions that are generated over such graphs are called triangular systems. We study consequences of triangular systems after grouping or reordering of the variables for analyses as chain graph models, i.e. for alternative recursive factorizations of the given density using joint conditional distributions. For this we introduce families of linear triangular equations which do not require assumptions of distributional form. The strength of the associations that are implied by such linear families for chain graph models is derived. The edge matrices of chain graphs that are implied by any triangular system are obtained by appropriately transforming the generating edge matrix. It is shown how induced independences and dependences can be studied by graphs, by edge matrix calculations and via the properties of densities. Some ways of using the results are illustrated.     

Pair‐copula constructions for non‐Gaussian DAG models

We propose a new type of multivariate statistical model that permits non‐Gaussian distributions as well as the inclusion of conditional independence assumptions specified by a directed acyclic graph. These models feature a specific factorisation of the likelihood that is based on pair‐copula constructions and hence involves only univariate distributions and bivariate copulas, of which some may be conditional. We demonstrate maximum‐likelihood estimation of the parameters of such models and compare them to various competing models from the literature. A simulation study investigates the effects of model misspecification and highlights the need for non‐Gaussian conditional independence models. The proposed methods are finally applied to modeling financial return data. The Canadian Journal of Statistics 40: 86–109; 2012 © 2012 Statistical Society of Canada     

Bayes linear analysis for graphical models: The geometric approach to local computation and interpretive graphics

This paper concerns the geometric treatment of graphical models using Bayes linear methods. We introduce Bayes linear separation as a second order generalised conditional independence relation, and Bayes linear graphical models are constructed using this property. A system of interpretive and diagnostic shadings are given, which summarise the analysis over the associated moral graph. Principles of local computation are outlined for the graphical models, and an algorithm for implementing such computation over the junction tree is described. The approach is illustrated with two examples. The first concerns sales forecasting using a multivariate dynamic linear model. The second concerns inference for the error variance matrices of the model for sales, and illustrates the generality of our geometric approach by treating the matrices directly as random objects. The examples are implemented using a freely available set of object-oriented programming tools for Bayes linear local computation and graphical diagnostic display.     

Quasi‐Symmetric Graphical Log‐Linear Models

Abstract. We propose an extension of graphical log‐linear models to allow for symmetry constraints on some interaction parameters that represent homologous factors. The conditional independence structure of such quasi‐symmetric (QS) graphical models is described by an undirected graph with coloured edges, in which a particular colour corresponds to a set of equality constraints on a set of parameters. Unlike standard QS models, the proposed models apply with contingency tables for which only some variables or sets of the variables have the same categories. We study the graphical properties of such models, including conditions for decomposition of model parameters and of maximum likelihood estimates.     

Identification of vector AR models with recursive structural errors using conditional independence graphs

In canonical vector time series autoregressions, which permit dependence only on past values, the errors generally show contemporaneous correlation. By contrast structural vector autoregressions allow contemporaneous series dependence and assume errors with no contemporaneous correlation. Such models having a recursive structure can be described by a directed acyclic graph. We show, with the use of a real example, how the identification of these models may be assisted by examination of the conditional independence graph of contemporaneous and lagged variables. In this example we identify the causal dependence of monthly Italian bank loan interest rates on government bond and repurchase agreement rates. When the number of series is larger, the structural modelling of the canonical errors alone is a useful initial step, and we first present such an example to demonstrate the general approach to identifying a directed graphical model.     

The evaluation of DNA evidence in pedigrees requiring population inference

Summary. The evaluation of nuclear DNA evidence for identification purposes is performed here taking account of the uncertainty about population parameters. Graphical models are used to detail the hypotheses being debated in a trial with the aim of obtaining a directed acyclic graph. Graphs also clarify the set of evidence that contributes to population inferences and they also describe the conditional independence structure of DNA evidence. Numerical illustrations are provided by re-examining three case-studies taken from the literature. Our calculations of the weight of evidence differ from those given by the authors of case-studies in that they reveal more conservative values.     

A general definition of influence between stochastic processes

We extend the study of weak local conditional independence (WCLI) based on a measurability condition made by (Commenges and Gégout-Petit J R Stat Soc B 71:1–18) to a larger class of processes that we call D￠{\bf {\mathcal{D}'}}. We also give a definition related to the same concept based on certain likelihood processes, using the Girsanov theorem. Under certain conditions, the two definitions coincide on D￠{\bf {\mathcal{D}'}}. These results may be used in causal models in that we define what may be the largest class of processes in which influences of one component of a stochastic process on another can be described without ambiguity. From WCLI we can construct a concept of strong local conditional independence (SCLI). When WCLI does not hold, there is a direct influence while when SCLI does not hold there is direct or indirect influence. We investigate whether WCLI and SCLI can be defined via conventional independence conditions and find that this is the case for the latter but not for the former. Finally we recall that causal interpretation does not follow from mere mathematical definitions, but requires working with a good system and with the true probability.     

PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE

This paper investigates the roles of partial correlation and conditional correlation as measures of the conditional independence of two random variables. It first establishes a sufficient condition for the coincidence of the partial correlation with the conditional correlation. The condition is satisfied not only for multivariate normal but also for elliptical, multivariate hypergeometric, multivariate negative hypergeometric, multinomial and Dirichlet distributions. Such families of distributions are characterized by a semigroup property as a parametric family of distributions. A necessary and sufficient condition for the coincidence of the partial covariance with the conditional covariance is also derived. However, a known family of multivariate distributions which satisfies this condition cannot be found, except for the multivariate normal. The paper also shows that conditional independence has no close ties with zero partial correlation except in the case of the multivariate normal distribution; it has rather close ties to the zero conditional correlation. It shows that the equivalence between zero conditional covariance and conditional independence for normal variables is retained by any monotone transformation of each variable. The results suggest that care must be taken when using such correlations as measures of conditional independence unless the joint distribution is known to be normal. Otherwise a new concept of conditional independence may need to be introduced in place of conditional independence through zero conditional correlation or other statistics.     

A method for consideration of conditional dependencies in the Fellegi and Sunter model of record linkage

An objective of Record Linkage is to link two data files by identifying common elements. A popular model for doing the separation is the probabilistic one from Fellegi and Sunter. To estimate the parameters needed for the model usually a mixture model is constructed and the EM algorithm is applied. For simplification, the assumption of conditional independence is often made. This assumption says that if several attributes of elements in the data are compared, then the results of the comparisons regarding the several attributes are independent within the mixture classes. A mixture model constructed with this assumption has been often used. Within this article a straightforward extension of the model is introduced which allows for conditional dependencies but is heavily dependent on the choice of the starting value. Therefore also an estimation procedure for the EM algorithm starting value is proposed. The two models are compared empirically in a simulation study based on telephone book entries. Particularly the effect of different starting values and conditional dependencies on the matching results is investigated.     

Labelled Graphical Models

A class of log‐linear models, referred to as labelled graphical models (LGMs), is introduced for multinomial distributions. These models generalize graphical models (GMs) by employing partial conditional independence restrictions which are valid only in subsets of an outcome space. Theoretical results concerning model identifiability, decomposability and estimation are derived. A decision theoretical framework and a search algorithm for the identification of plausible models are described. Real data sets are used to illustrate that LGMs may provide a simpler interpretation of a dependence structure than GMs.     

Specification testing in nonparametric AR‐ARCH models

In this paper, an autoregressive time series model with conditional heteroscedasticity is considered, where both conditional mean and conditional variance function are modeled nonparametrically. Tests for the model assumption of independence of innovations from past time series values are suggested. Tests based on weighted L²‐distances of empirical characteristic functions are considered as well as a Cramér–von Mises‐type test. The asymptotic distributions under the null hypothesis of independence are derived, and the consistency against fixed alternatives is shown. A smooth autoregressive residual bootstrap procedure is suggested, and its performance is shown in a simulation study.     

Prior distributions for stratified capture-recapture models

We consider the Arnason-Schwarz model, usually used to estimate survival and movement probabilities from capture-recapture data. A missing data structure of this model is constructed which allows a clear separation of information relative to capture and relative to movement. Extensions of the Arnason-Schwarz model are considered. For example, we consider a model that takes into account both the individual migration history and the individual reproduction history. Biological assumptions of these extensions are summarized via a directed graph. Owing to missing data, the posterior distribution of parameters is numerically intractable. To overcome those computational difficulties we advocate a Gibbs sampling algorithm that takes advantage of the missing data structure inherent in capture-recapture models. Prior information on survival, capture and movement probabilities typically consists of a prior mean and of a prior 95% credible confidence interval. Dirichlet distributions are used to incorporate some prior information on capture, survival probabilities, and movement probabilities. Finally, the influence of the prior on the Bayesian estimates of movement probabilities is examined.     

Prior distributions for stratified capture-recapture models

We consider the Arnason-Schwarz model, usually used to estimate survival and movement probabilities from capture-recapture data. A missing data structure of this model is constructed which allows a clear separation of information relative to capture and relative to movement. Extensions of the Arnason-Schwarz model are considered. For example, we consider a model that takes into account both the individual migration history and the individual reproduction history. Biological assumptions of these extensions are summarized via a directed graph. Owing to missing data, the posterior distribution of parameters is numerically intractable. To overcome those computational difficulties we advocate a Gibbs sampling algorithm that takes advantage of the missing data structure inherent in capture-recapture models. Prior information on survival, capture and movement probabilities typically consists of a prior mean and of a prior 95% credible confidence interval. Dirichlet distributions are used to incorporate some prior information on capture, survival probabilities, and movement probabilities. Finally, the influence of the prior on the Bayesian estimates of movement probabilities is examined.