Stratified Gaussian graphical models

Gaussian graphical models represent the backbone of the statistical toolbox for analyzing continuous multivariate systems. However, due to the intrinsic properties of the multivariate normal distribution, use of this model family may hide certain forms of context-specific independence that are natural to consider from an applied perspective. Such independencies have been earlier introduced to generalize discrete graphical models and Bayesian networks into more flexible model families. Here, we adapt the idea of context-specific independence to Gaussian graphical models by introducing a stratification of the Euclidean space such that a conditional independence may hold in certain segments but be absent elsewhere. It is shown that the stratified models define a curved exponential family, which retains considerable tractability for parameter estimation and model selection.     

Singular Gaussian graphical models: Structure learning

ABSTRACT

The goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.     

A SINful approach to Gaussian graphical model selection

Multivariate Gaussian graphical models are defined in terms of Markov properties, i.e., conditional independences, corresponding to missing edges in the graph. Thus model selection can be accomplished by testing these independences, which are equivalent to zero values of corresponding partial correlation coefficients. For concentration graphs, acyclic directed graphs, and chain graphs (both LWF and AMP classes), we apply Fisher's z-transform, Šidák's correlation inequality, and Holm's step-down procedure to simultaneously test the multiple hypotheses specified by these zero values. This simple method for model selection controls the overall error rate for incorrect edge inclusion. Prior information about the presence and/or absence of particular edges can be readily incorporated.     

Effectiveness of combinations of Gaussian graphical models for model building

Combining statistical models is an useful approach in all the research area where a global picture of the problem needs to be constructed by binding together evidence from different sources [M.S. Massa and S.L. Lauritzen Combining Statistical Models, M. Viana and H. Wynn, eds., American Mathematical Society, Providence, RI, 2010, pp. 239–259]. In this paper, we investigate the effectiveness of combining a fixed number of Gaussian graphical models respecting some consistency assumptions in problems of model building. In particular, we use the meta-Markov combination of Gaussian graphical models as detailed in Massa and Lauritzen and compare model selection results obtained by combining selections over smaller sets of variables with selection results over all variables of interest. In order to do so, we carry out some simulation studies in which different criteria are considered for the selection procedures. We conclude that the combination performs, generally, better than global estimation, is computationally simpler by virtue of having fewer and simpler models to work on, and has an intuitive appeal to a wide variety of contexts.     

Efficient estimation and model selection in large graphical models

We develop a computationally efficient method to determine the interaction structure in a multidimensional binary sample. We use an interaction model based on orthogonal functions, and give a result on independence properties in this model. Using this result we develop an efficient approximation algorithm for estimating the parameters in a given undirected model. To find the best model, we use a heuristic search algorithm in which the structure is determined incrementally. We also give an algorithm for reconstructing the causal directions, if such exist. We demonstrate that together these algorithms are capable of discovering almost all of the true structure for a problem with 121 variables, including many of the directions.     

Standard errors for the parameters of graphical Gaussian models

The comparison of an estimated parameter to its standard error, the Wald test, is a well known procedure of classical statistics. Here we discuss its application to graphical Gaussian model selection. First we derive the Fisher information matrix and its inverse about the parameters of any graphical Gaussian model. Both the covariance matrix and its inverse are considered and a comparative analysis of the asymptotic behaviour of their maximum likelihood estimators (m.l.e.s) is carried out. Then we give an example of model selection based on the standard errors. The method is shown to produce almost identical inference to likelihood ratio methods in the example considered.     

Joint estimation of multiple high‐dimensional Gaussian copula graphical models

A joint estimation approach for multiple high‐dimensional Gaussian copula graphical models is proposed, which achieves estimation robustness by exploiting non‐parametric rank‐based correlation coefficient estimators. Although we focus on continuous data in this paper, the proposed method can be extended to deal with binary or mixed data. Based on a weighted minimisation problem, the estimators can be obtained by implementing second‐order cone programming. Theoretical properties of the procedure are investigated. We show that the proposed joint estimation procedure leads to a faster convergence rate than estimating the graphs individually. It is also shown that the proposed procedure achieves an exact graph structure recovery with probability tending to 1 under certain regularity conditions. Besides theoretical analysis, we conduct numerical simulations to compare the estimation performance and graph recovery performance of some state‐of‐the‐art methods including both joint estimation methods and estimation methods for individuals. The proposed method is then applied to a gene expression data set, which illustrates its practical usefulness.     

Model selection using union-intersection principle for non nested models

In this article, we have extended the Vuong’s (1989 Vuong, Q.H. (1989). Likelihood ratio tests for model selection and non-nested hypothesis. Econometrica. 57:307–333.[Crossref], [Web of Science ®] , [Google Scholar]) model selection test to three models in accordance to union-intersection principle. Using the Kullback–Leibler criterion to measure the closeness of a model to the truth, we propose a simple likelihood ratio-based statistics for testing the null hypothesis that the competing models are equally close to the true data-generating process against the alternative hypothesis that at least one model is closer. We show that the distribution of the test statistic is asymptotically equal to the distribution of the maximum of dependent random variables with bivariate folded standard normal distribution. The density function of the maximum of dependent random variables with elliptically contoured distributions has been obtained by other researchers, but, not for distributions which do not belong to the elliptically contoured distributions family. In this article, the exact distribution of the maximum of dependent random variables with bivariate folded standard normal distribution is calculated as an asymptotic distribution of the proposed test statistic. The test is directional and is derived successively for the cases where the competing models are non nested and whether three, two, one, or none of them are misspecified.     

Robust estimation in the generalized poisson model

A computationally simple method of robust estimation in the generalized Poisson model is presented. Estimators are proved to be optimal in the sense of local minimax testing, conditionally on the explanatory variable. Results of a Monte Carlo experiment are supplemented where robust and efficient estimators are compared.     

A Multivariate Extension of Inverse Gaussian Distribution Derived from Inverse Relationship

Abstract

We propose a new multivariate extension of the inverse Gaussian distribution derived from a certain multivariate inverse relationship. First we define a multivariate extension of the inverse relationship between two sets of multivariate distributions, then define a reduced inverse relationship between two multivariate distributions. We derive the multivariate continuous distribution that has the reduced multivariate inverse relationship with a multivariate normal distribution and call it a multivariate inverse Gaussian distribution. This distribution is also characterized as the distribution of the location of a multivariate Brownian motion at some stopping time. The marginal distribution in one direction is the inverse Gaussian distribution, and the conditional distribution in the space perpendicular to this direction is a multivariate normal distribution. Mean, variance, and higher order cumulants are derived from the multivariate inverse relationship with a multivariate normal distribution. Other properties such as reproductivity and infinite divisibility are also given.     

Handling the effect of non-response in graphical models for longitudinal data

In this paper we investigated the use of attrition weights to cope with non-response when selecting graphical chain models for longitudinal data. We proposed a parametric bootstrap approach to account for the extra variability introduced by the estimation of the weights and compared this with results using standard test procedures.     

Understanding the Kalman Filter

This is an expository article. Here we show how the successfully used Kalman filter, popular with control engineers and other scientists, can be easily understood by statisticians if we use a Bayesian formulation and some well-known results in multivariate statistics. We also give a simple example illustrating the use of the Kalman filter for quality control work.     

Computation of the quadrivariate and pentavariate normal cumulative distribution functions

This article provides explicit integration rules for the quadrivariate and the pentavariate normal distribution. By analytically reducing the dimension of the problem and simplifying the functions to be integrated, these rules form the basis for a numerical evaluation scheme yielding an observed maximum error in the order of 10^{? 7} and a computational time of less than 10^{? 6} s. The implementation is very straightforward as it is based on a classical Gauss–Legendre quadrature. Order statistics are also dealt with.     

Global prior distributions for the analysis of discrete graphical models

Summary We propose a new class of prior distributions for the analysis of discrete graphical models. Such a class, obtained following a conditional approach, generalizes the hyper Dirichlet distributions of Dawid and Lauritzen (1993), since it can be extended to non decomposable graphical models. The two classes are compared in terms of model selection, with an application to a medical data-set illustrating the performance of the two resulting procedures. The proposed class turns out to select simpler, more par-simonious structures.     

On the Admissibility of Linear Estimators in a Multivariate Normal Distribution Under LINEX Loss Function

Consider an estimation problem of a linear combination of population means in a multivariate normal distribution under LINEX loss function. Necessary and sufficient conditions for linear estimators to be admissible are given. Further, it is shown that the result is an extension of the quadratic loss case as well as the univariate normal case.     

Performance of information criteria for spatial models

Model choice is one of the most crucial aspect in any statistical data analysis. It is well known that most models are just an approximation to the true data-generating process but among such model approximations, it is our goal to select the ‘best’ one. Researchers typically consider a finite number of plausible models in statistical applications, and the related statistical inference depends on the chosen model. Hence, model comparison is required to identify the ‘best’ model among several such candidate models. This article considers the problem of model selection for spatial data. The issue of model selection for spatial models has been addressed in the literature by the use of traditional information criteria-based methods, even though such criteria have been developed based on the assumption of independent observations. We evaluate the performance of some of the popular model selection critera via Monte Carlo simulation experiments using small to moderate samples. In particular, we compare the performance of some of the most popular information criteria such as Akaike information criterion (AIC), Bayesian information criterion, and corrected AIC in selecting the true model. The ability of these criteria to select the correct model is evaluated under several scenarios. This comparison is made using various spatial covariance models ranging from stationary isotropic to nonstationary models.     

A note on connectedness in fixed effects manova and gmanova models withmissing cells

Murray and Smith (1985) and Hocking (1985) give a generalized definition and test of connectedness in the case of missing cells using the univariate cell-means model with linear restrictions on the cell-means. The test of connectedness is here extended to multivariate fixed effects models, including the usual MANOVA model with linear restrictions, the MANOVA model with double linear restrictions, and the GMANOVA model.     

A Sequential Multi-Hypothesis Test for the Mean Function of a Discrete-Time Gaussian Process

Abstract

A sequential multi-hypothesis test for the mean function of a discrete-time Gaussian process with known covariance kernel is developed. It is obtained by applying the Bechhofer-Kiefer-Sobel generalized sequential probability ratio test GSPRT, and its properties are studied analytically. Selected applications to i.i.d. normal random variables, observation in a time series AR(1) model, and Wiener processes are given.     

On the use of area-wide models in the Euro-zone

This paper investigates the legitimacy of using area-wide models in predicting aggregate variables in the Euro-area. We aim to compare the performance of area-wide versus national specific models for modeling money demand when using different aggregation schemes. A generalized Grunfeld and Griliches criterion and the Vuong test are used to discriminate between competitive models. Results show that the use of different aggregation methods is not irrelevant. In fact, due to the volatility of the exchange rates, the aggregate models fit better than the disaggregate whenever we employ ECU exchange rates. However, for fixed exchange rates expressed in Euro, the disaggregate models outperform the aggregate ones. This paper was written during my visiting research period at the Department of Economics, University of Southampton. I wish to thank John Aldrich, Jan Podivinsky, Grayham Mizon and Akos Valentinyi. Financial support of the Universita degli Studi “Roma Tre” and the Marie Curie fellowship (HPMT-CT-2001-00353) are gratefully acknowledged.