Bayesian model-averaged regularization for Gaussian graphical models

The graphical lasso has now become a useful tool to estimate high-dimensional Gaussian graphical models, but its practical applications suffer from the problem of choosing regularization parameters in a data-dependent way. In this article, we propose a model-averaged method for estimating sparse inverse covariance matrices for Gaussian graphical models. We consider the graphical lasso regularization path as the model space for Bayesian model averaging and use Markov chain Monte Carlo techniques for the regularization path point selection. Numerical performance of our method is investigated using both simulated and real datasets, in comparison with some state-of-art model selection procedures.     

Stratified Gaussian graphical models

Gaussian graphical models represent the backbone of the statistical toolbox for analyzing continuous multivariate systems. However, due to the intrinsic properties of the multivariate normal distribution, use of this model family may hide certain forms of context-specific independence that are natural to consider from an applied perspective. Such independencies have been earlier introduced to generalize discrete graphical models and Bayesian networks into more flexible model families. Here, we adapt the idea of context-specific independence to Gaussian graphical models by introducing a stratification of the Euclidean space such that a conditional independence may hold in certain segments but be absent elsewhere. It is shown that the stratified models define a curved exponential family, which retains considerable tractability for parameter estimation and model selection.     

On the impact of contaminations in graphical Gaussian models

This paper analyzes the impact of some kinds of contaminant on model selection in graphical Gaussian models. We investigate four different kinds of contaminants, in order to consider the effect of gross errors, model deviations, and model misspecification. The aim of the work is to assess against which kinds of contaminant a model selection procedure for graphical Gaussian models has a more robust behavior. The analysis is based on simulated data. The simulation study shows that relatively few contaminated observations in even just one of the variables can have a significant impact on correct model selection, especially when the contaminated variable is a node in a separating set of the graph.     

Efficient estimation and model selection in large graphical models

We develop a computationally efficient method to determine the interaction structure in a multidimensional binary sample. We use an interaction model based on orthogonal functions, and give a result on independence properties in this model. Using this result we develop an efficient approximation algorithm for estimating the parameters in a given undirected model. To find the best model, we use a heuristic search algorithm in which the structure is determined incrementally. We also give an algorithm for reconstructing the causal directions, if such exist. We demonstrate that together these algorithms are capable of discovering almost all of the true structure for a problem with 121 variables, including many of the directions.     

Fitting Gaussian Markov Random Fields to Gaussian Fields

This paper discusses the following task often encountered in building Bayesian spatial models: construct a homogeneous Gaussian Markov random field (GMRF) on a lattice with correlation properties either as present in some observed data, or consistent with prior knowledge. The Markov property is essential in designing computationally efficient Markov chain Monte Carlo algorithms to analyse such models. We argue that we can restate both tasks as that of fitting a GMRF to a prescribed stationary Gaussian field on a lattice when both local and global properties are important. We demonstrate that using the KullbackLeibler discrepancy often fails for this task, giving severely undesirable behaviour of the correlation function for lags outside the neighbourhood. We propose a new criterion that resolves this difficulty, and demonstrate that GMRFs with small neighbourhoods can approximate Gaussian fields surprisingly well even with long correlation lengths. Finally, we discuss implications of our findings for likelihood based inference for general Markov random fields when global properties are also important.     

Effectiveness of combinations of Gaussian graphical models for model building

Combining statistical models is an useful approach in all the research area where a global picture of the problem needs to be constructed by binding together evidence from different sources [M.S. Massa and S.L. Lauritzen Combining Statistical Models, M. Viana and H. Wynn, eds., American Mathematical Society, Providence, RI, 2010, pp. 239–259]. In this paper, we investigate the effectiveness of combining a fixed number of Gaussian graphical models respecting some consistency assumptions in problems of model building. In particular, we use the meta-Markov combination of Gaussian graphical models as detailed in Massa and Lauritzen and compare model selection results obtained by combining selections over smaller sets of variables with selection results over all variables of interest. In order to do so, we carry out some simulation studies in which different criteria are considered for the selection procedures. We conclude that the combination performs, generally, better than global estimation, is computationally simpler by virtue of having fewer and simpler models to work on, and has an intuitive appeal to a wide variety of contexts.     

Standard errors for the parameters of graphical Gaussian models

The comparison of an estimated parameter to its standard error, the Wald test, is a well known procedure of classical statistics. Here we discuss its application to graphical Gaussian model selection. First we derive the Fisher information matrix and its inverse about the parameters of any graphical Gaussian model. Both the covariance matrix and its inverse are considered and a comparative analysis of the asymptotic behaviour of their maximum likelihood estimators (m.l.e.s) is carried out. Then we give an example of model selection based on the standard errors. The method is shown to produce almost identical inference to likelihood ratio methods in the example considered.     

Network-based genomic discovery: application and comparison of Markov random field models

Summary.  As biological knowledge accumulates rapidly, gene networks encoding genomewide gene–gene interactions have been constructed. As an improvement over the standard mixture model that tests all the genes identically and independently distributed a priori , Wei and co-workers have proposed modelling a gene network as a discrete or Gaussian Markov random field (MRF) in a mixture model to analyse genomic data. However, how these methods compare in practical applications is not well understood and this is the aim here. We also propose two novel constraints in prior specifications for the Gaussian MRF model and a fully Bayesian approach to the discrete MRF model. We assess the accuracy of estimating the false discovery rate by posterior probabilities in the context of MRF models. Applications to a chromatin immuno-precipitation–chip data set and simulated data show that the modified Gaussian MRF models have superior performance compared with other models, and both MRF-based mixture models, with reasonable robustness to misspecified gene networks, outperform the standard mixture model.     

Finding Relevant Linear Manifolds in Classification by Gaussian Mixtures

In this article, we present a strategy for producing low-dimensional projections that maximally separate the classes in Gaussian Mixture Model classification. The most revealing linear manifolds are those along which the classes are maximally separable. Here we consider a particular probability product kernel as a measure of similarity or affinity between the class-conditional distributions. It takes an appealing closed analytical form in the case of Gaussian mixture components. The performance of the proposed strategy has been evaluated on real data.     

Penalized maximum likelihood estimation for Gaussian hidden Markov models

ABSTRACT

The likelihood function of a Gaussian hidden Markov model is unbounded, which is why the maximum likelihood estimator (MLE) is not consistent. A penalized MLE is introduced along with a rigorous consistency proof.     

A latent Gaussian Markov random-field model for spatiotemporal rainfall disaggregation

Summary. Rainfall data are often collected at coarser spatial scales than required for input into hydrology and agricultural models. We therefore describe a spatiotemporal model which allows multiple imputation of rainfall at fine spatial resolutions, with a realistic dependence structure in both space and time and with the total rainfall at the coarse scale consistent with that observed. The method involves the transformation of the fine scale rainfall to a thresholded Gaussian process which we model as a Gaussian Markov random field. Gibbs sampling is then used to generate realizations of rainfall efficiently at the fine scale. Results compare favourably with previous, less elegant methods.     

Approximating hidden Gaussian Markov random fields

Summary.  Gaussian Markov random-field (GMRF) models are frequently used in a wide variety of applications. In most cases parts of the GMRF are observed through mutually independent data; hence the full conditional of the GMRF, a hidden GMRF (HGMRF), is of interest. We are concerned with the case where the likelihood is non-Gaussian, leading to non-Gaussian HGMRF models. Several researchers have constructed block sampling Markov chain Monte Carlo schemes based on approximations of the HGMRF by a GMRF, using a second-order expansion of the log-density at or near the mode. This is possible as the GMRF approximation can be sampled exactly with a known normalizing constant. The Markov property of the GMRF approximation yields computational efficiency.The main contribution in the paper is to go beyond the GMRF approximation and to construct a class of non-Gaussian approximations which adapt automatically to the particular HGMRF that is under study. The accuracy can be tuned by intuitive parameters to nearly any precision. These non-Gaussian approximations share the same computational complexity as those which are based on GMRFs and can be sampled exactly with computable normalizing constants. We apply our approximations in spatial disease mapping and model-based geostatistical models with different likelihoods, obtain procedures for block updating and construct Metropolized independence samplers.     

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Summary.  Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models , where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.     

Specifying a Gaussian Markov Random Field by a Sparse Cholesky Triangle

ABSTRACT

This note discusses the approach of specifying a Gaussian Markov random field (GMRF) by the Cholesky triangle of the precision matrix. A such representation can be made extremely sparse using numerical techniques for incomplete sparse Cholesky factorization, and provide very computational efficient representation for simulating from the GMRF. However, we provide theoretical and empirical justification showing that the sparse Cholesky triangle representation is fragile when conditioning a GMRF on a subset of the variables or observed data, meaning that the computational cost increases.     

The Performance of Gaussian and non Gaussian dynamic models in assessing market risk: The Implications for risk management

The developing markets are more volatile, unstable illiquid, and more prone to the external shocks. The non Gaussian VaR model gives more accurate risk models than Gaussian VaR models. Hence, the purpose of this study is to test if and how non Gaussian VaR models are comparatively better fit for risk modeling of developing markets than the Gaussian VaR models. The study measures the market risk for the daily closing price of Karachi Stock Exchange 100 index over the period of 2004–2016. The evaluation of VaR models suggests that non Gaussian dynamic model outperformed the Gaussian VaR models in developing markets.     

Fast sampling of Gaussian Markov random fields

This paper demonstrates how Gaussian Markov random fields (conditional autoregressions) can be sampled quickly by using numerical techniques for sparse matrices. The algorithm is general and efficient, and expands easily to various forms for conditional simulation and evaluation of normalization constants. We demonstrate its use by constructing efficient block updates in Markov chain Monte Carlo algorithms for disease mapping.     

Gaussian models for degradation processes-part I: Methods for the analysis of biomarker data

We present two stochastic models that describe the relationship between biomarker process values at random time points, event times, and a vector of covariates. In both models the biomarker processes are degradation processes that represent the decay of systems over time. In the first model the biomarker process is a Wiener process whose drift is a function of the covariate vector. In the second model the biomarker process is taken to be the difference between a stationary Gaussian process and a time drift whose drift parameter is a function of the covariates. For both models we present statistical methods for estimation of the regression coefficients. The first model is useful for predicting the residual time from study entry to the time a critical boundary is reached while the second model is useful for predicting the latency time from the infection until the time the presence of the infection is detected. We present our methods principally in the context of conducting inference in a population of HIV infected individuals.     

An approximation to cluster size distribution of two Gaussian random fields conjunction with application to FMRI data

In brain mapping, the regions of the brain that are ‘activated’ by a task or external stimulus are detected by thresholding an image of test statistics. Often the experiment is repeated on several different subjects or for several different stimuli on the same subject, and the researcher is interested in the common points in the brain where ‘activation’ occurs in all test statistic images. The conjunction is thus defined as those points in the brain that show ‘activation’ in all images. We are interested in which parts of the conjunction are noise, and which show true activation in all test statistic images. We would expect truly activated regions to be larger than usual, so our test statistic is based on the volume of clusters (connected components) of the conjunction. Our main result is an approximate P-value for this in the case of the conjunction of two Gaussian test statistic images. The results are applied to a functional magnetic resonance experiment in pain perception.     

Modeling material stress using integrated Gaussian Markov random fields

The equations of a physical constitutive model for material stress within tantalum grains were solved numerically using a tetrahedrally meshed volume. The resulting output included a scalar vonMises stress for each of the more than 94,000 tetrahedra within the finite element discretization. In this paper, we define an intricate statistical model for the spatial field of vonMises stress which uses the given grain geometry in a fundamental way. Our model relates the three-dimensional field to integrals of latent stochastic processes defined on the vertices of the one- and two-dimensional grain boundaries. An intuitive neighborhood structure of the said boundary nodes suggested the use of a latent Gaussian Markov random field (GMRF). However, despite the potential for computational gains afforded by GMRFs, the integral nature of our model and the sheer number of data points pose substantial challenges for a full Bayesian analysis. To overcome these problems and encourage efficient exploration of the posterior distribution, a number of techniques are now combined: parallel computing, sparse matrix methods, and a modification of a block update strategy within the sampling routine. In addition, we use an auxiliary variables approach to accommodate the presence of outliers in the data.