Fast sampling of Gaussian Markov random fields

This paper demonstrates how Gaussian Markov random fields (conditional autoregressions) can be sampled quickly by using numerical techniques for sparse matrices. The algorithm is general and efficient, and expands easily to various forms for conditional simulation and evaluation of normalization constants. We demonstrate its use by constructing efficient block updates in Markov chain Monte Carlo algorithms for disease mapping.     

Network-based genomic discovery: application and comparison of Markov random field models

Summary.  As biological knowledge accumulates rapidly, gene networks encoding genomewide gene–gene interactions have been constructed. As an improvement over the standard mixture model that tests all the genes identically and independently distributed a priori , Wei and co-workers have proposed modelling a gene network as a discrete or Gaussian Markov random field (MRF) in a mixture model to analyse genomic data. However, how these methods compare in practical applications is not well understood and this is the aim here. We also propose two novel constraints in prior specifications for the Gaussian MRF model and a fully Bayesian approach to the discrete MRF model. We assess the accuracy of estimating the false discovery rate by posterior probabilities in the context of MRF models. Applications to a chromatin immuno-precipitation–chip data set and simulated data show that the modified Gaussian MRF models have superior performance compared with other models, and both MRF-based mixture models, with reasonable robustness to misspecified gene networks, outperform the standard mixture model.     

Approximating hidden Gaussian Markov random fields

Summary.  Gaussian Markov random-field (GMRF) models are frequently used in a wide variety of applications. In most cases parts of the GMRF are observed through mutually independent data; hence the full conditional of the GMRF, a hidden GMRF (HGMRF), is of interest. We are concerned with the case where the likelihood is non-Gaussian, leading to non-Gaussian HGMRF models. Several researchers have constructed block sampling Markov chain Monte Carlo schemes based on approximations of the HGMRF by a GMRF, using a second-order expansion of the log-density at or near the mode. This is possible as the GMRF approximation can be sampled exactly with a known normalizing constant. The Markov property of the GMRF approximation yields computational efficiency.The main contribution in the paper is to go beyond the GMRF approximation and to construct a class of non-Gaussian approximations which adapt automatically to the particular HGMRF that is under study. The accuracy can be tuned by intuitive parameters to nearly any precision. These non-Gaussian approximations share the same computational complexity as those which are based on GMRFs and can be sampled exactly with computable normalizing constants. We apply our approximations in spatial disease mapping and model-based geostatistical models with different likelihoods, obtain procedures for block updating and construct Metropolized independence samplers.     

Tests for random time effects and spatial error correlation in panel regression models

We consider a panel model with spatial autocorrelation and heterogeneity across time. Various Lagrange multiplier and likelihood ratio test statistics are developed for testing time effects and spatial effects, jointly, marginally or conditionally. Limiting null distributions of the tests are derived. Size and power performances of the proposed tests are compared by a Monte-Carlo experiment.     

Modeling material stress using integrated Gaussian Markov random fields

The equations of a physical constitutive model for material stress within tantalum grains were solved numerically using a tetrahedrally meshed volume. The resulting output included a scalar vonMises stress for each of the more than 94,000 tetrahedra within the finite element discretization. In this paper, we define an intricate statistical model for the spatial field of vonMises stress which uses the given grain geometry in a fundamental way. Our model relates the three-dimensional field to integrals of latent stochastic processes defined on the vertices of the one- and two-dimensional grain boundaries. An intuitive neighborhood structure of the said boundary nodes suggested the use of a latent Gaussian Markov random field (GMRF). However, despite the potential for computational gains afforded by GMRFs, the integral nature of our model and the sheer number of data points pose substantial challenges for a full Bayesian analysis. To overcome these problems and encourage efficient exploration of the posterior distribution, a number of techniques are now combined: parallel computing, sparse matrix methods, and a modification of a block update strategy within the sampling routine. In addition, we use an auxiliary variables approach to accommodate the presence of outliers in the data.     

Bayesian inference for generalized additive mixed models based on Markov random field priors

Most regression problems in practice require flexible semiparametric forms of the predictor for modelling the dependence of responses on covariates. Moreover, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal or spatial data. We present a unified approach for Bayesian inference via Markov chain Monte Carlo simulation in generalized additive and semiparametric mixed models. Different types of covariates, such as the usual covariates with fixed effects, metrical covariates with non-linear effects, unstructured random effects, trend and seasonal components in longitudinal data and spatial covariates, are all treated within the same general framework by assigning appropriate Markov random field priors with different forms and degrees of smoothness. We applied the approach in several case-studies and consulting cases, showing that the methods are also computationally feasible in problems with many covariates and large data sets. In this paper, we choose two typical applications.     

Testing for random effects and serial correlation in spatial autoregressive models

This paper constructs and evaluates tests for random effects and serial correlation in spatial autoregressive panel data models. In these models, ignoring the presence of random effects not only produces misleading inference but inconsistent estimation of the regression coefficients. Two different estimation methods are considered: maximum likelihood and instrumental variables. For each estimator, optimal tests are constructed: Lagrange multiplier in the first case; Neyman's C(α)$C(\alpha )$ in the second. In addition, locally size-robust tests, for individual hypotheses under local misspecification of the unconsidered parameter, are constructed. Extensive Monte Carlo evidence is presented.     

Bayesian local influence analysis of skew-normal spatial dynamic panel data models

The existing studies on spatial dynamic panel data model (SDPDM) mainly focus on the normality assumption of response variables and random effects. This assumption may be inappropriate in some applications. This paper proposes a new SDPDM by assuming that response variables and random effects follow the multivariate skew-normal distribution. A Markov chain Monte Carlo algorithm is developed to evaluate Bayesian estimates of unknown parameters and random effects in skew-normal SDPDM by combining the Gibbs sampler and the Metropolis–Hastings algorithm. A Bayesian local influence analysis method is developed to simultaneously assess the effect of minor perturbations to the data, priors and sampling distributions. Simulation studies are conducted to investigate the finite-sample performance of the proposed methodologies. An example is illustrated by the proposed methodologies.     

Linear scalar-on-surface random effects regression models

Many research fields increasingly involve analyzing data of a complex structure. Models investigating the dependence of a response on a predictor have moved beyond the ordinary scalar-on-vector regression. We propose a regression model for a scalar response and a surface (or a bivariate function) predictor. The predictor has a random component and the regression model falls in the framework of linear random effects models. We estimate the model parameters via maximizing the log-likelihood with the ECME (Expectation/Conditional Maximization Either) algorithm. We use the approach to analyze a data set where the response is the neuroticism score and the predictor is the resting-state brain function image. In the simulations we tried, the approach has better performance than two other approaches, a functional principal component regression approach and a smooth scalar-on-image regression approach.     

Quantile regression for panel data models with fixed effects under random censoring

Abstract

The locally weighted censored quantile regression approach is proposed for panel data models with fixed effects, which allows for random censoring. The resulting estimators are obtained by employing the fixed effects quantile regression method. The weights are selected either parametrically, semi-parametrically or non-parametrically. The large panel data asymptotics are used in an attempt to cope with the incidental parameter problem. The consistency and limiting distribution of the proposed estimator are also derived. The finite sample performance of the proposed estimators are examined via Monte Carlo simulations.     

Centered parameterizations and dependence limitations in Markov random field models

Markov random field models incorporate terms representing local statistical dependence among variables in a discrete-index random field. Traditional parameterizations for models based on one-parameter exponential family conditional distributions contain components that would appear to reflect large-scale and small-scale model behaviors, and it is natural to attempt to match these structures with large-scale and small-scale patterns in a set of data. Traditional manners of parameterizing Markov random field models do not allow such correspondence, however. We propose an alternative centered parameterization that, while not leading to different models, allows a correspondence between model structures and data structures to be successfully accomplished. The ability to make these connections is important when incorporating covariate information into a model or if a sequence of models is fit over time to investigate and interpret possible changes in data structure. We demonstrate the improved interpretation that results from use of centered parameterizations. Centered parameterizations also lend themselves to computation of an interpretable decomposition of mean squared error, and this is demonstrated both analytically and through a simulated example. A breakdown in model behavior occurs even with centered parameterizations if dependence parameters in Markov random field models are allowed to become too large. This phenomenon is discussed and illustrated using an auto-logistic model.     

Selection of the neighborhood structure for space-time Markov random field models

A space-time, univariate dataset is assumed to have been sampled from a 3-dimensional Markov Random Field where the data dependence structure is modeled through pairwise interaction parameters. The likelihood function depends upon (1) an undirected, 3-dimensional graph, where edges connect observation points, and (2) the parameter dimension that captures possible space-time anisotropy of data interaction. Automatic model selection to discriminate both the graph and the model dimension is suggested on the basis of a penalized Pseudo-likelihood function. In most cases, the procedure can be implemented using standard statistical packages capable of GLM estimation. Weak consistency of the criterion is shown to hold under mild and easily verifiable sufficient conditions. Its performance in small samples is studied providing simulation results.     

A study on geographically weighted spatial autoregression models with spatial autoregressive disturbances

Abstract

Spatial heterogeneity and correlation are both considered in the geographical weighted spatial autoregressive model. At present, this kind of model has aroused the attention of some scholars. For the estimation of the model, the existing research is based on the assumption that the error terms are independent and identically distributed. In this article we use a computationally simple procedure for estimating the model with spatially autoregressive disturbance terms, both the estimates of constant coefficients and variable coefficients are obtained. Finally, we give the large sample properties of the estimators under some ordinary conditions. In addition, application study of the estimation methods involved will be further explored in a separate study.     

Robust estimation for functional coefficient regression models with spatial data

A global smoothing procedure is developed using B-spline function approximation for estimating the unknown functions of a functional coefficient regression model with spatial data. A general formulation is used to treat mean regression, median regression, quantile regression and robust mean regression in one setting. The global convergence rates of the estimators of unknown coefficient functions are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are given. Finite sample properties of our procedures are studied through Monte Carlo simulations. A housing data example is used to illustrate the proposed methodology.     

Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models

Markov chain Monte Carlo (MCMC) implementations of Bayesian inference for latent spatial Gaussian models are very computationally intensive, and restrictions on storage and computation time are limiting their application to large problems. Here we propose various parallel MCMC algorithms for such models. The algorithms' performance is discussed with respect to a simulation study, which demonstrates the increase in speed with which the algorithms explore the posterior distribution as a function of the number of processors. We also discuss how feasible problem size is increased by use of these algorithms.     

Welfare gains of the poor: An endogenous Bayesian approach with spatial random effects

We introduce a Bayesian instrumental variable procedure with spatial random effects that handles endogeneity, and spatial dependence with unobserved heterogeneity. We find through a limited Monte Carlo experiment that our proposal works well in terms of point estimates and prediction. We apply our method to analyze the welfare effects generated by a process of electricity tariff unification on the poorest households. In particular, we deduce an Equivalent Variation measure where there is a budget constraint for a two-tiered pricing scheme, and find that 10% of the poorest municipalities attained welfare gains above 2% of their initial income.     

Influence diagnostics in Gaussian spatial linear models

Spatial linear models have been applied in numerous fields such as agriculture, geoscience and environmental sciences, among many others. Spatial dependence structure modelling, using a geostatistical approach, is an indispensable tool to estimate the parameters that define this structure. However, this estimation may be greatly affected by the presence of atypical observations in the sampled data. The purpose of this paper is to use diagnostic techniques to assess the sensitivity of the maximum-likelihood estimators, covariance functions and linear predictor to small perturbations in the data and/or the spatial linear model assumptions. The methodology is illustrated with two real data sets. The results allowed us to conclude that the presence of atypical values in the sample data have a strong influence on thematic maps, changing the spatial dependence structure.     

Computational aspects of the EM algorithm for spatial econometric models with missing data

Maximum likelihood (ML) estimation with spatial econometric models is a long-standing problem that finds application in several areas of economic importance. The problem is particularly challenging in the presence of missing data, since there is an implied dependence between all units, irrespective of whether they are observed or not. Out of the several approaches adopted for ML estimation in this context, that of LeSage and Pace [Models for spatially dependent missing data. J Real Estate Financ Econ. 2004;29(2):233–254] stands out as one of the most commonly used with spatial econometric models due to its ability to scale with the number of units. Here, we review their algorithm, and consider several similar alternatives that are also suitable for large datasets. We compare the methods through an extensive empirical study and conclude that, while the approximate approaches are suitable for large sampling ratios, for small sampling ratios the only reliable algorithms are those that yield exact ML or restricted ML estimates.