Sampling hyperparameters in hierarchical models: Improving on Gibbs for high-dimensional latent fields and large datasets |
| |
Authors: | Richard A Norton J Andrés Christen Colin Fox |
| |
Institution: | 1. Department of Mathematics and Statistics, University of Otago, Dunedin, New Zealandrichard.norton@otago.ac.nz;3. Centro de Investigación en Matemáticas (CIMAT), CONACYT, Guanajuato, GT, Mexico;4. Department of Physics, University of Otago, Dunedin, New Zealand |
| |
Abstract: | ABSTRACTA common Bayesian hierarchical model is where high-dimensional observed data depend on high-dimensional latent variables that, in turn, depend on relatively few hyperparameters. When the full conditional distribution over latent variables has a known form, general MCMC sampling need only be performed on the low-dimensional marginal posterior distribution over hyperparameters. This improves on popular Gibbs sampling that computes over the full space. Sampling the marginal posterior over hyperparameters exhibits good scaling of compute cost with data size, particularly when that distribution depends on a low-dimensional sufficient statistic. |
| |
Keywords: | Gibbs sampling hierarchical models hyperparameters marginal algorithm marginal then conditional sampling sampling |
|
|