Sampling hyperparameters in hierarchical models: Improving on Gibbs for high-dimensional latent fields and large datasets

ABSTRACT

A common Bayesian hierarchical model is where high-dimensional observed data depend on high-dimensional latent variables that, in turn, depend on relatively few hyperparameters. When the full conditional distribution over latent variables has a known form, general MCMC sampling need only be performed on the low-dimensional marginal posterior distribution over hyperparameters. This improves on popular Gibbs sampling that computes over the full space. Sampling the marginal posterior over hyperparameters exhibits good scaling of compute cost with data size, particularly when that distribution depends on a low-dimensional sufficient statistic.     

Another Proof of Borel's Strong Law of Large Numbers

Let X be a continuous nonnegative random variable with finite first and second moments and a continuous pdf that is positive on the interior of its support. A nonzero limiting density at the origin and a coefficient of variation (CV) greater than 1 are shown to be sufficient conditions for the distribution truncated below at t > 0 to have a variance greater than the variance of the full distribution. Distributions that satisfy these conditions include those with decreasing hazard rates (e.g., the gamma and Weibull distributions with shape parameters less than 1) and the beta distribution with parameter values p and q for which q > p(p + q + 1). The bound T for which truncation at 0 < t < T increases the variance relative to the full distribution is shown to be greater than the (1 — 1/CV)th percentile of the full distribution.     

几何分布产品不完全数据场合下的统计分析

几何分布是离散型寿命分布中最为重要的分布之一,许多产品的寿命（比如开关等）都可以用几何分布来描述。由于几何分布的无记忆性,它在可靠性理论与应用概率模型中有着非常重要的地位。目前,对关于几何分布在全样本场合、截尾样本场合以及加速寿命试验场合下参数的统计分析已经有了广泛的研究。并且有着重要的理论与应用价值。因此将不完全数据场合下的几何分布问题转化为指数分布问题,再利用指数分布的已有结果首次得到了几何分布在缺失数据场合和分组数据场合下参数的近似点估计,Monte—Carlo模拟算例结果令人满意,说明该方法是可行的。     

A Bayesian semiparametric method for analyzing length-biased data

Survival data obtained from prevalent cohort study designs are often subject to length-biased sampling. Frequentist methods including estimating equation approaches, as well as full likelihood methods, are available for assessing covariate effects on survival from such data. Bayesian methods allow a perspective of probability interpretation for the parameters of interest, and may easily provide the predictive distribution for future observations while incorporating weak prior knowledge on the baseline hazard function. There is lack of Bayesian methods for analyzing length-biased data. In this paper, we propose Bayesian methods for analyzing length-biased data under a proportional hazards model. The prior distribution for the cumulative hazard function is specified semiparametrically using I-Splines. Bayesian conditional and full likelihood approaches are developed for analyzing simulated and real data.     

A Bayesian Shrinkage Model for Incomplete Longitudinal Binary Data with Application to the Breast Cancer Prevention Trial

We consider inference in randomized longitudinal studies with missing data that is generated by skipped clinic visits and loss to follow-up. In this setting, it is well known that full data estimands are not identified unless unverified assumptions are imposed. We assume a non-future dependence model for the drop-out mechanism and partial ignorability for the intermittent missingness. We posit an exponential tilt model that links non-identifiable distributions and distributions identified under partial ignorability. This exponential tilt model is indexed by non-identified parameters, which are assumed to have an informative prior distribution, elicited from subject-matter experts. Under this model, full data estimands are shown to be expressed as functionals of the distribution of the observed data. To avoid the curse of dimensionality, we model the distribution of the observed data using a Bayesian shrinkage model. In a simulation study, we compare our approach to a fully parametric and a fully saturated model for the distribution of the observed data. Our methodology is motivated by, and applied to, data from the Breast Cancer Prevention Trial.     

Efficient estimation in a regression model with missing responses

This article examines methods to efficiently estimate the mean response in a linear model with an unknown error distribution under the assumption that the responses are missing at random. We show how the asymptotic variance is affected by the estimator of the regression parameter, and by the imputation method. To estimate the regression parameter, the ordinary least squares is efficient only if the error distribution happens to be normal. If the errors are not normal, then we propose a one step improvement estimator or a maximum empirical likelihood estimator to efficiently estimate the parameter.To investigate the imputation’s impact on the estimation of the mean response, we compare the listwise deletion method and the propensity score method (which do not use imputation at all), and two imputation methods. We demonstrate that listwise deletion and the propensity score method are inefficient. Partial imputation, where only the missing responses are imputed, is compared to full imputation, where both missing and non-missing responses are imputed. Our results reveal that, in general, full imputation is better than partial imputation. However, when the regression parameter is estimated very poorly, the partial imputation will outperform full imputation. The efficient estimator for the mean response is the full imputation estimator that utilizes an efficient estimator of the parameter.     

On a criterion for extrapolation in multivariate normal regression

In a full (or less than full) rank normal multivariate regression model, the existence of an unbiased estimate of the distribution associated to any future observation vector is studied. A necessary and sufficient condition in terms of the point at which the future observation vector will be taken introduced to consider the validity of extrapolation. The results given here generalize some results of Silvey [3] and the related results of O’Reilly [1].     

MAXIMUM LIKELIHOOD ESTIMATION FOR GENERALISED LOGISTIC DISTRIBUTIONS

ABSTRACT

Maximum likelihood estimation for the type I generalised logistic distributions is investigated. We show that the maximum likelihood estimation usually exists, except when the so-called embedded model problem occurs. A full set of embedded distributions is derived, including Gumbel distribution and a two-parameter reciprocal exponential distribution. Properties relating the embedded distributions are given. We also provide criteria to determine when the embedded distribution occurs. Examples are given for illustration.     

Estimating duration distribution aided by auxiliary longitudinal measures in presence of missing time origin

Lifetime Data Analysis - Understanding the distribution of an event duration time is essential in many studies. The exact time to the event is often unavailable, and thus so is the full event...     

A full likelihood procedure of exchangeable negative binomials for modelling correlated and overdispersed count data

This paper introduces an exchangeable negative binomial distribution resulting from relaxing the independence of the Bernoulli sequence associated with a negative binomial distribution to exchangeability. It is demonstrated that the introduced distribution is a mixture of negative binomial distributions and can be characterized by infinitely many parameters that form a completely monotone sequence. The moments of the distribution are derived and a small simulation is conducted to illustrate the distribution. For data analytic purposes, two methods, truncation and completely-monotone links, are given for converting the saturated distribution of infinitely many parameters to parsimonious distributions of finitely many parameters. A full likelihood procedure is described which can be used to investigate correlated and overdispersed count data common in biomedical sciences and teratology. In the end, the introduced distribution is applied to analyze a real clinical data of burn wounds on patients.     

Bayesian penalized smoothing approaches in models specified using differential equations with unknown error distributions

A full Bayesian approach based on ordinary differential equation (ODE)-penalized B-splines and penalized Gaussian mixture is proposed to jointly estimate ODE-parameters, state function and error distribution from the observation of some state functions involved in systems of affine differential equations. Simulations inspired by pharmacokinetic (PK) studies show that the proposed method provides comparable results to the method based on the standard ODE-penalized B-spline approach (i.e. with the Gaussian error distribution assumption) and outperforms the standard ODE-penalized B-splines when the distribution is not Gaussian. This methodology is illustrated on a PK data set.     

A Note on Distinguishing Random Trees Populations

ABSTRACT

Several new results are presented for a class of univariate distributions for which the maximum likelihood estimate of the population mean is the sample mean. It is shown that the convolution of any two such distributions also belongs to this class of functions. It is also shown that the marginal distribution for the sample mean captures all of the Fisher information for the population mean contained in the full distribution. Parameters orthogonal to the mean are found for special cases of these distributions. If the distribution is conditioned on the sample mean, the conditional distribution depends on the parameters only through parameters orthogonal to the mean.     

Bayesian analysis in multivariate regression models with conjugate priors

In this paper, we consider the full rank multivariate regression model with matrix elliptically contoured distributed errors. We formulate a conjugate prior distribution for matrix elliptical models and derive the posterior distributions of mean and scale matrices. In the sequel, some characteristics of regression matrix parameters are also proposed.     

A Monte Carlo study of some limited and full information simultaneous equation estimators with normal and nonnormal autocorrelated disturbances

The purpose of this paper is to examine the small sample properties of various limited and full information estimators of the structural coefficients of a system of two equations. Specifically, we consider a first-order autoregressive error structure under normal and nonnormal disturbances — for four different covariance structures — and report on a Monte Carlo study of the small sample behavior of limited and full information estimators according to the criteria of bias and dispersion. The results show that the differences in performance of the estimators for the alternative forms of the disturbance distributions are large. Moreover, none of the examined estimators is superior relative to the others, in the sense that its bias and dispersion are the smallest for at least one form of the disturbance distribution. Finally, no combination of highly or lowly autocorrelated disturbances favors some specific limited or full information estimator.     

The truncated generalized poisson distribution and its estimation

A discrete probability model always gets truncated during the sampling process and the point of truncation depends upon the sample size. Also, the generalized Poisson distribution cannot be used with full justification when the second parameter is negative. To avoid these problems a truncated generalized Poisson distribution is defined and studied. Estimation of its parameters by moments method, maximum likelihood method and a mixed method are considered. Some examples are given to illustrate the effect on the parameters’ estimates when a non-truncated GPD is used instead of a truncated GPD.     

AN EXACT TEST FOR HAZARD SIMILARITY

An exact test is developed for hazard similarity and in particular for exponentiality. This test is distinct from more common goodness-of-fit tests such as the Kolmogorov–Smirnov goodness-of-fit test, as it does not require full specification of the null distribution. This test is obtained through a characterization of hazard-similar distributions and a generalization of Fisher's test for association.     

C219. Comment on ”calculating binomial probabilities when the trial probabilities are unequal“

In complex models like hidden Markov chains, the convergence of the MCMC algorithms used to approximate the posterior distribution and the Bayes estimates of the parameters of interest must be controlled in a robust manner. We propose in this paper a series of online controls, which rely on classical non-parametric tests, to evaluate independence from the start-up distribution, stability of the Markov chain, and asymptotic normality. These tests lead to graphical control spreadsheets which arepresentedin the set-up of normalmixture hidden Markov chains to compare the full Gibbs sampler with an aggregated Gibbs sampler based on the forward – backward formulas.     

Goodness-of-fit testing by transforming to normality: comparison between classical and characteristic function-based methods

Chen and Balakrishnan [Chen, G. and Balakrishnan, N., 1995, A general purpose approximate goodness-of-fit test. Journal of Quality Technology, 27, 154–161] proposed an approximate method of goodness-of-fit testing that avoids the use of extensive tables. This procedure first transforms the data to normality, and subsequently applies the classical tests for normality based on the empirical distribution function, and critical points thereof. In this paper, we investigate the potential of this method in comparison to a corresponding goodness-of-fit test which instead of the empirical distribution function, utilizes the empirical characteristic function. Both methods are in full generality as they may be applied to arbitrary laws with continuous distribution function, provided that an efficient method of estimation exists for the parameters of the hypothesized distribution.     

Parallel Markov chain Monte Carlo for Bayesian hierarchical models with big data,in two stages

Due to the escalating growth of big data sets in recent years, new Bayesian Markov chain Monte Carlo (MCMC) parallel computing methods have been developed. These methods partition large data sets by observations into subsets. However, for Bayesian nested hierarchical models, typically only a few parameters are common for the full data set, with most parameters being group specific. Thus, parallel Bayesian MCMC methods that take into account the structure of the model and split the full data set by groups rather than by observations are a more natural approach for analysis. Here, we adapt and extend a recently introduced two-stage Bayesian hierarchical modeling approach, and we partition complete data sets by groups. In stage 1, the group-specific parameters are estimated independently in parallel. The stage 1 posteriors are used as proposal distributions in stage 2, where the target distribution is the full model. Using three-level and four-level models, we show in both simulation and real data studies that results of our method agree closely with the full data analysis, with greatly increased MCMC efficiency and greatly reduced computation times. The advantages of our method versus existing parallel MCMC computing methods are also described.     

Bayesian quantile regression for longitudinal data models

In this paper, we discuss a fully Bayesian quantile inference using Markov Chain Monte Carlo (MCMC) method for longitudinal data models with random effects. Under the assumption of error term subject to asymmetric Laplace distribution, we establish a hierarchical Bayesian model and obtain the posterior distribution of unknown parameters at τ-th level. We overcome the current computational limitations using two approaches. One is the general MCMC technique with Metropolis–Hastings algorithm and another is the Gibbs sampling from the full conditional distribution. These two methods outperform the traditional frequentist methods under a wide array of simulated data models and are flexible enough to easily accommodate changes in the number of random effects and in their assumed distribution. We apply the Gibbs sampling method to analyse a mouse growth data and some different conclusions from those in the literatures are obtained.