Modeling Longitudinal Obesity Data with Intermittent Missingness Using a New Latent Variable Model

We propose a latent variable model for informative missingness in longitudinal studies which is an extension of latent dropout class model. In our model, the value of the latent variable is affected by the missingness pattern and it is also used as a covariate in modeling the longitudinal response. So the latent variable links the longitudinal response and the missingness process. In our model, the latent variable is continuous instead of categorical and we assume that it is from a normal distribution. The EM algorithm is used to obtain the estimates of the parameter we are interested in and Gauss–Hermite quadrature is used to approximate the integration of the latent variable. The standard errors of the parameter estimates can be obtained from the bootstrap method or from the inverse of the Fisher information matrix of the final marginal likelihood. Comparisons are made to the mixed model and complete-case analysis in terms of a clinical trial dataset, which is Weight Gain Prevention among Women (WGPW) study. We use the generalized Pearson residuals to assess the fit of the proposed latent variable model.     

Pointwise and functional approximations in Monte Carlo maximum likelihood estimation

We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.     

Monte Carlo approximation of likelihood function in spatial GLMMs through an empirical Bayes method

In spatial generalized linear mixed models (SGLMMs), statistical inference encounters problems, since random effects in the model imply high-dimensional integrals to calculate the marginal likelihood function. In this article, we temporarily treat parameters as random variables and express the marginal likelihood function as a posterior expectation. Hence, the marginal likelihood function is approximated using the obtained samples from the posterior density of the latent variables and parameters given the data. However, in this setting, misspecification of prior distribution of correlation function parameter and problems associated with convergence of Markov chain Monte Carlo (MCMC) methods could have an unpleasant influence on the likelihood approximation. To avoid these challenges, we utilize an empirical Bayes approach to estimate prior hyperparameters. We also use a computationally efficient hybrid algorithm by combining inverse Bayes formula (IBF) and Gibbs sampler procedures. A simulation study is conducted to assess the performance of our method. Finally, we illustrate the method applying a dataset of standard penetration test of soil in an area in south of Iran.     

Sampling of pairs in pairwise likelihood estimation for latent variable models with categorical observed variables

Pairwise likelihood is a limited information estimation method that has also been used for estimating the parameters of latent variable and structural equation models. Pairwise likelihood is a special case of composite likelihood methods that uses lower-order conditional or marginal log-likelihoods instead of the full log-likelihood. The composite likelihood to be maximized is a weighted sum of marginal or conditional log-likelihoods. Weighting has been proposed for increasing efficiency, but the choice of weights is not straightforward in most applications. Furthermore, the importance of leaving out higher-order scores to avoid duplicating lower-order marginal information has been pointed out. In this paper, we approach the problem of weighting from a sampling perspective. More specifically, we propose a sampling method for selecting pairs based on their contribution to the total variance from all pairs. The sampling approach does not aim to increase efficiency but to decrease the estimation time, especially in models with a large number of observed categorical variables. We demonstrate the performance of the proposed methodology using simulated examples and a real application.

     

On the estimation of normal copula discrete regression models using the continuous extension and simulated likelihood

The continuous extension of a discrete random variable is amongst the computational methods used for estimation of multivariate normal copula-based models with discrete margins. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the asymptotic and small-sample efficiency of two variants of the method for estimating the multivariate normal copula with univariate binary, Poisson, and negative binomial regressions, and show that they lead to biased estimates for the latent correlations, and the univariate marginal parameters that are not regression coefficients. We implement a maximum simulated likelihood method, which is based on evaluating the multidimensional integrals of the likelihood with randomized quasi-Monte Carlo methods. Asymptotic and small-sample efficiency calculations show that our method is nearly as efficient as maximum likelihood for fully specified multivariate normal copula-based models. An illustrative example is given to show the use of our simulated likelihood method.     

Composite Likelihood Estimation for Multivariate Probit Latent Traits Models

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the marginal composite likelihood approach for the probit latent traits models. This method belonging to the broad class of pseudo-likelihood involves marginal pairs probabilities of the responses which has analytical expression. The different results are illustrated with a simulation study and with an analysis of real data from health related quality of life.     

ESTIMATION IN RICKER'S TWO-RELEASE METHOD: A BAYESIAN APPROACH

The Ricker's two‐release method is a simplified version of the Jolly‐Seber method, from Seber's Estimation of Animal Abundance (1982) , used to estimate survival rate and abundance in animal populations. This method assumes there is only a single recapture sample and no immigration, emigration or recruitment. In this paper, we propose a Bayesian analysis for this method to estimate the survival rate and the capture probability, employing Markov chain Monte Carlo methods and a latent variable analysis. The performance of the proposed method is illustrated with a simulation study as well as a real data set. The results show that the proposed method provides favourable inference for the survival rate when compared with the modified maximum likelihood method.     

A latent variable regression model for asymmetric bivariate ordered categorical data

In many areas of medical research, especially in studies that involve paired organs, a bivariate ordered categorical response should be analyzed. Using a bivariate continuous distribution as the latent variable is an interesting strategy for analyzing these data sets. In this context, the bivariate standard normal distribution, which leads to the bivariate cumulative probit regression model, is the most common choice. In this paper, we introduce another latent variable regression model for modeling bivariate ordered categorical responses. This model may be an appropriate alternative for the bivariate cumulative probit regression model, when postulating a symmetric form for marginal or joint distribution of response data does not appear to be a valid assumption. We also develop the necessary numerical procedure to obtain the maximum likelihood estimates of the model parameters. To illustrate the proposed model, we analyze data from an epidemiologic study to identify some of the most important risk indicators of periodontal disease among students 15-19 years in Tehran, Iran.     

Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

In latent variable models parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample’s divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.     

Point and Interval Estimations of Marginal Risk Difference by Logistic Model

We use logistic model to get point and interval estimates of the marginal risk difference in observational studies and randomized trials with dichotomous outcome. We prove that the maximum likelihood estimate of the marginal risk difference is unbiased for finite sample and highly robust to the effects of dispersing covariates. We use approximate normal distribution of the maximum likelihood estimates of the logistic model parameters to get approximate distribution of the maximum likelihood estimate of the marginal risk difference and then the interval estimate of the marginal risk difference. We illustrate application of the method by a real medical example.     

A rare event approach to high-dimensional approximate Bayesian computation

Approximate Bayesian computation (ABC) methods permit approximate inference for intractable likelihoods when it is possible to simulate from the model. However, they perform poorly for high-dimensional data and in practice must usually be used in conjunction with dimension reduction methods, resulting in a loss of accuracy which is hard to quantify or control. We propose a new ABC method for high-dimensional data based on rare event methods which we refer to as RE-ABC. This uses a latent variable representation of the model. For a given parameter value, we estimate the probability of the rare event that the latent variables correspond to data roughly consistent with the observations. This is performed using sequential Monte Carlo and slice sampling to systematically search the space of latent variables. In contrast, standard ABC can be viewed as using a more naive Monte Carlo estimate. We use our rare event probability estimator as a likelihood estimate within the pseudo-marginal Metropolis–Hastings algorithm for parameter inference. We provide asymptotics showing that RE-ABC has a lower computational cost for high-dimensional data than standard ABC methods. We also illustrate our approach empirically, on a Gaussian distribution and an application in infectious disease modelling.     

Properties of maximum likelihood estimates of the ratio of parameters in ordinal response regression models

The properties of a method of estimating the ratio of parameters for ordered categorical response regression models are discussed. If the link function relating the response variable to the linear combination of covariates is unknown then it is only possible to estimate the ratio of regression parameters. This ratio of parameters has a substitutability or relative importance interpretation.

The maximum likelihood estimate of the ratio of parameters, assuming a logistic function (McCullagh, 1980), is found to have very small bias for a wide variety of true link functions. Further it is shown using Monte Carlo simulations that this maximum likelihood estimate, has good coverage properties, even if the link function is incorrectly specified. It is demonstrated that combining adjacent categories to make the response binary can result in an analysis which is appreciably less efficient. The size of the efficiency loss on, among other factors, the marginal distribution in the ordered categories     

On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models

While much used in practice, latent variable models raise challenging estimation problems due to the intractability of their likelihood. Monte Carlo maximum likelihood (MCML), as proposed by Geyer & Thompson (1992 ), is a simulation-based approach to maximum likelihood approximation applicable to general latent variable models. MCML can be described as an importance sampling method in which the likelihood ratio is approximated by Monte Carlo averages of importance ratios simulated from the complete data model corresponding to an arbitrary value of the unknown parameter. This paper studies the asymptotic (in the number of observations) performance of the MCML method in the case of latent variable models with independent observations. This is in contrast with previous works on the same topic which only considered conditional convergence to the maximum likelihood estimator, for a fixed set of observations. A first important result is that when is fixed, the MCML method can only be consistent if the number of simulations grows exponentially fast with the number of observations. If on the other hand, is obtained from a consistent sequence of estimates of the unknown parameter, then the requirements on the number of simulations are shown to be much weaker.     

A mixture latent variable model for modeling mixed data in heterogeneous populations and its applications

Latent variable models are widely used for jointly modeling of mixed data including nominal, ordinal, count and continuous data. In this paper, we consider a latent variable model for jointly modeling relationships between mixed binary, count and continuous variables with some observed covariates. We assume that, given a latent variable, mixed variables of interest are independent and count and continuous variables have Poisson distribution and normal distribution, respectively. As such data may be extracted from different subpopulations, consideration of an unobserved heterogeneity has to be taken into account. A mixture distribution is considered (for the distribution of the latent variable) which accounts the heterogeneity. The generalized EM algorithm which uses the Newton–Raphson algorithm inside the EM algorithm is used to compute the maximum likelihood estimates of parameters. The standard errors of the maximum likelihood estimates are computed by using the supplemented EM algorithm. Analysis of the primary biliary cirrhosis data is presented as an application of the proposed model.     

Robust multivariate transformations to normality: Constructed variables and likelihood ratio tests

The assumption of multivariate normality provides the customary powerful and convenient ways of analysing multivariate data: if the data are not normal, the analysis may often be simplified by an appropriate transformation. In this context, the most widely used test is the likelihood ratio, which requires the maximum likelihood estimate of the transformation parameter for each variable. Given that this estimate can only be found numerically, when the number of variables is large (> 20) it is impossible or infeasible to compute the test. In this paper we introduce alternative tests which do not require the maximum likelihood estimate of the transformation parameters and prove algebraically their relationships. We also give insights both using theoretical arguments and a robust simulation study, based on the forward search algorithm, about the distribution of the tests previously introduced.     

Bayesian marginal inference via candidate's formula

Computing marginal probabilities is an important and fundamental issue in Bayesian inference. We present a simple method which arises from a likelihood identity for computation. The likelihood identity, called Candidate's formula, sets the marginal probability as a ratio of the prior likelihood to the posterior density. Based on Markov chain Monte Carlo output simulated from the posterior distribution, a nonparametric kernel estimate is used to estimate the posterior density contained in that ratio. This derived nonparametric Candidate's estimate requires only one evaluation of the posterior density estimate at a point. The optimal point for such evaluation can be chosen to minimize the expected mean square relative error. The results show that the best point is not necessarily the posterior mode, but rather a point compromising between high density and low Hessian. For high dimensional problems, we introduce a variance reduction approach to ease the tension caused by data sparseness. A simulation study is presented.     

Converting observed likelihood to levels of significance for transformation models

Inference for a scalar parameter in the pressence of nuisance parameters requires high dimensional integrations of the joint density of the pivotal quantities. Recent development in asymptotic methods provides accurate approximations for significance levels and thus confidence intervals for a scalar component parameter. In this paper, a simple, efficient and accurate numerical procedure is first developed for the location model and is then extended to the location-scale model and the linear regression model. This numerical procedure only requires a fine tabulation of the parameter and the observed log likelihood function, which can be either the full, marginal or conditional observed log likelihood function, as input and output is the corresponding significance function. Numerical results showed that this approximation is not only simple but also very accurate. It outperformed the usual approximations such as the signed likelihood ratio statistic, the maximum likelihood estimate and the score statistic.     

Marginal likelihood estimation from the Metropolis output: tips and tricks for efficient implementation in generalized linear latent variable models

The marginal likelihood can be notoriously difficult to compute, and particularly so in high-dimensional problems. Chib and Jeliazkov employed the local reversibility of the Metropolis–Hastings algorithm to construct an estimator in models where full conditional densities are not available analytically. The estimator is free of distributional assumptions and is directly linked to the simulation algorithm. However, it generally requires a sequence of reduced Markov chain Monte Carlo runs which makes the method computationally demanding especially in cases when the parameter space is large. In this article, we study the implementation of this estimator on latent variable models which embed independence of the responses to the observables given the latent variables (conditional or local independence). This property is employed in the construction of a multi-block Metropolis-within-Gibbs algorithm that allows to compute the estimator in a single run, regardless of the dimensionality of the parameter space. The counterpart one-block algorithm is also considered here, by pointing out the difference between the two approaches. The paper closes with the illustration of the estimator in simulated and real-life data sets.     

Generalized Estimating Equations to Binary Probit Model

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the generalized estimating equations for the probit latent traits models. This method belonging to the broad class of semi parametric approaches involves marginal joint moments of order 1 and 2, which has analytical expression. The different results are illustrated with a simulation study.     

Another look at fisher'S solution to the problem of the weighted mean

The marginal likelihood function of the common mean of two normal populations is considered. Transformed versions of the marginal likelihood function are plotted to illustrate the difficulties of the point estimate approach. Conditions for bimodality and asymmetry are also discussed