Pointwise and functional approximations in Monte Carlo maximum likelihood estimation

We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.     

Generalized Estimating Equations to Binary Probit Model

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the generalized estimating equations for the probit latent traits models. This method belonging to the broad class of semi parametric approaches involves marginal joint moments of order 1 and 2, which has analytical expression. The different results are illustrated with a simulation study.     

A New Estimator of a Circular Median

The INARCH(1) model for overdispersed time series of counts has a simple structure, a parsimonious parametrization, and a great potential for applications in practice. We analyze two approaches to approximate the marginal process distribution: a Markov chain approach and the Poisson–Charlier expansion. Then approaches for estimating the two model parameters are discussed. We derive explicit expressions for the asymptotic distribution of the maximum likelihood and conditional least squares estimators. They are used for constructing simultaneous confidence regions, the finite-sample performance of which is analyzed in a simulation study. A real-data example from economics illustrates the application of the INARCH(1) model.     

Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

In latent variable models parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample’s divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.     

Approximate multivariate conditional inference using the adjusted profile likelihood

The author proposes saddlepoint approximation methods that are adapted to multivariate conditional inference in canonical exponential familles. Several approaches to approximating conditional discrete distributions involve dividing an approximation to the full joint mass function, summed over tail regions of interest, by an approximate marginal density. The author first approximates this conditional likelihood by the adjusted profile likelihood, and then applies a multivariate saddlepoint approximation. He also presents formulas to aid in performing simultaneously the profiling and maximizing steps.     

Another look at fisher'S solution to the problem of the weighted mean

The marginal likelihood function of the common mean of two normal populations is considered. Transformed versions of the marginal likelihood function are plotted to illustrate the difficulties of the point estimate approach. Conditions for bimodality and asymmetry are also discussed     

Discussion on “A comparison of Monte Carlo methods for computing marginal likelihoods of item response theory models”

In this discussion, the sensitivity of the result by the choice of parameters a and b in one of approaches reviewed by the authors to calculate the marginal likelihood for 2 parameter logistic item response theory model is investigated using a small simulation study.     

Regression analysis for discrete event history or failure time data

The paper deals with discrete-time regression models to analyze multistate—multiepisode models for event history data or failure time data collected in follow-up studies, retrospective studies, or longitudinal panels. The models are applicable if the events are not dated exactly but only a time interval is recorded. The models include individual specific parameters to account for unobserved heterogeneity. The explantory variables may be time-varying and random with distributions depending on the observed history of the process. Different estimation procedures are considered: Estimation of structural as well as individual specific parameters by maximization of a joint likelihood function, estimation of the structural parameters by maximization of a conditional likelihood function conditioning on a set of sufficient statistics for the individual specific parameters, and estimation of the structural parameters by maximization of a marginal likelihood function assuming that the individual specific parameters follow a distribution. The advantages and limitations of the different approaches are discussed.     

Statistical Inference for the Multidimensional Mixed Rasch Model

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the GEE approach. This method involves the approximations of the marginal likelihood and joint moments of the variables. It is also proposed an approximate Akaike and Bayesian information criterions based on the approximate marginal likelihood using the estimation of the parameters by the GEE approach. The different results are illustrated with a simulation study and with an analysis of real data from health-related quality of life.     

Analysis of the linear correlation coefficient using pseudo-likelihoods

Summary The problem of the inferential analysis of the linear correlation coefficient of normal bivariate populations is tackled, both from the likelihood and Bayesian viewpoints. In particular it is shown how, using pseudo-likelihood (marginal likelihood function and profile likelihood), hypotheses such asH ₀:ϱ=ϱ₀ andH ₀:ϱ_x=ϱ_y can be verified without prohibitive computation effort. The results of marginal and profile likelihood are compared and it is shown that these two methods are virtually equivalent even for small sample sizes. Furthermore, in suitable conditions, the posterior distribution of the coefficient ϱ can be readily obtained, using the exact form or different approximate formulations of the marginal or profile likelihood. Lastly some possible prior distributions of ϱ are illustrated and some explanatory examples are presented.     

Fisher information in randomly sampled sib pairs and extremely discordant sib pairs in genetic analysis for a quantitative trait locus

The conditional mixture likelihood method using the absolute difference of the trait values of a sib pair to estimate genetic parameters underlies commonly used method in linkage analysis. Here, the statistical properties of the model are examined. The marginal model with a pseudo-likelihood function based on a sample of the absolute difference of sib-traits is also studied. Both approaches are compared numerically. When genotyping is much more expensive than screening a quantitative trait, it is known that extremely discordant sib pairs provide more powerful linkage tests than randomly sampled sib pairs. The Fisher information about genetic parameters contained in extremely discordant sib pairs is calculated using the marginal mixture model. Our results supplement current research showing that extremely discordant sib pairs are powerful for the linkage detection by demonstrating they also contain more information about other genetic parameters.     

Inequalities between expected marginal log‐likelihoods,with implications for likelihood‐based model complexity and comparison measures

A multi‐level model allows the possibility of marginalization across levels in different ways, yielding more than one possible marginal likelihood. Since log‐likelihoods are often used in classical model comparison, the question to ask is which likelihood should be chosen for a given model. The authors employ a Bayesian framework to shed some light on qualitative comparison of the likelihoods associated with a given model. They connect these results to related issues of the effective number of parameters, penalty function, and consistent definition of a likelihood‐based model choice criterion. In particular, with a two‐stage model they show that, very generally, regardless of hyperprior specification or how much data is collected or what the realized values are, a priori, the first‐stage likelihood is expected to be smaller than the marginal likelihood. A posteriori, these expectations are reversed and the disparities worsen with increasing sample size and with increasing number of model levels.     

Objective Bayesian analysis for a spatial model with nugget effects

The focus of this paper is objective priors for spatially correlated data with nugget effects. In addition to the Jeffreys priors and commonly used reference priors, two types of “exact” reference priors are derived based on improper marginal likelihoods. An “equivalence” theorem is developed in the sense that the expectation of any function of the score functions of the marginal likelihood function can be taken under marginal likelihoods. Interestingly, these two types of reference priors are identical.     

The Bayesian elastic net regression

A Bayesian elastic net approach is presented for variable selection and coefficient estimation in linear regression models. A simple Gibbs sampling algorithm was developed for posterior inference using a location-scale mixture representation of the Bayesian elastic net prior for the regression coefficients. The penalty parameters are chosen through an empirical method that maximizes the data marginal likelihood. Both simulated and real data examples show that the proposed method performs well in comparison to the other approaches.     

Forecast Combination and Model Averaging Using Predictive Measures

We extend the standard approach to Bayesian forecast combination by forming the weights for the model averaged forecast from the predictive likelihood rather than the standard marginal likelihood. The use of predictive measures of fit offers greater protection against in-sample overfitting when uninformative priors on the model parameters are used and improves forecast performance. For the predictive likelihood we argue that the forecast weights have good large and small sample properties. This is confirmed in a simulation study and in an application to forecasts of the Swedish inflation rate, where forecast combination using the predictive likelihood outperforms standard Bayesian model averaging using the marginal likelihood.     

Composite Likelihood Estimation for Multivariate Probit Latent Traits Models

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the marginal composite likelihood approach for the probit latent traits models. This method belonging to the broad class of pseudo-likelihood involves marginal pairs probabilities of the responses which has analytical expression. The different results are illustrated with a simulation study and with an analysis of real data from health related quality of life.     

Testtesting for the equality of scale parameters of k(> 2) exponential populations based on complete and type ii censored samples

This paper is concerned with testing the equality of scale parameters of K(> 2) two-parameter exponential distributions in presence of unspecified location parameters based on complete and type II censored samples. We develop a marginal likelihood ratio statistic, a quadratic statistic (Qu) (Nelson, 1982) based on maximum marginal likelihood estimates of the scale parameters under the null and the alternative hypotheses, a C(a) statistic (CPL) (Neyman, 1959) based on the profile likelihood estimate of the scale parameter under the null hypothesis and an extremal scale parameter ratio statistic (ESP) (McCool, 1979). We show that the marginal likelihood ratio statistic is equivalent to the modified Bartlett test statistic. We use Bartlett's small sample correction to the marginal likelihood ratio statistic and call it the modified marginal likelihood ratio statistic (MLB). We then compare the four statistics, MLBi Qut CPL and ESP in terms of size and power by using Monte Carlo simulation experiments. For the variety of sample sizes and censoring combinations and nominal levels considered the statistic MLB holds nominal level most accurately and based on empirically calculated critical values, this statistic performs best or as good as others in most situations. Two examples are given.     

Time-varying coefficient models with ARMA–GARCH structures for longitudinal data analysis

Time-varying coefficient models with autoregressive and moving-average–generalized autoregressive conditional heteroscedasticity structure are proposed for examining the time-varying effects of risk factors in longitudinal studies. Compared with existing models in the literature, the proposed models give explicit patterns for the time-varying coefficients. Maximum likelihood and marginal likelihood (based on a Laplace approximation) are used to estimate the parameters in the proposed models. Simulation studies are conducted to evaluate the performance of these two estimation methods, which is measured in terms of the Kullback–Leibler divergence and the root mean square error. The marginal likelihood approach leads to the more accurate parameter estimates, although it is more computationally intensive. The proposed models are applied to the Framingham Heart Study to investigate the time-varying effects of covariates on coronary heart disease incidence. The Bayesian information criterion is used for specifying the time series structures of the coefficients of the risk factors.     

Dealing with label switching in mixture models

In a Bayesian analysis of finite mixture models, parameter estimation and clustering are sometimes less straightforward than might be expected. In particular, the common practice of estimating parameters by their posterior mean, and summarizing joint posterior distributions by marginal distributions, often leads to nonsensical answers. This is due to the so-called 'label switching' problem, which is caused by symmetry in the likelihood of the model parameters. A frequent response to this problem is to remove the symmetry by using artificial identifiability constraints. We demonstrate that this fails in general to solve the problem, and we describe an alternative class of approaches, relabelling algorithms , which arise from attempting to minimize the posterior expected loss under a class of loss functions. We describe in detail one particularly simple and general relabelling algorithm and illustrate its success in dealing with the label switching problem on two examples.     

Point and Interval Estimations of Marginal Risk Difference by Logistic Model

We use logistic model to get point and interval estimates of the marginal risk difference in observational studies and randomized trials with dichotomous outcome. We prove that the maximum likelihood estimate of the marginal risk difference is unbiased for finite sample and highly robust to the effects of dispersing covariates. We use approximate normal distribution of the maximum likelihood estimates of the logistic model parameters to get approximate distribution of the maximum likelihood estimate of the marginal risk difference and then the interval estimate of the marginal risk difference. We illustrate application of the method by a real medical example.