Improved estimation procedures for multilevel models with binary response: a case-study

During recent years, analysts have been relying on approximate methods of inference to estimate multilevel models for binary or count data. In an earlier study of random-intercept models for binary outcomes we used simulated data to demonstrate that one such approximation, known as marginal quasi-likelihood, leads to a substantial attenuation bias in the estimates of both fixed and random effects whenever the random effects are non-trivial. In this paper, we fit three-level random-intercept models to actual data for two binary outcomes, to assess whether refined approximation procedures, namely penalized quasi-likelihood and second-order improvements to marginal and penalized quasi-likelihood, also underestimate the underlying parameters. The extent of the bias is assessed by two standards of comparison: exact maximum likelihood estimates, based on a Gauss–Hermite numerical quadrature procedure, and a set of Bayesian estimates, obtained from Gibbs sampling with diffuse priors. We also examine the effectiveness of a parametric bootstrap procedure for reducing the bias. The results indicate that second-order penalized quasi-likelihood estimates provide a considerable improvement over the other approximations, but all the methods of approximate inference result in a substantial underestimation of the fixed and random effects when the random effects are sizable. We also find that the parametric bootstrap method can eliminate the bias but is computationally very intensive.     

Approximate composite marginal likelihood inference in spatial generalized linear mixed models

Non-Gaussian spatial responses are usually modeled using spatial generalized linear mixed model with spatial random effects. The likelihood function of this model cannot usually be given in a closed form, thus the maximum likelihood approach is very challenging. There are numerical ways to maximize the likelihood function, such as Monte Carlo Expectation Maximization and Quadrature Pairwise Expectation Maximization algorithms. They can be applied but may in such cases be computationally very slow or even prohibitive. Gauss–Hermite quadrature approximation only suitable for low-dimensional latent variables and its accuracy depends on the number of quadrature points. Here, we propose a new approximate pairwise maximum likelihood method to the inference of the spatial generalized linear mixed model. This approximate method is fast and deterministic, using no sampling-based strategies. The performance of the proposed method is illustrated through two simulation examples and practical aspects are investigated through a case study on a rainfall data set.     

Kernel-based global MLE of partial linear random effects models for longitudinal data

Random effects models have been playing a critical role for modelling longitudinal data. However, there are little studies on the kernel-based maximum likelihood method for semiparametric random effects models. In this paper, based on kernel and likelihood methods, we propose a pooled global maximum likelihood method for the partial linear random effects models. The pooled global maximum likelihood method employs the local approximations of the nonparametric function at a group of grid points simultaneously, instead of one point. Gaussian quadrature is used to approximate the integration of likelihood with respect to random effects. The asymptotic properties of the proposed estimators are rigorously studied. Simulation studies are conducted to demonstrate the performance of the proposed approach. We also apply the proposed method to analyse correlated medical costs in the Medical Expenditure Panel Survey data set.     

Prediction in Multilevel Logistic Regression

The purpose of this article is to present a new method to predict the response variable of an observation in a new cluster for a multilevel logistic regression. The central idea is based on the empirical best estimator for the random effect. Two estimation methods for multilevel model are compared: penalized quasi-likelihood and Gauss–Hermite quadrature. The performance measures for the prediction of the probability for a new cluster observation of the multilevel logistic model in comparison with the usual logistic model are examined through simulations and an application.     

Analysis of censored longitudinal data with skewness and a terminal event

In HIV/AIDS study, the measurements viral load are often highly skewed and left-censored because of a lower detection limit. Furthermore, a terminal event (e.g., death) stops the follow-up process. The time to terminal event may be dependent on the viral load measurements. In this article, we present a joint analysis framework to model the censored longitudinal data with skewness and a terminal event process. The estimation is carried out by adaptive Gaussian quadrature techniques in SAS procedure NLMIXED. The proposed model is evaluated by a simulation study and is applied to the motivating Multicenter AIDS Cohort Study (MACS).     

Performance of likelihood-based estimation methods for multilevel binary regression models

By means of a fractional factorial simulation experiment, we compare the performance of penalised quasi-likelihood (PQL), non-adaptive Gaussian quadrature and adaptive Gaussian quadrature in estimating parameters for multilevel logistic regression models. The comparison is done in terms of bias, mean-squared error (MSE), numerical convergence and computational efficiency. It turns out that in terms of MSE, standard versions of the quadrature methods perform relatively poorly in comparison with PQL.     

Using the Fast Fourier Transform to Compute Multiple Comparisons With the Best and Subset Selection Critical Values

Multiple comparison methods are widely implemented in statistical packages and heavily used. To obtain the critical value of a multiple comparison method for a given confidence level, a double integral equation must be solved. Current computer implementations evaluate one double integral for each candidate critical value using Gaussian quadrature. Consequently, iterative refinement of the critical value can slow the response time enough to hamper interactive data analysis. However, for balanced designs, to obtain the critical value for multiple comparisons with the best, subset selection, and one-sided multiple comparison with a control, if one regards the inner integral as a function of the outer integration variable, then this function can be obtained by discrete convolution using the Fast Fourier Transform (FFT). Exploiting the fact that this function need not be re-evaluated during iterative refinement of the critical value, it is shown that the FFT method obtains critical values at least four times as accurate and two to five times as fast as the Gaussian quadrature method.     

An EM-type algorithm for multivariate mixture models

This paper introduces a new approach, based on dependent univariate GLMs, for fitting multivariate mixture models. This approach is a multivariate generalization of the method for univariate mixtures presented by Hinde (1982). Its accuracy and efficiency are compared with direct maximization of the log-likelihood. Using a simulation study, we also compare the efficiency of Monte Carlo and Gaussian quadrature methods for approximating the mixture distribution. The new approach with Gaussian quadrature outperforms the alternative methods considered. The work is motivated by the multivariate mixture models which have been proposed for modelling changes of employment states at an individual level. Similar formulations are of interest for modelling movement between other social and economic states and multivariate mixture models also occur in biostatistics and epidemiology.     

Bayesian estimation with integrated nested Laplace approximation for binary logit mixed models

In multilevel models for binary responses, estimation is computationally challenging due to the need to evaluate intractable integrals. In this paper, we investigate the performance of integrated nested Laplace approximation (INLA), a fast deterministic method for Bayesian inference. In particular, we conduct an extensive simulation study to compare the results obtained with INLA to the results obtained with a traditional stochastic method for Bayesian inference (MCMC Gibbs sampling), and with maximum likelihood through adaptive quadrature. Particular attention is devoted to the case of small number of clusters. The specification of the prior distribution for the cluster variance plays a crucial role and it turns out to be more relevant than the choice of the estimation method. The simulations show that INLA has an excellent performance as it achieves good accuracy (similar to MCMC) with reduced computational times (similar to adaptive quadrature).     

A general maximum likelihood analysis of overdispersion in generalized linear models

This paper presents an EM algorithm for maximum likelihood estimation in generalized linear models with overdispersion. The algorithm is initially derived as a form of Gaussian quadrature assuming a normal mixing distribution, but with only slight variation it can be used for a completely unknown mixing distribution, giving a straightforward method for the fully non-parametric ML estimation of this distribution. This is of value because the ML estimates of the GLM parameters may be sensitive to the specification of a parametric form for the mixing distribution. A listing of a GLIM4 algorithm for fitting the overdispersed binomial logit model is given in an appendix.A simple method is given for obtaining correct standard errors for parameter estimates when using the EM algorithm.Several examples are discussed.     

Efficient survey sampling of households via Gaussian quadrature

Summary.  The collection of data through surveys is a costly and time-consuming process, particularly when complex economic data are involved. The paper presents an efficient approach, based on Gaussian quadrature, to survey sampling when some information is available about the target population. Using household data from Mozambique, we demonstrate that Gaussian quadrature subsamples, based on relatively easy to observe household characteristics such as size and educational attainment of members, generate better estimates of the moments of household expenditure than random samples of equal size.     

Bayesian quadrature with non-normal approximating functions

We consider an efficient Bayesian approach to estimating integration-based posterior summaries from a separate Bayesian application. In Bayesian quadrature we model an intractable posterior density function f(·) as a Gaussian process, using an approximating function g(·), and find a posterior distribution for the integral of f(·), conditional on a few evaluations of f (·) at selected design points. Bayesian quadrature using normal g (·) is called Bayes-Hermite quadrature. We extend this theory by allowing g(·) to be chosen from two wider classes of functions. One is a family of skew densities and the other is the family of finite mixtures of normal densities. For the family of skew densities we describe an iterative updating procedure to select the most suitable approximation and apply the method to two simulated posterior density functions.     

An L1-type estimator of multivariate location and shape

The main goal of the paper is to specify a suitable multivariate multilevel model for polytomous responses with a non-ignorable missing data mechanism in order to determine the factors which influence the way of acquisition of the skills of the graduates and to evaluate the degree programmes on the basis of the adequacy of the skills they give to their graduates. The application is based on data gathered by a telephone survey conducted, about two years after the degree, on the graduates of year 2000 of the University of Florence. A multilevel multinomial logit model for the response of interest is fitted simultaneously with a multilevel logit model for the selection mechanism by means of maximum likelihood with adaptive Gaussian quadrature. In the application the multilevel structure has a crucial role, while selection bias results negligible. The analysis of the empirical Bayes residuals allows to detect some extreme degree programmes to be further inspected.     

Likelihood ratio test for univariate Gaussian mixture

We consider Gaussian mixtures and particularly the problem of testing homogeneity, that is testing no mixture, against a mixture with two components. Seven distinct cases are addressed, corresponding to the possible restrictions on the parameters. For each case, we give a statistic that we claim to be the likelihood ratio test statistic. The proof is given in a simple case. With the help of a bound for the maximum of a Gaussian process we calculate the percentile points. The results are illustrated by simulation.     

Gauss-laguerre dynamics:an efficient method of computing integrals in multiple comparisons

Computing with Gauss-Laguerre quadratures is a common practice when computing the distribution of the studentized version of many statistics used in normal theory multiple comparisons. We study the dynamics of the points associated with such quadrature and apply the results to the simple example of the studentized maximum modulus. Because most distributions used in multiple comparisons require relatively expensive numerical quadrature for evaluation of their non-studentized integrand, the efficiency of the suggested method is clear as it requires fewer integrand evaluations than the standard Gauss-Laguerre quadrature.     

Generalized Gauss–Hermite filtering

We consider a generalization of the Gauss–Hermite filter (GHF), where the filter density is represented by a Hermite expansion with leading Gaussian term (GGHF). Thus, the usual GHF is included as a special case. The moment equations for the time update are solved stepwise by Gauss–Hermite integration, and the measurement update is computed by the Bayes formula, again using numerical integration. The performance of the filter is compared numerically with the GHF, the UKF (unscented Kalman filter) and the EKF (extended Kalman filter) and leads to a lower mean squared filter error.     

Spherical radial approximation for nested mixed effects models

We consider a likelihood approximation in generalized linear mixed-effects models (GLMM) with multilevel nested random effects. Likelihood evaluation in such models is difficult, hindered by the need for high dimensional integration, where the dimension is proportional to the number of units per level and the number of random effects per unit. Various integration approaches have been proposed, including the penalized quasi-likelihood method, Laplace approximation, quadrature approximation, simulation, and MCMC algorithms. We propose a new quadrature approximation method, which is based on the spherical radial integration approach of Monahan and Genz (J Am Stat Assoc 92:664–674 1997), and at the same time takes advantage of the hierarchical structure of the integration. Our new hierarchical spherical radial method has a time complexity that is linear in the number of units per level and the number of random effects per unit, in contrast to the exponential complexity of the adaptive Gaussian quadrature method of Pinheiro and Chao (J Comput Graph Stat 15:58–81 2006) for the same problem. Using a spline approximation, the generalized additive mixed models (GAMM) are GLMMs with two levels of nested random effects. We apply our method to estimation of GAMMs. We compare it with competing methods through simulations and apply our method to analyze virologic and immunologic responses in an AIDS clinical trial. An R package is written and available at http://?users.?wpi.?edu/?~jgagnon/?software.?html.     

Tracking Edges, Corners and Vertices in an Image

Abstract.  In a range of imaging problems, particularly those where the images are of man-made objects, edges join at points which comprise three or more distinct boundaries between textures. In such cases the set of edges in the plane forms what a mathematician would call a planar graph. Smooth edges in the graph meet one another at junctions, called 'vertices', the 'degrees' of which denote the respective numbers of edges that join there. Conventional image reconstruction methods do not always draw clear distinctions among different degrees of junction, however. In such cases the algorithm is, in a sense, too locally adaptive; it inserts junctions without checking more globally to determine whether another configuration might be more suitable. In this paper we suggest an alternative approach to edge reconstruction, which combines a junction classification step with an edge-tracking routine. The algorithm still makes its decisions locally, so that the method retains an adaptive character. However, the fact that it focuses specifically on estimating the degree of a junction means that it is relatively unlikely to insert multiple low-degree junctions when evidence in the data supports the existence of a single high-degree junction. Numerical and theoretical properties of the method are explored, and theoretical optimality is discussed. The technique is based on local least-squares, or local likelihood in the case of Gaussian data. This feature, and the fact that the algorithm takes a tracking approach which does not require analysis of the full spatial data set, mean that it is relatively simple to implement.     

Testing Equality of Order Constrained Inverse Gaussian Means

The inverse Gaussian family of non negative, skewed random variables is analytically simple, and its inference theory is well known to be analogous to the normal theory in numerous ways. Hence, it is widely used for modeling non negative positively skewed data. In this note, we consider the problem of testing homogeneity of order restricted means of several inverse Gaussian populations with a common unknown scale parameter using an approach based on the classical methods, such as Fisher's, for combining independent tests. Unlike the likelihood approach which can only be readily applied to a limited number of restrictions and the settings of equal sample sizes, this approach is applicable to problems involving a broad variety of order restrictions and arbitrary sample size settings, and most importantly, no new null distributions are needed. An empirical power study shows that, in case of the simple order, the test based on Fisher's combination method compares reasonably with the corresponding likelihood ratio procedure.     

Confirmatory Factor Analysis of Ordinal Data Using Full‐Information Adaptive Quadrature

We conducted confirmatory factor analysis (CFA) of responses (N=803) to a self‐reported measure of optimism, using full‐information estimation via adaptive quadrature (AQ), an alternative estimation method for ordinal data. We evaluated AQ results in terms of the number of iterations required to achieve convergence, model fit, parameter estimates, standard errors (SE), and statistical significance, across four link‐functions (logit, probit, log‐log, complimentary log‐log) using 3–10 and 20 quadrature points. We compared AQ results with those obtained using maximum likelihood, robust maximum likelihood, and robust diagonally weighted least‐squares estimation. Compared to the other two link‐functions, logit and probit not only produced fit statistics, parameters estimates, SEs, and levels of significance that varied less across numbers of quadrature points, but also fitted the data better and provided larger completely standardised loadings than did maximum likelihood and diagonally weighted least‐squares. Our findings demonstrate the viability of using full‐information AQ to estimate CFA models with real‐world ordinal data.