Robust estimation in generalized linear mixed models

Generalized linear mixed models (GLMMs) are widely used to analyse non-normal response data with extra-variation, but non-robust estimators are still routinely used. We propose robust methods for maximum quasi-likelihood and residual maximum quasi-likelihood estimation to limit the influence of outlying observations in GLMMs. The estimation procedure parallels the development of robust estimation methods in linear mixed models, but with adjustments in the dependent variable and the variance component. The methods proposed are applied to three data sets and a comparison is made with the nonparametric maximum likelihood approach. When applied to a set of epileptic seizure data, the methods proposed have the desired effect of limiting the influence of outlying observations on the parameter estimates. Simulation shows that one of the residual maximum quasi-likelihood proposals has a smaller bias than those of the other estimation methods. We further discuss the equivalence of two GLMM formulations when the response variable follows an exponential family. Their extensions to robust GLMMs and their comparative advantages in modelling are described. Some possible modifications of the robust GLMM estimation methods are given to provide further flexibility for applying the method.     

QUALITY IMPROVEMENT FROM DISASSEMBLY-REASSEMBLY EXPERIMENTS

The goal of achieving high quality products has led to an emphasis on reducing variation in performance characteristics. It may often happen that one of the product's components is responsible for much of the observed variation. This research is stimulated by the problem of detecting a component that impairs quality by systematically inflating the variance in a product that is assembled from “interchangeable components.” We consider the class of “disassembly-reassembly” experiments, in which components are swapped among assemblies. The specific units used in the experiment are sampled from a large population of units, so it is natural to measure the influence of each factor by its variance component. We present the model for these experiments as a special case of the mixed linear model, compare several estimators for the variance components and consider the problem of testing hypotheses to identify troublesome components.     

Random regression models for human immunodeficiency virus ribonucleic acid data subject to left censoring and informative drop-outs

Objectives in many longitudinal studies of individuals infected with the human immunodeficiency virus (HIV) include the estimation of population average trajectories of HIV ribonucleic acid (RNA) over time and tests for differences in trajectory across subgroups. Special features that are often inherent in the underlying data include a tendency for some HIV RNA levels to be below an assay detection limit, and for individuals with high initial levels or high ranges of change to drop out of the study early because of illness or death. We develop a likelihood for the observed data that incorporates both of these features. Informative drop-outs are handled by means of an approach previously published by Schluchter. Using data from the HIV Epidemiology Research Study, we implement a maximum likelihood procedure to estimate initial HIV RNA levels and slopes within a population, compare these parameters across subgroups of HIV-infected women and illustrate the importance of appropriate treatment of left censoring and informative drop-outs. We also assess model assumptions and consider the prediction of random intercepts and slopes in this setting. The results suggest that marked bias in estimates of fixed effects, variance components and standard errors in the analysis of HIV RNA data might be avoided by the use of methods like those illustrated.     

Avoiding 'data snooping' in multilevel and mixed effects models

Summary.  Multilevel or mixed effects models are commonly applied to hierarchical data. The level 2 residuals, which are otherwise known as random effects, are often of both substantive and diagnostic interest. Substantively, they are frequently used for institutional comparisons or rankings. Diagnostically, they are used to assess the model assumptions at the group level. Inference on the level 2 residuals, however, typically does not account for 'data snooping', i.e. for the harmful effects of carrying out a multitude of hypothesis tests at the same time. We provide a very general framework that encompasses both of the following inference problems: inference on the 'absolute' level 2 residuals to determine which are significantly different from 0, and inference on any prespecified number of pairwise comparisons. Thus, the user has the choice of testing the comparisons of interest. As our methods are flexible with respect to the estimation method that is invoked, the user may choose the desired estimation method accordingly. We demonstrate the methods with the London education authority data, the wafer data and the National Educational Longitudinal Study data.     

Harmonic mean approach to unbalanced mixed effects models under heteroscedastic variances

In this paper we consider unbalanced mixed models (Scheffe's model) under heteroscedastic variances. By using the harmonic mean approach, It is shown that the problems appear to be anologous to those problems from balanced mixed models under homoscedastic variance. Thus, by using harmonic mean approach, statistical inferences about fixed effects and variance components are derived by using those from balanced models under homoscedastic variance. Laguerre polynomial expansion is used Lo approximate sampling distributions of relevant statistics.     

Checking identifiability of covariance parameters in linear mixed effects models

To build a linear mixed effects model, one needs to specify the random effects and often the associated parametrized covariance matrix structure. Inappropriate specification of the structures can result in the covariance parameters of the model not identifiable. Non-identifiability can result in extraordinary wide confidence intervals, and unreliable parameter inference. Sometimes software produces implication of model non-identifiability, but not always. In the simulation of fitting non-identifiable models we tried, about half of the times the software output did not look abnormal. We derive necessary and sufficient conditions of covariance parameters identifiability which does not require any prior model fitting. The results are easy to implement and are applicable to commonly used covariance matrix structures.     

Estimation in mixed models through three step minimization

The aim of this article is to present an estimation procedure for both fixed effects and variance components in linear mixed models. This procedure consists of a maximum-likelihood method which we call Three Step Minimization (TSM). The major contribution of this method is that when variances tend to be null standard algorithms behave badly, unlike the TSM method, which uses a grid search algorithm in a compact set. A numerical application with real and simulated data is provided.     

Generalized mixed estimator for nonlinear models: a maximum likelihood approach

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

Estimating multiple-membership logit models with mixed effects: indirect inference versus data cloning

Multiple-membership logit models with random effects are models for clustered binary data, where each statistical unit can belong to more than one group. The likelihood function of these models is analytically intractable. We propose two different approaches for parameter estimation: indirect inference and data cloning (DC). The former is a non-likelihood-based method which uses an auxiliary model to select reasonable estimates. We propose an auxiliary model with the same dimension of parameter space as the target model, which is particularly convenient to reach good estimates very fast. The latter method computes maximum likelihood estimates through the posterior distribution of an adequate Bayesian model, fitted to cloned data. We implement a DC algorithm specifically for multiple-membership models. A Monte Carlo experiment compares the two methods on simulated data. For further comparison, we also report Bayesian posterior mean and Integrated Nested Laplace Approximation hybrid DC estimates. Simulations show a negligible loss of efficiency for the indirect inference estimator, compensated by a relevant computational gain. The approaches are then illustrated with two real examples on matched paired data.     

Inverse prediction for heteroscedastic response using mixed models software

Distributions of a response y (height, for example) differ with values of a factor t (such as age). Given a response y_* for a subject of unknown t_*, the objective of inverse prediction is to infer the value of t_* and to provide a defensible confidence set for it. Training data provide values of y observed on subjects at known values of t. Models relating the mean and variance of y to t can be formulated as mixed (fixed and random) models in terms of sets of functions of t, such as polynomial spline functions. A confidence set on t_* can then be had as those hypothetical values of t for which y_* is not detected as an outlier when compared to the model fit to the training data. With nonconstant variance, the p-values for these tests are approximate. This article describes how versatile models for this problem can be formulated in such a way that the computations can be accomplished with widely available software for mixed models, such as SAS PROC MIXED. Coverage probabilities of confidence sets on t_* are illustrated in an example.     

Restricted likelihood ratio lack-of-fit tests using mixed spline models

Summary.  Penalized regression spline models afford a simple mixed model representation in which variance components control the degree of non-linearity in the smooth function estimates. This motivates the study of lack-of-fit tests based on the restricted maximum likelihood ratio statistic which tests whether variance components are 0 against the alternative of taking on positive values. For this one-sided testing problem a further complication is that the variance component belongs to the boundary of the parameter space under the null hypothesis. Conditions are obtained on the design of the regression spline models under which asymptotic distribution theory applies, and finite sample approximations to the asymptotic distribution are provided. Test statistics are studied for simple as well as multiple-regression models.     

Kernel-based global MLE of partial linear random effects models for longitudinal data

Random effects models have been playing a critical role for modelling longitudinal data. However, there are little studies on the kernel-based maximum likelihood method for semiparametric random effects models. In this paper, based on kernel and likelihood methods, we propose a pooled global maximum likelihood method for the partial linear random effects models. The pooled global maximum likelihood method employs the local approximations of the nonparametric function at a group of grid points simultaneously, instead of one point. Gaussian quadrature is used to approximate the integration of likelihood with respect to random effects. The asymptotic properties of the proposed estimators are rigorously studied. Simulation studies are conducted to demonstrate the performance of the proposed approach. We also apply the proposed method to analyse correlated medical costs in the Medical Expenditure Panel Survey data set.     

A Bayesian hierarchical model for categorical longitudinal data from a social survey of immigrants

Summary.  The paper investigates a Bayesian hierarchical model for the analysis of categorical longitudinal data from a large social survey of immigrants to Australia. Data for each subject are observed on three separate occasions, or waves, of the survey. One of the features of the data set is that observations for some variables are missing for at least one wave. A model for the employment status of immigrants is developed by introducing, at the first stage of a hierarchical model, a multinomial model for the response and then subsequent terms are introduced to explain wave and subject effects. To estimate the model, we use the Gibbs sampler, which allows missing data for both the response and the explanatory variables to be imputed at each iteration of the algorithm, given some appropriate prior distributions. After accounting for significant covariate effects in the model, results show that the relative probability of remaining unemployed diminished with time following arrival in Australia.     

Robust designs for linear mixed effects models

Summary.  In health sciences, medicine and social sciences linear mixed effects models are often used to analyse time-structured data. The search for optimal designs for these models is often hampered by two problems. The first problem is that these designs are only locally optimal. The second problem is that an optimal design for one model may not be optimal for other models. In this paper the maximin principle is adopted to handle both problems, simultaneously. The maximin criterion is formulated by means of a relative efficiency measure, which gives an indication of how much efficiency is lost when the uncertainty about the models over a prior domain of parameters is taken into account. The procedure is illustrated by means of three growth studies. Results are presented for a vocabulary growth study from education, a bone gain study from medical research and an epidemiological decline in height study. It is shown that, for the mixed effects polynomial models that are applied to these studies, the maximin designs remain highly efficient for different sets of models and combinations of parameter values.     

Geoadditive models

Summary. A study into geographical variability of reproductive health outcomes (e.g. birth weight) in Upper Cape Cod, Massachusetts, USA, benefits from geostatistical mapping or kriging . However, also observed are some continuous covariates (e.g. maternal age) that exhibit pronounced non-linear relationships with the response variable. To account for such effects properly we merge kriging with additive models to obtain what we call geoadditive models . The merging becomes effortless by expressing both as linear mixed models. The resulting mixed model representation for the geoadditive model allows for fitting and diagnosis using standard methodology and software.     

Estimation of Variance Components for a Linear Toeplitz Model

The linear Toeplitz covariance structure model of order one is considered. We give some elegant explicit expressions of the Locally Minimum Variance Quadratic Unbiased Estimators of its covariance parameters. We deduce from a Monte Carlo method some properties of their Gaussian maximum likelihood estimators. Finally, for small sample sizes, these two types of estimators are compared with the intuitive empirical estimators and it is shown that the empirical biased estimators should be used.     

The EM Algorithm—an Old Folk-song Sung to a Fast New Tune

Celebrating the 20th anniversary of the presentation of the paper by Dempster, Laird and Rubin which popularized the EM algorithm, we investigate, after a brief historical account, strategies that aim to make the EM algorithm converge faster while maintaining its simplicity and stability (e.g. automatic monotone convergence in likelihood). First we introduce the idea of a 'working parameter' to facilitate the search for efficient data augmentation schemes and thus fast EM implementations. Second, summarizing various recent extensions of the EM algorithm, we formulate a general alternating expectation–conditional maximization algorithm AECM that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations. We illustrate these methods using multivariate t -models with known or unknown degrees of freedom and Poisson models for image reconstruction. We show, through both empirical and theoretical evidence, the potential for a dramatic reduction in computational time with little increase in human effort. We also discuss the intrinsic connection between EM-type algorithms and the Gibbs sampler, and the possibility of using the techniques presented here to speed up the latter. The main conclusion of the paper is that, with the help of statistical considerations, it is possible to construct algorithms that are simple, stable and fast.     

Estimating common parameters in heterogeneous random effects models

A question of fundamental importance for meta-analysis of heterogeneous multidimensional data studies is how to form a best consensus estimator of common parameters, and what uncertainty to attach to the estimate. This issue is addressed for a class of unbalanced linear designs which include classical growth curve models. The solution obtained is similar to the popular DerSimonian and Laird (1986) method for a simple meta-analysis model. By using almost unbiased variance estimators, an estimator of the covariance matrix of this procedure is derived. Combination of these methods is illustrated by two examples and are compared via simulation.     

Testing random effects in linear mixed models: another look at the F‐test (with discussion)

This article re‐examines the F‐test based on linear combinations of the responses, or FLC test, for testing random effects in linear mixed models. In current statistical practice, the FLC test is underused and we argue that it should be reconsidered as a valuable method for use with linear mixed models. We present a new, more general derivation of the FLC test which applies to a broad class of linear mixed models where the random effects can be correlated. We highlight three advantages of the FLC test that are often overlooked in modern applications of linear mixed models, namely its computation speed, its generality, and its exactness as a test. Empirical studies provide new insight into the finite sample performance of the FLC test, identifying cases where it is competitive or even outperforms modern methods in terms of power, as well as settings in which it performs worse than simulation‐based methods for testing random effects. In all circumstances, the FLC test is faster to compute.