Prediction in linear mixed models

Following estimation of effects from a linear mixed model, it is often useful to form predicted values for certain factor/variate combinations. The process has been well defined for linear models, but the introduction of random effects into the model means that a decision has to be made about the inclusion or exclusion of random model terms from the predictions. This paper discusses the interpretation of predictions formed including or excluding random terms. Four datasets are used to illustrate circumstances where different prediction strategies may be appropriate: in an orthogonal design, an unbalanced nested structure, a model with cubic smoothing spline terms and for kriging after spatial analysis. The examples also show the need for different weighting schemes that recognize nesting and aliasing during prediction, and the necessity of being able to detect inestimable predictions.     

Improvement of mixed predictors in linear mixed models

In this paper, we introduce stochastic-restricted Liu predictors which will be defined by combining in a special way the two approaches followed in obtaining the mixed predictors and the Liu predictors in the linear mixed models. Superiorities of the linear combination of the new predictor to the Liu and mixed predictors are done in the sense of mean square error matrix criterion. Finally, numerical examples and a simulation study are done to illustrate the findings. In numerical examples, we took some arbitrary observations from the data as the prior information since we did not have historical data or additional information about the data sets. The results show that this case does the new estimator gain efficiency over the constituent estimators and provide accurate estimation and prediction of the data.     

Reversible jump methods for generalised linear models and generalised linear mixed models

A reversible jump algorithm for Bayesian model determination among generalised linear models, under relatively diffuse prior distributions for the model parameters, is proposed. Orthogonal projections of the current linear predictor are used so that knowledge from the current model parameters is used to make effective proposals. This idea is generalised to moves of a reversible jump algorithm for model determination among generalised linear mixed models. Therefore, this algorithm exploits the full flexibility available in the reversible jump method. The algorithm is demonstrated via two examples and compared to existing methods.     

Leverage analysis for linear mixed models

We consider a generalized leverage matrix useful for the identification of influential units and observations in linear mixed models and show how a decomposition of this matrix may be employed to identify high leverage points for both the marginal fitted values and the random effect component of the conditional fitted values. We illustrate the different uses of the two components of the decomposition with a simulated example as well as with a real data set.     

Generalized linear mixed models: a review and some extensions

Breslow and Clayton (J Am Stat Assoc 88:9–25,1993) was, and still is, a highly influential paper mobilizing the use of generalized linear mixed models in epidemiology and a wide variety of fields. An important aspect is the feasibility in implementation through the ready availability of related software in SAS (SAS Institute, PROC GLIMMIX, SAS Institute Inc., URL , 2007), S-plus (Insightful Corporation, S-PLUS 8, Insightful Corporation, Seattle, WA, URL , 2007), and R (R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, URL , 2006) for example, facilitating its broad usage. This paper reviews background to generalized linear mixed models and the inferential techniques which have been developed for them. To provide the reader with a flavor of the utility and wide applicability of this fundamental methodology we consider a few extensions including additive models, models for zero-heavy data, and models accommodating latent clusters.     

Adaptive fitting of linear mixed-effects models with correlated random effects

Linear mixed-effects model has been widely used in longitudinal data analyses. In practice, the fitting algorithm can fail to converge due to boundary issues of the estimated random-effects covariance matrix G, that is, being near-singular, non-positive definite, or both. Current available algorithms are not computationally optimal because the condition number of matrix G is unnecessarily increased when the random-effects correlation estimate is not zero. We propose an adaptive fitting (AF) algorithm using an optimal linear transformation of the random-effects design matrix. It is a data-driven adaptive procedure, aiming at reducing subsequent random-effects correlation estimates down to zero in the optimal transformed estimation space. Simulations show that AF significantly improves the convergent properties, especially under small sample size, relative large noise and high correlation settings. One real data for insulin-like growth factor protein is used to illustrate the application of this algorithm implemented with software package R (nlme).     

Robust designs for linear mixed effects models

Summary.  In health sciences, medicine and social sciences linear mixed effects models are often used to analyse time-structured data. The search for optimal designs for these models is often hampered by two problems. The first problem is that these designs are only locally optimal. The second problem is that an optimal design for one model may not be optimal for other models. In this paper the maximin principle is adopted to handle both problems, simultaneously. The maximin criterion is formulated by means of a relative efficiency measure, which gives an indication of how much efficiency is lost when the uncertainty about the models over a prior domain of parameters is taken into account. The procedure is illustrated by means of three growth studies. Results are presented for a vocabulary growth study from education, a bone gain study from medical research and an epidemiological decline in height study. It is shown that, for the mixed effects polynomial models that are applied to these studies, the maximin designs remain highly efficient for different sets of models and combinations of parameter values.     

Local influence for generalized linear mixed models

The authors describe a method for assessing model inadequacy in maximum likelihood estimation of a generalized linear mixed model. They treat the latent random effects in the model as missing data and develop the influence analysis on the basis of a Q‐function which is associated with the conditional expectation of the complete‐data log‐likelihood function in the EM algorithm. They propose a procedure to detect influential observations in six model perturbation schemes. They also illustrate their methodology in a hypothetical situation and in two real cases.     

Robust estimation in generalized linear mixed models

Generalized linear mixed models (GLMMs) are widely used to analyse non-normal response data with extra-variation, but non-robust estimators are still routinely used. We propose robust methods for maximum quasi-likelihood and residual maximum quasi-likelihood estimation to limit the influence of outlying observations in GLMMs. The estimation procedure parallels the development of robust estimation methods in linear mixed models, but with adjustments in the dependent variable and the variance component. The methods proposed are applied to three data sets and a comparison is made with the nonparametric maximum likelihood approach. When applied to a set of epileptic seizure data, the methods proposed have the desired effect of limiting the influence of outlying observations on the parameter estimates. Simulation shows that one of the residual maximum quasi-likelihood proposals has a smaller bias than those of the other estimation methods. We further discuss the equivalence of two GLMM formulations when the response variable follows an exponential family. Their extensions to robust GLMMs and their comparative advantages in modelling are described. Some possible modifications of the robust GLMM estimation methods are given to provide further flexibility for applying the method.     

Efficient estimation of general linear mixed effects models

An extension of the linear growth curve model (Biometrics 38 (1982) 963) was proposed by Stukel and Demidenko (Biometrics 53 (1997) 720) to study the effects of population covariates on one or more characteristics of the curve, when the characteristics are expressed as linear combinations of the growth curve parameters. In the present paper, this general growth curve model receives a comprehensive theoretical treatment. A two-stage estimator, consisting of a generalized least squares estimator under constraints for the population parameters and a moment estimator for the variance parameters, is developed for application in the non-Gaussian error situation. Two likelihood based estimators, global maximum likelihood and second-stage maximum likelihood, are also developed. It is shown that all three estimators are consistent, asymptotically normally distributed, and efficient, and are equivalent when the number of individuals tends to infinity. An expression for the bias in the estimator of the population parameters is derived under second stage model misspecification. We show that if parameters that are not of primary interest are incorrectly specified, bias may occur in parameters that are of interest using the standard growth curve model. The general growth curve model does not require specification of such nuisance parameters and is robust in terms of bias. The general linear growth curve model is used to study the effects of host sex on pancreatic tumor growth in rats.     

A cell means analysis of mixed linear models

The purpose of this paper is to describe a simple procedure for the estima-tion of parameters in the unbalanced mixed linear model. There are implications for hypothesis testing and interval estimation, A feature of these estimators is that they are expressed in terms of simple formulas. This has obvious advantages for computations and small sample analysis. In addition, the formulas suggest useful diagnostic procedures for assessing the quality of the data as well as possible defects in the model assumptions. The concepts are illustrated with several examples. Evidence is presented to indicate that, in cases of modest imbalance, these estimators are highly efficient and dominate AOV estimates over most of the parameter space. In cases of more extreme imbalance, the results are qualitatively the same but the estimators are less efficient than the AOV estimators for small values of the parameters. The extension of this method to factorial models with missing cells is not complete.     

A comparison of mixed and minimax estimators of linear models

In this paper the stochastic properties of two estimators of linear models, mixed and minimax, based on different types of prior information, are compared using quadratic risk as the criterion for superiority. A necessary and sufficient condition for the minimax estimator to be superior to the comparable mixed estimator is derived as well as a simpler necessary but not sufficient condition.     

Restricted likelihood inference for generalized linear mixed models

We aim to promote the use of the modified profile likelihood function for estimating the variance parameters of a GLMM in analogy to the REML criterion for linear mixed models. Our approach is based on both quasi-Monte Carlo integration and numerical quadrature, obtaining in either case simulation-free inferential results. We will illustrate our idea by applying it to regression models with binary responses or count data and independent clusters, covering also the case of two-part models. Two real data examples and three simulation studies support the use of the proposed solution as a natural extension of REML for GLMMs. An R package implementing the methodology is available online.     

Robust linear mixed models for Small Area Estimation

Hierarchical models are popular in many applied statistics fields including Small Area Estimation. One well known model employed in this particular field is the Fay–Herriot model, in which unobservable parameters are assumed to be Gaussian. In Hierarchical models assumptions about unobservable quantities are difficult to check. For a special case of the Fay–Herriot model, Sinharay and Stern [2003. Posterior predictive model checking in Hierarchical models. J. Statist. Plann. Inference 111, 209–221] showed that violations of the assumptions about the random effects are difficult to detect using posterior predictive checks. In this present paper we consider two extensions of the Fay–Herriot model in which the random effects are assumed to be distributed according to either an exponential power (EP) distribution or a skewed EP distribution. We aim to explore the robustness of the Fay–Herriot model for the estimation of individual area means as well as the empirical distribution function of their ‘ensemble’. Our findings, which are based on a simulation experiment, are largely consistent with those of Sinharay and Stern as far as the efficient estimation of individual small area parameters is concerned. However, when estimating the empirical distribution function of the ‘ensemble’ of small area parameters, results are more sensitive to the failure of distributional assumptions.     

Sequential D-optimal designs for generalized linear mixed models

We discuss the construction of D-optimal sequential designs for the analysis of longitudinal data or repeated measurements using generalized linear mixed models (GLMMs). We investigate the performance of the design through a simulation study, which indicates that the proposed design can be very successful in improving the efficiency of the ML estimators in GLMMs relative to some common competitors. Our simulations also suggest that the usual normal-theory inference procedures remain valid under the sequential sampling schemes. We also present an example using real data obtained from a clinical study.     

Bayesian quantile regression for skew-normal linear mixed models

Linear mixed models have been widely used to analyze repeated measures data which arise in many studies. In most applications, it is assumed that both the random effects and the within-subjects errors are normally distributed. This can be extremely restrictive, obscuring important features of within-and among-subject variations. Here, quantile regression in the Bayesian framework for the linear mixed models is described to carry out the robust inferences. We also relax the normality assumption for the random effects by using a multivariate skew-normal distribution, which includes the normal ones as a special case and provides robust estimation in the linear mixed models. For posterior inference, we propose a Gibbs sampling algorithm based on a mixture representation of the asymmetric Laplace distribution and multivariate skew-normal distribution. The procedures are demonstrated by both simulated and real data examples.     

Estimation of group means in generalized linear mixed models

In this study, we investigate the concept of the mean response for a treatment group mean as well as its estimation and prediction for generalized linear models with a subject‐wise random effect. Generalized linear models are commonly used to analyze categorical data. The model‐based mean for a treatment group usually estimates the response at the mean covariate. However, the mean response for the treatment group for studied population is at least equally important in the context of clinical trials. New methods were proposed to estimate such a mean response in generalized linear models; however, this has only been done when there are no random effects in the model. We suggest that, in a generalized linear mixed model (GLMM), there are at least two possible definitions of a treatment group mean response that can serve as estimation/prediction targets. The estimation of these treatment group means is important for healthcare professionals to be able to understand the absolute benefit vs risk. For both of these treatment group means, we propose a new set of methods that suggests how to estimate/predict both of them in a GLMMs with a univariate subject‐wise random effect. Our methods also suggest an easy way of constructing corresponding confidence and prediction intervals for both possible treatment group means. Simulations show that proposed confidence and prediction intervals provide correct empirical coverage probability under most circumstances. Proposed methods have also been applied to analyze hypoglycemia data from diabetes clinical trials.     

Mixture of linear mixed models using multivariate t distribution

Linear mixed models are widely used when multiple correlated measurements are made on each unit of interest. In many applications, the units may form several distinct clusters, and such heterogeneity can be more appropriately modelled by a finite mixture linear mixed model. The classical estimation approach, in which both the random effects and the error parts are assumed to follow normal distribution, is sensitive to outliers, and failure to accommodate outliers may greatly jeopardize the model estimation and inference. We propose a new mixture linear mixed model using multivariate t distribution. For each mixture component, we assume the response and the random effects jointly follow a multivariate t distribution, to conveniently robustify the estimation procedure. An efficient expectation conditional maximization algorithm is developed for conducting maximum likelihood estimation. The degrees of freedom parameters of the t distributions are chosen data adaptively, for achieving flexible trade-off between estimation robustness and efficiency. Simulation studies and an application on analysing lung growth longitudinal data showcase the efficacy of the proposed approach.