ASYMPTOTIC PROPERTIES OF RESTRICTED MAXIMUM LIKELIHOOD (REML) ESTIMATES FOR HIERARCHICAL MIXED LINEAR MODELS

This paper explores the asymptotic distribution of the restricted maximum likelihood estimator of the variance components in a general mixed model. Restricting attention to hierarchical models, central limit theorems are obtained using elementary arguments with only mild conditions on the covariates in the fixed part of the model and without having to assume that the data are either normally or spherically symmetrically distributed. Further, the REML and maximum likelihood estimators are shown to be asymptotically equivalent in this general framework, and the asymptotic distribution of the weighted least squares estimator (based on the REML estimator) of the fixed effect parameters is derived.     

Generalized Estimation of the BLUP in Mixed-Effects Models: A Comparison with ML and REML

The Best Linear Unbiased Predictor (BLUP) in mixed models is a function of the variance components and they are estimated using maximum likelihood (ML) or restricted ML methods. Nonconvergence of BLUP would occur due to a drawback of the standard likelihood-based approaches. In such situations, ML and REML either do not provide any BLUPs or all become equal. To overcome this drawback, we provide a generalized estimate (GE) of BLUP that does not suffer from the problem of negative or zero variance components, and compare its performance against the ML and REML estimates of BLUP. Simulated and published data are used to compare BLUP.     

A note on model selection using information criteria for general linear models estimated using REML

It is common practice to compare the fit of non‐nested models using the Akaike (AIC) or Bayesian (BIC) information criteria. The basis of these criteria is the log‐likelihood evaluated at the maximum likelihood estimates of the unknown parameters. For the general linear model (and the linear mixed model, which is a special case), estimation is usually carried out using residual or restricted maximum likelihood (REML). However, for models with different fixed effects, the residual likelihoods are not comparable and hence information criteria based on the residual likelihood cannot be used. For model selection, it is often suggested that the models are refitted using maximum likelihood to enable the criteria to be used. The first aim of this paper is to highlight that both the AIC and BIC can be used for the general linear model by using the full log‐likelihood evaluated at the REML estimates. The second aim is to provide a derivation of the criteria under REML estimation. This aim is achieved by noting that the full likelihood can be decomposed into a marginal (residual) and conditional likelihood and this decomposition then incorporates aspects of both the fixed effects and variance parameters. Using this decomposition, the appropriate information criteria for model selection of models which differ in their fixed effects specification can be derived. An example is presented to illustrate the results and code is available for analyses using the ASReml‐R package.     

Modifications of REML algorithm for HGLMs

Hierarchical generalized linear models (HGLMs) have become popular in data analysis. However, their maximum likelihood (ML) and restricted maximum likelihood (REML) estimators are often difficult to compute, especially when the random effects are correlated; this is because obtaining the likelihood function involves high-dimensional integration. Recently, an h-likelihood method that does not involve numerical integration has been proposed. In this study, we show how an h-likelihood method can be implemented by modifying the existing ML and REML procedures. A small simulation study is carried out to investigate the performances of the proposed methods for HGLMs with correlated random effects.     

A RESTRICTED MAXIMUM LIKELIHOOD PROCEDURE FOR ESTIMATING THE VARIANCE FUNCTION OF AN IMMUNOASSAY

Restricted maximum likelihood (REML) is a procedure for estimating a variance function in a heteroscedastic linear model. Although REML has been extended to non-linear models, the case in which the data are dominated by replicated observations with unknown values of the independent variable of interest, such as the concentration of a substance in a blood sample, has not been considered. We derive a REML procedure for an immunoassay and show that the resulting estimator is superior to those currently being used. Some interesting properties of the REML estimator are derived, and its relationship to other estimators is discussed.     

ASYMPTOTIC LIKELIHOOD APPROXIMATIONS USING A PARTIAL LAPLACE APPROXIMATION

Elimination of a nuisance variable is often non‐trivial and may involve the evaluation of an intractable integral. One approach to evaluate these integrals is to use the Laplace approximation. This paper concentrates on a new approximation, called the partial Laplace approximation, that is useful when the integrand can be partitioned into two multiplicative disjoint functions. The technique is applied to the linear mixed model and shows that the approximate likelihood obtained can be partitioned to provide a conditional likelihood for the location parameters and a marginal likelihood for the scale parameters equivalent to restricted maximum likelihood (REML). Similarly, the partial Laplace approximation is applied to the t‐distribution to obtain an approximate REML for the scale parameter. A simulation study reveals that, in comparison to maximum likelihood, the scale parameter estimates of the t‐distribution obtained from the approximate REML show reduced bias.     

A Monte Carlo study of REML and robust rank-based analyses for the random intercept mixed model

Restricted maximum likelihood (REML) methods are traditionally used for analyzing mixed models. Based on a multivariate normal likelihood, these analyses are sensitive to outliers. Recently developed robust rank-based procedures offer a complete analysis of mixed model: estimation of fixed effects, standard errors, and estimation of variance components. The results of a large Monte Carlo study are presented, comparing these two analyses for many situations over multivariate normal and contaminated normal distributions. The rank-based analyses are much more powerful and efficient than the REML analyses over all non-normal situations, while losing little power for normal errors.     

Likelihood inference for small variance components

The authors explore likelihood‐based methods for making inferences about the components of variance in a general normal mixed linear model. In particular, they use local asymptotic approximations to construct confidence intervals for the components of variance when the components are close to the boundary of the parameter space. In the process, they explore the question of how to profile the restricted likelihood (REML). Also, they show that general REML estimates are less likely to fall on the boundary of the parameter space than maximum‐likelihood estimates and that the likelihood‐ratio test based on the local asymptotic approximation has higher power than the likelihood‐ratio test based on the usual chi‐squared approximation. They examine the finite‐sample properties of the proposed intervals by means of a simulation study.     

Reml estimation for repeated measures analysis

Models for repeated measures or growth curves consist of a mean response plus error and the errors are usually correlated. Both maximum likelihood and residual maximum likelihood (REML) estimators of a regression model with dependent errors are derived for cases in which the variance matrix of the error model admits a convenient Cholesky factorisation. This factorisation may be linked to methods for producing recursive estimates of the regression parameters and recursive residuals to provide a convenient computational method. The method is used to develop a general approach to repeated measures analysis.     

Editorial collaborators

A simulation study of the binomial-logit model with correlated random effects is carried out based on the generalized linear mixed model (GLMM) methodology. Simulated data with various numbers of regression parameters and different values of the variance component are considered. The performance of approximate maximum likelihood (ML) and residual maximum likelihood (REML) estimators is evaluated. For a range of true parameter values, we report the average biases of estimators, the standard error of the average bias and the standard error of estimates over the simulations. In general, in terms of bias, the two methods do not show significant differences in estimating regression parameters. The REML estimation method is slightly better in reducing the bias of variance component estimates.     

Simplex Mixed‐Effects Models for Longitudinal Proportional Data

Abstract. Continuous proportional outcomes are collected from many practical studies, where responses are confined within the unit interval (0,1). Utilizing Barndorff‐Nielsen and Jørgensen's simplex distribution, we propose a new type of generalized linear mixed‐effects model for longitudinal proportional data, where the expected value of proportion is directly modelled through a logit function of fixed and random effects. We establish statistical inference along the lines of Breslow and Clayton's penalized quasi‐likelihood (PQL) and restricted maximum likelihood (REML) in the proposed model. We derive the PQL/REML using the high‐order multivariate Laplace approximation, which gives satisfactory estimation of the model parameters. The proposed model and inference are illustrated by simulation studies and a data example. The simulation studies conclude that the fourth order approximate PQL/REML performs satisfactorily. The data example shows that Aitchison's technique of the normal linear mixed model for logit‐transformed proportional outcomes is not robust against outliers.     

The three-fold nested random effects model

Estimation of the variance components and the mean of the balanced and unbalanced threefold nested design is considered. The relative merits of the following procedures are evaluated: Analysis of variance (ANOVA), maximum likelihood (ML), restricted maximum likelihood (REML), and minimum variance quadratic unbiased estimator (MIVQUE). A new procedure called the weighted analysis of means (WAM) estimator which utilizes prior information on the variance components is proposed. It is found to have optimum properties similar to the REML and MIVQUE, and it is also computationally simpler. For the mean, the overall sample average, grand mean, unweighted mean, and generalized least-squares (GLS) estimator with its weights obtained from the above estimators for the variance components are considered. Comparisons of the above procedures for the variance components and the mean are made from exact expressions for the biases and mean square errors (MSEs) of the estimators and from empirical investigations.     

A new REML (parameter expanded) EM algorithm for linear mixed models

Linear mixed models are regularly applied to animal and plant breeding data to evaluate genetic potential. Residual maximum likelihood (REML) is the preferred method for estimating variance parameters associated with this type of model. Typically an iterative algorithm is required for the estimation of variance parameters. Two algorithms which can be used for this purpose are the expectation‐maximisation (EM) algorithm and the parameter expanded EM (PX‐EM) algorithm. Both, particularly the EM algorithm, can be slow to converge when compared to a Newton‐Raphson type scheme such as the average information (AI) algorithm. The EM and PX‐EM algorithms require specification of the complete data, including the incomplete and missing data. We consider a new incomplete data specification based on a conditional derivation of REML. We illustrate the use of the resulting new algorithm through two examples: a sire model for lamb weight data and a balanced incomplete block soybean variety trial. In the cases where the AI algorithm failed, a REML PX‐EM based on the new incomplete data specification converged in 28% to 30% fewer iterations than the alternative REML PX‐EM specification. For the soybean example a REML EM algorithm using the new specification converged in fewer iterations than the current standard specification of a REML PX‐EM algorithm. The new specification integrates linear mixed models, Henderson's mixed model equations, REML and the REML EM algorithm into a cohesive framework.     

A cautionary note on the analysis of randomized block designs with a few missing values

The randomized block design is routinely employed in the social and biopharmaceutical sciences. With no missing values, analysis of variance (AOV) can be used to analyze such experiments. However, if some data are missing, the AOV formulae are no longer applicable, and iterative methods such as restricted maximum likelihood (REML) are recommended, assuming block effects are treated as random. Despite the well-known advantages of REML, methods like AOV based on complete cases (blocks) only (CC-AOV) continue to be used by researchers, particularly in situations where routinely only a few missing values are encountered. Reasons for this appear to include a natural proclivity for non-iterative, summary-statistic-based methods, and a presumption that CC-AOV is only trivially less efficient than REML with only a few missing values (say≤10%). The purpose of this note is two-fold. First, to caution that CC-AOV can be considerably less powerful than REML even with only a few missing values. Second, to offer a summary-statistic-based, pairwise-available-case-estimation (PACE) alternative to CC-AOV. PACE, which is identical to AOV (and REML) with no missing values, outperforms CC-AOV in terms of statistical power. However, it is recommended in lieu of REMLonly if software to implement the latter is unavailable, or the use of a “transparent” formula-based approach is deemed necessary. An example using real data is provided for illustration.     

Inference about regression parameters using highly stratified survey count data with over-dispersion and repeated measurements

We study methods to estimate regression and variance parameters for over-dispersed and correlated count data from highly stratified surveys. Our application involves counts of fish catches from stratified research surveys and we propose a novel model in fisheries science to address changes in survey protocols. A challenge with this model is the large number of nuisance parameters which leads to computational issues and biased statistical inferences. We use a computationally efficient profile generalized estimating equation method and compare it to marginal maximum likelihood (MLE) and restricted MLE (REML) methods. We use REML to address bias and inaccurate confidence intervals because of many nuisance parameters. The marginal MLE and REML approaches involve intractable integrals and we used a new R package that is designed for estimating complex nonlinear models that may include random effects. We conclude from simulation analyses that the REML method provides more reliable statistical inferences among the three methods we investigated.     

Restricted likelihood inference for generalized linear mixed models

We aim to promote the use of the modified profile likelihood function for estimating the variance parameters of a GLMM in analogy to the REML criterion for linear mixed models. Our approach is based on both quasi-Monte Carlo integration and numerical quadrature, obtaining in either case simulation-free inferential results. We will illustrate our idea by applying it to regression models with binary responses or count data and independent clusters, covering also the case of two-part models. Two real data examples and three simulation studies support the use of the proposed solution as a natural extension of REML for GLMMs. An R package implementing the methodology is available online.     

Two-Step Residual-Based Estimation of Error Variances for Generalized Least Squares in Split-Plot Experiments

In split-plot experiments, estimation of unknown parameters by generalized least squares (GLS), as opposed to ordinary least squares (OLS), is required, owing to the existence of whole- and subplot errors. However, estimating the error variances is often necessary for GLS. Restricted maximum likelihood (REML) is an established method for estimating the error variances, and its benefits have been highlighted in many previous studies. This article proposes a new two-step residual-based approach for estimating error variances. Results of numerical simulations indicate that the proposed method performs sufficiently well to be considered as a suitable alternative to REML.     

Random-effect models with singular precision

We show that smoothing spline, intrinsic autoregression (IAR) and state-space model can be formulated as partially specified random-effect model with singular precision (SP). Various fitting methods have been suggested for the aforementioned models and this paper investigates the relationships among them, once the models have been placed under a single framework. Some methods have been previously shown to give the best linear unbiased predictors (BLUPs) under some random-effect models and here we show that they are in fact uniformly BLUPs (UBLUPs) under a class of models that are generated by the SP of random effects. We offer some new interpretations of the UBLUPs under models of SP and define BLUE and BLUP in these partially specified models without having to specify the covariance. We also show how the full likelihood inferences for random-effect models can be made for these models, so that the maximum likelihood (ML) and restricted maximum likelihood (REML) estimators can be used for the smoothing parameters in splines, etc.     

Profile likelihood-based confidence interval for the dispersion parameter in count data

The importance of the dispersion parameter in counts occurring in toxicology, biology, clinical medicine, epidemiology, and other similar studies is well known. A couple of procedures for the construction of confidence intervals (CIs) of the dispersion parameter have been investigated, but little attention has been paid to the accuracy of its CIs. In this paper, we introduce the profile likelihood (PL) approach and the hybrid profile variance (HPV) approach for constructing the CIs of the dispersion parameter for counts based on the negative binomial model. The non-parametric bootstrap (NPB) approach based on the maximum likelihood (ML) estimates of the dispersion parameter is also considered. We then compare our proposed approaches with an asymptotic approach based on the ML and the restricted ML (REML) estimates of the dispersion parameter as well as the parametric bootstrap (PB) approach based on the ML estimates of the dispersion parameter. As assessed by Monte Carlo simulations, the PL approach has the best small-sample performance, followed by the REML, HPV, NPB, and PB approaches. Three examples to biological count data are presented.