Robust estimation of variance components

New robust estimates for variance components are introduced. Two simple models are considered: the balanced one-way classification model with a random factor and the balanced mixed model with one random factor and one fixed factor. However, the method of estimation proposed can be extended to more complex models. The new method of estimation we propose is based on the relationship between the variance components and the coefficients of the least-mean-squared-error predictor between two observations of the same group. This relationship enables us to transform the problem of estimating the variance components into the problem of estimating the coefficients of a simple linear regression model. The variance-component estimators derived from the least-squares regression estimates are shown to coincide with the maximum-likelihood estimates. Robust estimates of the variance components can be obtained by replacing the least-squares estimates by robust regression estimates. In particular, a Monte Carlo study shows that for outlier-contaminated normal samples, the estimates of variance components derived from GM regression estimates and the derived test outperform other robust procedures.     

Efficient estimation of variance components in nonparametric mixed-effects models with large samples

Linear mixed-effects (LME) regression models are a popular approach for analyzing correlated data. Nonparametric extensions of the LME regression model have been proposed, but the heavy computational cost makes these extensions impractical for analyzing large samples. In particular, simultaneous estimation of the variance components and smoothing parameters poses a computational challenge when working with large samples. To overcome this computational burden, we propose a two-stage estimation procedure for fitting nonparametric mixed-effects regression models. Our results reveal that, compared to currently popular approaches, our two-stage approach produces more accurate estimates that can be computed in a fraction of the time.     

Minimum variance unbiased invariant estimation of variance components under normality

In a variance components model for normally distributed data, for a specified vector of linear combinations of the variance components, necessary and sufficient conditions are given under which the vector has a uniformly minimum variance unbiased translation-invariant estimator. The competing class of estimators is not restricted to those that are quadratic. For classification models, the conditions are translated into easy-to-check partial balance requirements on the incidence array.     

Mixed effects smoothing spline analysis of variance

We propose a general family of nonparametric mixed effects models. Smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function. The random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters. These parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method. We derive a connection between a nonparametric mixed effects model and a linear mixed effects model. This connection suggests a way of fitting a nonparametric mixed effects model by using existing programs. The classical two-way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis-of-variance decompositions to build nonparametric mixed effects models. Similarly to the classical analysis of variance, components of these nonparametric mixed effects models can be interpreted as main effects and interactions. The penalized likelihood estimates of the fixed effects in a two-way mixed model are extensions of James–Stein shrinkage estimates to correlated observations. In an example three nested nonparametric mixed effects models are fitted to a longitudinal data set.     

Estimation of variance components in staggered nested designs

SUMMARY Variance components are estimated by two different methods for a general p stage random-effects staggered nested design. In addition to estimation from an analysis of variance, a new approach is introduced. The main features of this new technique are its simplicity and its ability to yield non-negative estimates of the variance components. The performances of the two procedures are compared using simulation and the meansquared-error criterion.     

Generalized Estimation of the BLUP in Mixed-Effects Models: A Comparison with ML and REML

The Best Linear Unbiased Predictor (BLUP) in mixed models is a function of the variance components and they are estimated using maximum likelihood (ML) or restricted ML methods. Nonconvergence of BLUP would occur due to a drawback of the standard likelihood-based approaches. In such situations, ML and REML either do not provide any BLUPs or all become equal. To overcome this drawback, we provide a generalized estimate (GE) of BLUP that does not suffer from the problem of negative or zero variance components, and compare its performance against the ML and REML estimates of BLUP. Simulated and published data are used to compare BLUP.     

Likelihood ratio tests for variance components in linear mixed models

Although the asymptotic distributions of the likelihood ratio for testing hypotheses of null variance components in linear mixed models derived by Stram and Lee [1994. Variance components testing in longitudinal mixed effects model. Biometrics 50, 1171–1177] are valid, their proof is based on the work of Self and Liang [1987. Asymptotic properties of maximum likelihood estimators and likelihood tests under nonstandard conditions. J. Amer. Statist. Assoc. 82, 605–610] which requires identically distributed random variables, an assumption not always valid in longitudinal data problems. We use the less restrictive results of Vu and Zhou [1997. Generalization of likelihood ratio tests under nonstandard conditions. Ann. Statist. 25, 897–916] to prove that the proposed mixture of chi-squared distributions is the actual asymptotic distribution of such likelihood ratios used as test statistics for null variance components in models with one or two random effects. We also consider a limited simulation study to evaluate the appropriateness of the asymptotic distribution of such likelihood ratios in moderately sized samples.     

Adaptive likelihood estimator of conditional variance function

Modelling volatility in the form of conditional variance function has been a popular method mainly due to its application in financial risk management. Among others, we distinguish the parametric GARCH models and the nonparametric local polynomial approximation using weighted least squares or gaussian likelihood function. We introduce an alternative likelihood estimate of conditional variance and we show that substitution of the error density with its estimate yields similar asymptotic properties, that is, the proposed estimate is adaptive to the error distribution. Theoretical comparison with existing estimates reveals substantial gains in efficiency, especially if error distribution has fatter tails than Gaussian distribution. Simulated data confirm the theoretical findings while an empirical example demonstrates the gains of the proposed estimate.     

A new approach to variance component estimation with diagnostic implications

Closed form expressions are developed for the estimators of functions of the variance components in balanced, mixed, linear models. These estimators are averages of sample covariances (variances) which offer diagnostic information on the data and the model. The cause of negative estimates may be revealed. Examples illustrate the basic concepts.     

Bootstrap tests for variance components in generalized linear mixed models

In many applications of generalized linear mixed models to clustered correlated or longitudinal data, often we are interested in testing whether a random effects variance component is zero. The usual asymptotic mixture of chi‐square distributions of the score statistic for testing constrained variance components does not necessarily hold. In this article, the author proposes and explores a parametric bootstrap test that appears to be valid based on its estimated level of significance under the null hypothesis. Results from a simulation study indicate that the bootstrap test has a level much closer to the nominal one while the asymptotic test is conservative, and is more powerful than the usual asymptotic score test based on a mixture of chi‐squares. The proposed bootstrap test is illustrated using two sets of real‐life data obtained from clinical trials. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

Generalized confidence intervals for proportions of total variance in mixed linear models

Exact confidence intervals for a proportion of total variance, based on pivotal quantities, only exist for mixed linear models having two variance components. Generalized confidence intervals (GCIs) introduced by Weerahandi [1993. Generalized confidence intervals (Corr: 94V89 p726). J. Am. Statist. Assoc. 88, 899–905] are based on generalized pivotal quantities (GPQs) and can be constructed for a much wider range of models. In this paper, the author investigates the coverage probabilities, as well as the utility of GCIs, for a proportion of total variance in mixed linear models having more than two variance components. Particular attention is given to the formation of GPQs and GCIs in mixed linear models having three variance components in situations where the data exhibit complete balance, partial balance, and partial imbalance. The GCI procedure is quite general and provides a useful method to construct confidence intervals in a variety of applications.     

Diagnosing explainable heterogeneity of variance in random‐effects models

Data‐analytic tools for models other than the normal linear regression model are relatively rare. Here we develop plots and diagnostic statistics for nonconstant variance for the random‐effects model (REM). REMs for longitudinal data include both within‐ and between‐subject variances. A basic assumption is that the two variance terms are constant across subjects. However, we often find that these variances are functions of covariates, and the data set has what we call explainable heterogeneity, which needs to be allowed for in the model. We characterize several types of heterogeneity of variance in REMs and develop three diagnostic tests using the score statistic: one for each of the two variance terms, and the third for a form of multivariate nonconstant variance. For each test we present an adjusted residual plot which can identify cases that are unusually influential on the outcome of the test.     

Confidence intervals and tests on the variance ratio in random models with two variance components

The LM test is modified to test any value of the ratio of two variance components in a mixed effects linear model with two variance components. The test is exact, so it can be used to construct exact confidence intervals on this ratio.Exact Neyman-Pearson (NP) tests on the variance ratio are described.Their powers provide attainable upper bounds on powers of tests on the variance ratio.Efficiencies of LM tests, which include ANOVA tests, and NP tests are compared for unbalanced, random, one-way ANOVA models.Confidence intervals corresponding to LM tests and NP tests are described.     

A note on variance estimation in the Cox proportional hazards model

The Cox proportional hazards model is widely used in clinical trials with time-to-event outcomes to compare an experimental treatment with the standard of care. At the design stage of a trial the number of events required to achieve a desired power needs to be determined, which is frequently based on estimating the variance of the maximum partial likelihood estimate of the regression parameter with a function of the number of events. Underestimating the variance at the design stage will lead to insufficiently powered studies, and overestimating the variance will lead to unnecessarily large trials. A simple approach to estimating the variance is introduced, which is compared with two widely adopted approaches in practice. Simulation results show that the proposed approach outperforms the standard ones and gives nearly unbiased estimates of the variance.     

Efficient Estimation of an Additive Quantile Regression Model

Abstract. In this paper, two non‐parametric estimators are proposed for estimating the components of an additive quantile regression model. The first estimator is a computationally convenient approach which can be viewed as a more viable alternative to existing kernel‐based approaches. The second estimator involves sequential fitting by univariate local polynomial quantile regressions for each additive component with the other additive components replaced by the corresponding estimates from the first estimator. The purpose of the extra local averaging is to reduce the variance of the first estimator. We show that the second estimator achieves oracle efficiency in the sense that each estimated additive component has the same variance as in the case when all other additive components were known. Asymptotic properties are derived for both estimators under dependent processes that are strictly stationary and absolutely regular. We also provide a demonstrative empirical application of additive quantile models to ambulance travel times.     

Modelling conditional variance function in industrial data: A case study

We study the suitability of different modelling methods for joint prediction of mean and variance based on large data sets. We review the approaches to the modelling of conditional variance function that are capable of handling a problem where conditional variance depends on about 10 explanatory variables and training dataset consists of 100 000 observations. We present a promising approach for neural network modelling of mean and dispersion. We compare different approaches in predicting the mechanical properties of steel in two case data sets collected from the production line of a steel plate mill. As a conclusion we give some recommendations concerning the modelling of conditional variance in large datasets.     

Extending the Mann–Whitney–Wilcoxon rank sum test to longitudinal regression analysis

Outliers are commonly observed in psychosocial research, generally resulting in biased estimates when comparing group differences using popular mean-based models such as the analysis of variance model. Rank-based methods such as the popular Mann–Whitney–Wilcoxon (MWW) rank sum test are more effective to address such outliers. However, available methods for inference are limited to cross-sectional data and cannot be applied to longitudinal studies under missing data. In this paper, we propose a generalized MWW test for comparing multiple groups with covariates within a longitudinal data setting, by utilizing the functional response models. Inference is based on a class of U-statistics-based weighted generalized estimating equations, providing consistent and asymptotically normal estimates not only under complete but missing data as well. The proposed approach is illustrated with both real and simulated study data.     

Estimating variance components in hiearchical structures using minque and restricted maximum livelihood

This paper presents a compact and efficient method for computing MINQUE and restricted maximum likelihood estimates of variance components for hierarchical classification models. The method does not require storage of large matrices and consequently removes limitations on the size of the structure, Ibcperience indicates that iterative MINQUE computations converge very rapidly to the restricted maximum likelihood estimates.     

Imputation of missing variance data using non-linear mixed effects modelling to enable an inverse variance weighted meta-analysis of summary-level longitudinal data: a case study

Missing variances, on the basis of the summary-level data, can be a problem when an inverse variance weighted meta-analysis is undertaken. A wide range of approaches in dealing with this issue exist, such as excluding data without a variance measure, using a function of sample size as a weight and imputing the missing standard errors/deviations. A non-linear mixed effects modelling approach was taken to describe the time-course of standard deviations across 14 studies. The model was then used to make predictions of the missing standard deviations, thus, enabling a precision weighted model-based meta-analysis of a mean pain endpoint over time. Maximum likelihood and Bayesian approaches were implemented with example code to illustrate how this imputation can be carried out and to compare the output from each method. The resultant imputations were nearly identical for the two approaches. This modelling approach acknowledges the fact that standard deviations are not necessarily constant over time and can differ between treatments and across studies in a predictable way.