Fast and robust estimators of variance components in the nested error model

Usual fitting methods for the nested error linear regression model are known to be very sensitive to the effect of even a single outlier. Robust approaches for the unbalanced nested error model with proved robustness and efficiency properties, such as M-estimators, are typically obtained through iterative algorithms. These algorithms are often computationally intensive and require robust estimates of the same parameters to start the algorithms, but so far no robust starting values have been proposed for this model. This paper proposes computationally fast robust estimators for the variance components under an unbalanced nested error model, based on a simple robustification of the fitting-of-constants method or Henderson method III. These estimators can be used as starting values for other iterative methods. Our simulations show that they are highly robust to various types of contamination of different magnitude.     

Maximum likelihood estimation of variance components of heteroscedastic random anova model

the estimation of variance components of heteroscedastic random model is discussed in this paper. Maximum Likelihood (ML) is described for one-way heteroscedastic random models. The proportionality condition that cell variance is proportional to the cell sample size, is used to eliminate the efffect of heteroscedasticity. The algebraic expressions of the estimators are obtained for the model. It is seen that the algebraic expressions of the estimators depend mainly on the inverse of the variance-covariance matrix of the observation vector. So, the variance-covariance matrix is obtained and the formulae for the inversions are given. A Monte Carlo study is conducted. Five different variance patterns with different numbers of cells are considered in this study. For each variance pattern, 1000 Monte Carlo samples are drawn. Then the Monte Carlo biases and Monte Carlo MSE’s of the estimators of variance components are calculated. In respect of both bias and MSE, the Maximum Likelihood (ML) estimators of variance components are found to be sufficiently good.     

Variance least squares estimators for multivariate linear mixed model

Non-iterative, distribution-free, and unbiased estimators of variance components by least squares method are derived for multivariate linear mixed model. A general inter-cluster variance matrix, a same-member only general inter-response variance matrix, and an uncorrelated intra-cluster error structure for each response are assumed. Projection method is suggested when unbiased estimators of variance components are not nonnegative definite matrices. A simulation study is conducted to investigate the properties of the proposed estimators in terms of bias and mean square error with comparison to the Gaussian (restricted) maximum likelihood estimators. The proposed estimators are illustrated by an application of gene expression familial study.     

A comment on the orthogonalization of B-spline basis functions and their derivatives

Through the use of a matrix representation for B-splines presented by Qin (Vis. Comput. 16:177–186, 2000) we are able to reexamine calculus operations on B-spline basis functions. In this matrix framework the problem associated with generating orthogonal splines is reexamined, and we show that this approach can simplify the operations involved to linear matrix operations. We apply these results to a recent paper (Zhou et al. in Biometrika 95:601–619, 2008) on hierarchical functional data analysis using a principal components approach, where a numerical integration scheme was used to orthogonalize a set of B-spline basis functions. These orthogonalized basis functions, along with their estimated derivatives, are then used to construct estimates of mean functions and functional principal components. By applying the methods presented here such algorithms can benefit from increased speed and precision. An R package is available to do the computations.     

HIGH DIMENSIONAL COVARIANCE MATRIX ESTIMATION IN APPROXIMATE FACTOR MODELS

The variance covariance matrix plays a central role in the inferential theories of high dimensional factor models in finance and economics. Popular regularization methods of directly exploiting sparsity are not directly applicable to many financial problems. Classical methods of estimating the covariance matrices are based on the strict factor models, assuming independent idiosyncratic components. This assumption, however, is restrictive in practical applications. By assuming sparse error covariance matrix, we allow the presence of the cross-sectional correlation even after taking out common factors, and it enables us to combine the merits of both methods. We estimate the sparse covariance using the adaptive thresholding technique as in Cai and Liu (2011), taking into account the fact that direct observations of the idiosyncratic components are unavailable. The impact of high dimensionality on the covariance matrix estimation based on the factor structure is then studied.     

A Comparison of Methods of Fitting Models to Twin Data

Data on twins are used to infer a genetic component of variance for various quantitative human characteristics. There are several statistical approaches available to analyze twin data. Here we compare three approaches for fitting variance components models to the relationship between height and bi-illiocristal diameter across ages in a sample of male and female Polish twins aged 8–17. Two of the approaches assume a multivariate normal model for the data, with one basing the likelihood on the raw data and the other using the distribution of the sample covariance matrix. The third approach uses a robust modification of the multivariate normal log-likelihood to downweight abnormal observations. The statistical theory underlying the methods is outlined, and the implementation of the methods is discussed.     

Graphical technique for comparing designs for random models

Methods for comparing designs for a random (or mixed) linear model have focused primarily on criteria based on single-valued functions. In general, these functions are difficult to use, because of their complex forms, in addition to their dependence on the model's unknown variance components. In this paper, a graphical approach is presented for comparing designs for random models. The one-way model is used for illustration. The proposed approach is based on using quantiles of an estimator of a function of the variance components. The dependence of these quantiles on the true values of the variance components is depicted by plotting the so-called quantile dispersion graphs (QDGs), which provide a comprehensive picture of the quality of estimation obtained with a given design. The QDGs can therefore be used to compare several candidate designs. Two methods of estimation of variance components are considered, namely analysis of variance and maximum-likelihood estimation.     

Characterization of an optimal matrix estimator under convex loss function

Characterization of an optimal vector estimator and an optimal matrix estimator are obtained. In each case appropriate convex loss functions are considered. The results are illustrated through the problems of simultaneous unbiased estimation, simultaneous equivariant estimation and simultaneous unbiased prediction. Further an optimality criterion is proposed for matrix unbiased estimation and it is shown that the matrix unbiased estimation of a matrix parametric function and the minimum variance unbiased estimation of its components are equivalent.     

Estimation of variance components in staggered nested designs

SUMMARY Variance components are estimated by two different methods for a general p stage random-effects staggered nested design. In addition to estimation from an analysis of variance, a new approach is introduced. The main features of this new technique are its simplicity and its ability to yield non-negative estimates of the variance components. The performances of the two procedures are compared using simulation and the meansquared-error criterion.     

Nonparametric analysis of treatment effects with missing observations

This paper develops a test for comparing treatment effects when observations are missing at random for repeated measures data on independent subjects. It is assumed that missingness at any occasion follows a Bernoulli distribution. It is shown that the distribution of the vector of linear rank statistics depends on the unknown parameters of the probability law that governs missingness, which is absent in the existing conditional methods employing rank statistics. This dependence is through the variance–covariance matrix of the vector of linear ranks. The test statistic is a quadratic form in the linear rank statistics when the variance–covariance matrix is estimated. The limiting distribution of the test statistic is derived under the null hypothesis. Several methods of estimating the unknown components of the variance–covariance matrix are considered. The estimate that produces stable empirical Type I error rate while maintaining the highest power among the competing tests is recommended for implementation in practice. Simulation studies are also presented to show the advantage of the proposed test over other rank-based tests that do not account for the randomness in the missing data pattern. Our method is shown to have the highest power while also maintaining near-nominal Type I error rates. Our results clearly illustrate that even for an ignorable missingness mechanism, the randomness in the pattern of missingness cannot be ignored. A real data example is presented to highlight the effectiveness of the proposed method.     

Linear Regression With Nested Errors Using Probability‐Linked Data

Probabilistic matching of records is widely used to create linked data sets for use in health science, epidemiological, economic, demographic and sociological research. Clearly, this type of matching can lead to linkage errors, which in turn can lead to bias and increased variability when standard statistical estimation techniques are used with the linked data. In this paper we develop unbiased regression parameter estimates to be used when fitting a linear model with nested errors to probabilistically linked data. Since estimation of variance components is typically an important objective when fitting such a model, we also develop appropriate modifications to standard methods of variance components estimation in order to account for linkage error. In particular, we focus on three widely used methods of variance components estimation: analysis of variance, maximum likelihood and restricted maximum likelihood. Simulation results show that our estimators perform reasonably well when compared to standard estimation methods that ignore linkage errors.     

Estimation of variance components in the mixed effects models: A comparison between analysis of variance and spectral decomposition

The mixed effects models with two variance components are often used to analyze longitudinal data. For these models, we compare two approaches to estimating the variance components, the analysis of variance approach and the spectral decomposition approach. We establish a necessary and sufficient condition for the two approaches to yield identical estimates, and some sufficient conditions for the superiority of one approach over the other, under the mean squared error criterion. Applications of the methods to circular models and longitudinal data are discussed. Furthermore, simulation results indicate that better estimates of variance components do not necessarily imply higher power of the tests or shorter confidence intervals.     

Bartlett's Test Applied to Variance Component Models

Exact methods for testing equality between variance components obtained from several cases of the same type of balanced orthogonal design are discussed. In particular, methods for successively testing equality of a number of components using Bartlett's tests are outlined for univariate and multivariate responses. Two clinical trial examples of repeated‐measures data are presented.     

Robust estimation in the multivariate normal model with variance components

Fisher consistent and Fréchet differentiable statistical functionals have been already used by Bednarski and Zontek [Robust estimation of parameters in a mixed unbalanced model. Ann Statist. 1996;24(4):1493–1510] to get a robust estimator of parameters in a two-way crossed classification mixed model. This way of robust estimation appears also in the variance components model with a commutative covariance matrix [Zmy?lony, Zontek. Robust M-estimator of parameters in variance components model. Discuss Math Probab Stat. 2002;22:61–71]. In this paper it is shown that a modification of this method does not involve any assumptions about commutation of covariance matrix. The theoretical results have been completed with computer simulation studies. Robustness of considered estimator and possibility of approximation of the estimator's distribution with some multivariate normal distribution for both model and contaminated data have been confirmed there.     

Generalized Estimation of the BLUP in Mixed-Effects Models: A Comparison with ML and REML

The Best Linear Unbiased Predictor (BLUP) in mixed models is a function of the variance components and they are estimated using maximum likelihood (ML) or restricted ML methods. Nonconvergence of BLUP would occur due to a drawback of the standard likelihood-based approaches. In such situations, ML and REML either do not provide any BLUPs or all become equal. To overcome this drawback, we provide a generalized estimate (GE) of BLUP that does not suffer from the problem of negative or zero variance components, and compare its performance against the ML and REML estimates of BLUP. Simulated and published data are used to compare BLUP.     

Estimation using penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models

We consider two estimation schemes based on penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models. The asymptotic bias in regression coefficients and variance components estimated by penalized quasilikelihood (PQL) is studied for small values of the variance components. We show the PQL estimators of both regression coefficients and variance components in Poisson mixed models have a smaller order of bias compared to those for binomial data. Unbiased estimating equations based on quasi-pseudo-likelihood are proposed and are shown to yield consistent estimators under some regularity conditions. The finite sample performance of these two methods is compared through a simulation study.     

Estimation of Change Point in Generalized Variance Control Chart

In this study, using maximum likelihood estimation, a considerably effective change point model is proposed for the generalized variance control chart in which the required statistics are calculated with its distributional properties. The procedure, when used with generalized variance control charts, would be helpful for practitioners both controlling the multivariate process dispersion and detecting the time of the change in variance-covariance matrix of a process. The procedure starts after the chart issues a signal. Several structural changes for the variance-covariance matrix are considered and the precision and the accuracy of the proposed method is discussed.     

Iterative weighted estimation based on variance modelling in linear regression models

ABSTRACT

The estimation of variance function plays an extremely important role in statistical inference of the regression models. In this paper we propose a variance modelling method for constructing the variance structure via combining the exponential polynomial modelling method and the kernel smoothing technique. A simple estimation method for the parameters in heteroscedastic linear regression models is developed when the covariance matrix is unknown diagonal and the variance function is a positive function of the mean. The consistency and asymptotic normality of the resulting estimators are established under some mild assumptions. In particular, a simple version of bootstrap test is adapted to test misspecification of the variance function. Some Monte Carlo simulation studies are carried out to examine the finite sample performance of the proposed methods. Finally, the methodologies are illustrated by the ozone concentration dataset.     

General formulae for expectations, variances and covariances of the mean squares for staggered nested designs

Staggered nested experimental designs are the most popular class of unbalanced nested designs. Using a special notation which covers the particular structure of the staggered nested design, this paper systematically derives the canonical form for the arbitrary m-factors. Under the normality assumption for every random variable, a vector comprising m canonical variables from each experimental unit is normally independently and identically distributed. Every sum of squares used in the analysis of variance (ANOVA) can be expressed as the sum of squares of the corresponding canonical variables. Hence, general formulae for the expectations, variances and covariances of the mean squares are directly obtained from the canonical form. Applying the formulae, the explicit forms of the ANOVA estimators of the variance components and unbiased estimators of the ratios of the variance components are introduced in this paper. The formulae are easily applied to obtain the variances and covariances of any linear combinations of the mean squares, especially the ANOVA estimators of the variance components. These results are eff ectively applied for the standardization of measurement methods.