A prediction analysis in a constrained multivariate general linear model with future observations

Abstract

We give a mathematical analysis to some fundamental prediction problems on a constrained multivariate general linear model (CMGLM) with future observations, including the derivation of analytical formulas for calculating the best linear unbiased predictors (BLUPs) of all unknown parameter matrices, and the presentation of many novel and valuable properties of the BLUPs.     

Effect of adding regressors on the equality of the BLUEs under two linear models

In this paper we consider the estimation of regression coefficients in two partitioned linear models, shortly denoted as , and , which differ only in their covariance matrices. We call and full models, and correspondingly, and small models. We give a necessary and sufficient condition for the equality between the best linear unbiased estimators (BLUEs) of X₁β₁ under and . In particular, we consider the equality of the BLUEs under the full models assuming that they are equal under the small models.     

Reference priors for linear models with general covariance structures

We develop a new class of reference priors for linear models with general covariance structures. A general Markov chain Monte Carlo algorithm is also proposed for implementing the computation. We present several examples to demonstrate the results: Bayesian penalized spline smoothing, a Bayesian approach to bivariate smoothing for a spatial model, and prior specification for structural equation models.     

A doubly robust goodness-of-fit test in general linear models with missing covariates

In this article, we utilize a form of general linear model where missing data occurred randomly on the covariates. We propose a test function based on the doubly robust method to investigate goodness of fit of the model. For this aim, kernel method is used to estimate unknown functions under estimating equation method. Doubly robustness and asymptotic properties of the test function are obtained under local and alternative hypotheses. Furthermore, we investigate the power of the proposed test function by means of some simulation studies and finally we apply this method on analyzing a real dataset.     

A new multicollinearity diagnostic for generalized linear models

We propose a new collinearity diagnostic tool for generalized linear models. The new diagnostic tool is termed the weighted variance inflation factor (WVIF) behaving exactly the same as the traditional variance inflation factor in the context of regression diagnostic, given data matrix normalized. Compared to the use of condition number (CN), WVIF shows more reliable information on how severe the situation is, when data collinearity does exist. An alternative estimator, a by-product of the new diagnostic, outperforms the ridge estimator in the presence of data collinearity in both aspects of WVIF and CN. Evidences are given through analyzing various real-world numerical examples.     

Log linear models for multivariate paired comparison experiments with ties

A log linear multivariate paired comparison model for ties is proposed in which the cell probabilities under independence are those given by Davidson (1970). Altham's (1970) generalized measure of association (iv) is used to compare the association structure between two models, one having full, the other having reduced association structure. Based on the model with reduced association structure, the analysis of data from a consumer preference experiment is presented.     

Goodness-of-fit tests for general linear models with covariates missed at random

In this paper, we consider a model checking problem for general linear models with randomly missing covariates. Two types of score type tests with inverse probability weight, which is estimated by parameter and nonparameter methods respectively, are proposed to this goodness of fit problem. The asymptotic properties of the test statistics are developed under the null and local alternative hypothesis. Simulation study is carried out to present the performance of the sizes and powers of the tests. We illustrate the proposed method with a data set on monozygotic twins.     

Empirical likelihood for generalized linear models with fixed and adaptive designs

Empirical likelihood inference for generalized linear models with fixed and adaptive designs is considered. It is shown that the empirical log-likelihood ratio at the true parameters converges to the standard chi-square distribution. Furthermore, we obtain the maximum empirical likelihood estimate of the unknown parameter and the resulting estimator is shown to be asymptotically normal. Some simulations are conducted to illustrate the proposed method.     

Influence on tests with focus on linear models

To assess the influence of single observations on the parameter estimates, case-deletion diagnostics are commonly used in linear regression models; one example is Cook's distance. For nested parametric models we consider a deletion diagnostic for evaluating the influence of a single observation on the likelihood ratio (LR) test. In order to have a common scale as reference, the asymptotic distribution of the diagnostic is derived and the values of the diagnostic are converted to percentiles. We focus on linear models and general linear models, and in these cases explicit results are derived. The performance of the diagnostic is explored in two small bench mark examples from linear regression and in a larger linear mixed model example.     

Outlier detection and robust estimation in linear regression models with fixed group effects

This work studies outlier detection and robust estimation with data that are naturally distributed into groups and which follow approximately a linear regression model with fixed group effects. For this, several methods are considered. First, the robust fitting method of Peña and Yohai [A fast procedure for outlier diagnostics in large regression problems. J Am Stat Assoc. 1999;94:434–445], called principal sensitivity components (PSC) method, is adapted to the grouped data structure and the mentioned model. The robust methods RDL₁ of Hubert and Rousseeuw [Robust regression with both continuous and binary regressors. J Stat Plan Inference. 1997;57:153–163] and M-S of Maronna and Yohai [Robust regression with both continuous and categorical predictors. Journal of Statistical Planning and Inference 2000;89:197–214] are also considered. These three methods are compared in terms of their effectiveness in outlier detection and their robustness through simulations, considering several contamination scenarios and growing contamination levels. Results indicate that the adapted PSC procedure is able to detect a high percentage of true outliers and a small number of false outliers. It is appropriate when the contamination is in the error term or in the covariates, detecting also possibly masked high leverage points. Moreover, in simulations the final robust regression estimator preserved good efficiency under Normality while keeping good robustness properties.     

A new orthogonality-based estimation for varying-coefficient partially linear models

Varying coefficient partially linear models are usually used for longitudinal data analysis, and an interest is mainly to improve efficiency of regression coefficients. By the orthogonality estimation technology and the quadratic inference function method, we propose a new orthogonality-based estimation method to estimate parameter and nonparametric components in varying coefficient partially linear models with longitudinal data. The proposed procedure can separately estimate the parametric and nonparametric components, and the resulting estimators do not affect each other. Under some mild conditions, we establish some asymptotic properties of the resulting estimators. Furthermore, the finite sample performance of the proposed procedure is assessed by some simulation experiments.     

A new test for random effects in linear mixed models with longitudinal data

As it is known, testing the existence of random effects is often transferred to testing their zero variances/covariance matrices. It is a nonstandard testing problem because the hypothetical values are on the boundary of the whole space. In the literature, a difference-based test was proposed, which has asymptotically tractable null distribution and is then easy to implement. However, the projection method on which the difference-based test relies may affect and deteriorate its performance when covariates associated with fixed effects and covariates associated with random effects are highly correlated. In the paper, for linear mixed models (LMM) with longitudinal data, a new test is proposed to avoid this problem. The new test is also asymptotically distribution-free and more powerful than the difference-based test, particularly when the above correlation is high. The new test is consistent against all global alternatives and can detect local alternatives converging to the null at a rate as close as to m^−1/2$m^{-1/2}$ with m being the number of subjects. Simulations are carried out to examine the performance and a real data analysis is performed for illustration.     

A test of separate hypotheses for comparing linear mixed models with non nested fixed effects

As researchers increasingly rely on linear mixed models to characterize longitudinal data, there is a need for improved techniques for selecting among this class of models which requires specification of both fixed and random effects via a mean model and variance-covariance structure. The process is further complicated when fixed and/or random effects are non nested between models. This paper explores the development of a hypothesis test to compare non nested linear mixed models based on extensions of the work begun by Sir David Cox. We assess the robustness of this approach for comparing models containing correlated measures of body fat for predicting longitudinal cardiometabolic risk.     

A comparison of bootstrap approaches for estimating uncertainty of parameters in linear mixed‐effects models

A version of the nonparametric bootstrap, which resamples the entire subjects from original data, called the case bootstrap, has been increasingly used for estimating uncertainty of parameters in mixed‐effects models. It is usually applied to obtain more robust estimates of the parameters and more realistic confidence intervals (CIs). Alternative bootstrap methods, such as residual bootstrap and parametric bootstrap that resample both random effects and residuals, have been proposed to better take into account the hierarchical structure of multi‐level and longitudinal data. However, few studies have been performed to compare these different approaches. In this study, we used simulation to evaluate bootstrap methods proposed for linear mixed‐effect models. We also compared the results obtained by maximum likelihood (ML) and restricted maximum likelihood (REML). Our simulation studies evidenced the good performance of the case bootstrap as well as the bootstraps of both random effects and residuals. On the other hand, the bootstrap methods that resample only the residuals and the bootstraps combining case and residuals performed poorly. REML and ML provided similar bootstrap estimates of uncertainty, but there was slightly more bias and poorer coverage rate for variance parameters with ML in the sparse design. We applied the proposed methods to a real dataset from a study investigating the natural evolution of Parkinson's disease and were able to confirm that the methods provide plausible estimates of uncertainty. Given that most real‐life datasets tend to exhibit heterogeneity in sampling schedules, the residual bootstraps would be expected to perform better than the case bootstrap. Copyright © 2013 John Wiley & Sons, Ltd.     

A non-iterative sampling Bayesian method for linear mixed models with normal independent distributions

In this paper, we utilize normal/independent (NI) distributions as a tool for robust modeling of linear mixed models (LMM) under a Bayesian paradigm. The purpose is to develop a non-iterative sampling method to obtain i.i.d. samples approximately from the observed posterior distribution by combining the inverse Bayes formulae, sampling/importance resampling and posterior mode estimates from the expectation maximization algorithm to LMMs with NI distributions, as suggested by Tan et al. [33 Tan, M., Tian, G. and Ng, K. 2003. A noniterative sampling method for computing posteriors in the structure of EM-type algorithms. Statist. Sinica, 13(3): 625–640. [Web of Science ®] , [Google Scholar]]. The proposed algorithm provides a novel alternative to perfect sampling and eliminates the convergence problems of Markov chain Monte Carlo methods. In order to examine the robust aspects of the NI class, against outlying and influential observations, we present a Bayesian case deletion influence diagnostics based on the Kullback–Leibler divergence. Further, some discussions on model selection criteria are given. The new methodologies are exemplified through a real data set, illustrating the usefulness of the proposed methodology.     

Estimation of variance components in the mixed effects models: A comparison between analysis of variance and spectral decomposition

The mixed effects models with two variance components are often used to analyze longitudinal data. For these models, we compare two approaches to estimating the variance components, the analysis of variance approach and the spectral decomposition approach. We establish a necessary and sufficient condition for the two approaches to yield identical estimates, and some sufficient conditions for the superiority of one approach over the other, under the mean squared error criterion. Applications of the methods to circular models and longitudinal data are discussed. Furthermore, simulation results indicate that better estimates of variance components do not necessarily imply higher power of the tests or shorter confidence intervals.