Some factors affecting statistical power of approximate tests in the linear mixed model for longitudinal data

Different longitudinal study designs require different statistical analysis methods and different methods of sample size determination. Statistical power analysis is a flexible approach to sample size determination for longitudinal studies. However, different power analyses are required for different statistical tests which arises from the difference between different statistical methods. In this paper, the simulation-based power calculations of F-tests with Containment, Kenward-Roger or Satterthwaite approximation of degrees of freedom are examined for sample size determination in the context of a special case of linear mixed models (LMMs), which is frequently used in the analysis of longitudinal data. Essentially, the roles of some factors, such as variance–covariance structure of random effects [unstructured UN or factor analytic FA0], autocorrelation structure among errors over time [independent IND, first-order autoregressive AR1 or first-order moving average MA1], parameter estimation methods [maximum likelihood ML and restricted maximum likelihood REML] and iterative algorithms [ridge-stabilized Newton-Raphson and Quasi-Newton] on statistical power of approximate F-tests in the LMM are examined together, which has not been considered previously. The greatest factor affecting statistical power is found to be the variance–covariance structure of random effects in the LMM. It appears that the simulation-based analysis in this study gives an interesting insight into statistical power of approximate F-tests for fixed effects in LMMs for longitudinal data.     

The two-sample -test with a known ratio of variances

We consider the two-sample t-test where error variances are unknown but with known relationships between them. This situation arises, for example, when two measuring instruments average different number of replicates to report the response. In particular we compare our procedure with the usual Satterthwaite approximation in the two sample t-test with variances unequal. Our procedure uses the knowledge of a known ratio of variances while the Satterthwaite approximation assumes only that the two variances are unequal. Simulations show that our procedure has both better size and better power than the Satterthwaite approximation. Finally, we consider an extension of our results to the General Linear Model.     

Approximate testing in two-stage nonlinear mixed models

We investigate here small sample properties of approximate F-tests about fixed effects parameters in nonlinear mixed models. For estimation of population fixed effects parameters as well as variance components, we apply the two-stage approach. This method is useful and popular when the number of observations per sampling unit is large enough. The approximate F-test is constructed based on large-sample approximation to the distribution of nonlinear least-squares estimates of subject-specific parameters. We recommend a modified test statistic that takes into consideration approximation to the large-sample Fisher information matrix (See [Volaufova J, Burton JH. Note on hypothesis testing in mixed models. Oral presentation at: LINSTAT 2012/21st IWMS; 2012; Bedlewo, Poland]). Our main focus is on comparing finite sample properties of broadly used approximate tests (Wald test and likelihood ratio test) and the modified F-test under the null hypothesis, especially accuracy of p-values (See [Volaufova J, LaMotte L. Comparison of approximate tests of fixed effects in linear repeated measures design models with covariates. Tatra Mountains. 2008;39:17–25]). For that purpose two extensive simulation studies are conducted based on pharmacokinetic models (See [Hartford A, Davidian M. Consequences of misspecifying assumptions in nonlinear mixed effects models. Comput Stat and Data Anal. 2000;34:139–164; Pinheiro J, Bates D. Approximations to the log-likelihood function in the non-linear mixed-effects model. J Comput Graph Stat. 1995;4(1):12–35]).     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

Maximum studentized score tests for the detection of outliers in time series regression models

Efficient score tests exist among others, for testing the presence of additive and/or innovative outliers that are the result of the shifted mean of the error process under the regression model. A sample influence function of autocorrelation-based diagnostic technique also exists for the detection of outliers that are the result of the shifted autocorrelations. The later diagnostic technique is however not useful if the outlying observation does not affect the autocorrelation structure but is generated due to an inflation in the variance of the error process under the regression model. In this paper, we develop a unified maximum studentized type test which is applicable for testing the additive and innovative outliers as well as variance shifted outliers that may or may not affect the autocorrelation structure of the outlier free time series observations. Since the computation of the p-values for the maximum studentized type test is not easy in general, we propose a Satterthwaite type approximation based on suitable doubly non-central F-distributions for finding such p-values [F.E. Satterthwaite, An approximate distribution of estimates of variance components, Biometrics 2 (1946), pp. 110–114]. The approximations are evaluated through a simulation study, for example, for the detection of additive and innovative outliers as well as variance shifted outliers that do not affect the autocorrelation structure of the outlier free time series observations. Some simulation results on model misspecification effects on outlier detection are also provided.     

Optimal weighted two-sample t-test with partially paired data in a unified framework

In this paper, we provide a unified framework for two-sample t-test with partially paired data. We show that many existing two-sample t-tests with partially paired data can be viewed as special members in our unified framework. Some shortcomings of these t-tests are discussed. We also propose the asymptotically optimal weighted linear combination of the test statistics comparing all four paired and unpaired data sets. Simulation studies are used to illustrate the performance of our proposed asymptotically optimal weighted combinations of test statistics and compare with some existing methods. It is found that our proposed test statistic is generally more powerful. Three real data sets about CD4 count, DNA extraction concentrations, and the quality of sleep are also analyzed by using our newly introduced test statistic.     

Testing for Slope Heterogeneity Bias in Panel Data Models

ABSTRACT

Standard econometric methods can overlook individual heterogeneity in empirical work, generating inconsistent parameter estimates in panel data models. We propose the use of methods that allow researchers to easily identify, quantify, and address estimation issues arising from individual slope heterogeneity. We first characterize the bias in the standard fixed effects estimator when the true econometric model allows for heterogeneous slope coefficients. We then introduce a new test to check whether the fixed effects estimation is subject to heterogeneity bias. The procedure tests the population moment conditions required for fixed effects to consistently estimate the relevant parameters in the model. We establish the limiting distribution of the test and show that it is very simple to implement in practice. Examining firm investment models to showcase our approach, we show that heterogeneity bias-robust methods identify cash flow as a more important driver of investment than previously reported. Our study demonstrates analytically, via simulations, and empirically the importance of carefully accounting for individual specific slope heterogeneity in drawing conclusions about economic behavior.     

Approximate Small-Sample Tests of Fixed Effects in Nonlinear Mixed Models

Nonlinear mixed effect models have been studied extensively over several decades, particularly in pharmacokinetic and pharmacodynamic applications. Here, we focus on investigating the performance of commonly applied tests of linear hypotheses about the fixed effect parameters under different approximations to the likelihood function and to the estimated covariance matrix of the estimators. Included are the first-order approximation (FIRO), first-order conditional approximation (FOCE), and Gaussian quadrature approximation (AGQ) estimation methods. There is no straightforward way to mimic the approximations and adjustments taken in linear mixed models, such as the Kackar–Harville–Jeske–Kenward–Roger approach. By simulations, we illustrate the accuracy of p-values for the tests considered here. The observed results indicate that FOCE and AGQ estimation methods outperform FIRO. The test with an adjustment coefficient that takes into consideration the number of sampling units and the number of fixed effect parameters (Gallant-type) seems to perform closest to desirable even for small-sample sizes.     

Robustness without rank order statistics

An alternative to conventional rank tests based on a Euclidean distance analysis space is described. Comparisons based on exact probability values among classical two-sample t-tests and the Wilcoxon–Mann–Whitney test illustrate the advantages of the Euclidean distance analysis space alternative.     

Testing hypotheses about the common mean of normal distributions

An overview of hypothesis testing for the common mean of independent normal distributions is given. The case of two populations is studied in detail. A number of different types of tests are studied. Among them are a test based on the maximum of the two available t-tests, Fisher's combined test, a test based on Graybill–Deal's estimator, an approximation to the likelihood ratio test, and some tests derived using some Bayesian considerations for improper priors along with intuitive considerations. Based on some theoretical findings and mostly based on a Monte Carlo study the conclusions are that for the most part the Bayes-intuitive type tests are superior and can be recommended. When the variances of the populations are close the approximate likelihood ratio test does best.     

Testing inference in heteroskedastic linear regressions: a comparison of two alternative approaches

We consider the issue of performing testing inferences on the parameters that index the linear regression model under heteroskedasticity of unknown form. Quasi-t test statistics use asymptotically correct standard errors obtained from heteroskedasticity-consistent covariance matrix estimators. An alternative approach involves making an assumption about the functional form of the response variances and jointly modelling mean and dispersion effects. In this paper we compare the accuracy of testing inferences made using the two approaches. We consider several different quasi-t tests and also z tests performed after estimated generalized least squares estimation which was carried out using three different estimation strategies. The numerical evidence shows that some quasi-t tests are typically considerably less size distorted in small samples than the tests carried out after the jointly modelling of mean and dispersion effects. Finally, we present and discuss two empirical applications.     

UNIFORM ASYMPTOTIC EXPANSIONS APPLIED TO SEQUENTIAL t - and t2-TESTS

Using some uniform asymptotic expansions for parabolic cylinder functions recently developed by Olver (1959), various integrals associated with the sequential t- and t² -tests are evaluated asymptotically in terms of the sample size. Then the continuation region inequalities for these tests are inverted and expressed in terms of well known test criteria. It should be pointed out that the inversion of the continuation regions in terms of the well known statistics yields forms for the sequential tests that are more easily applicable by the practitioner than the forms yielded by the method of Rushton. Furthermore, using these inequalities and the asymptotic normality of the test criteria, finite sure termination of sequential t- and t²-test procedures readily follow. Based on simulation studies, power comparisons of the two approximations are also made.     

Assessing the performance of variational methods for mixed logistic regression models

We present a variational estimation method for the mixed logistic regression model. The method is based on a lower bound approximation of the logistic function [Jaakkola, J.S. and Jordan, M.I., 2000, Bayesian parameter estimation via variational methods. Statistics & Computing, 10, 25–37.]. Based on the approximation, an EM algorithm can be derived that results in a considerable simplification of the maximization problem in that it does not require the numerical evaluation of integrals over the random effects. We assess the performance of the variational method for the mixed logistic regression model in a simulation study and an empirical data example, and compare it to Laplace's method. The results indicate that the variational method is a viable choice for estimating the fixed effects of the mixed logistic regression model under the condition that the number of outcomes within each cluster is sufficiently high.     

Bootstrapping p Values and Power in the First-Order Autoregression: A Monte Carlo Investigation

The small-sample behavior of the bootstrap is investigated as a method for estimating p values and power in the stationary first-order autoregressive model. Monte Carlo methods are used to examine the bootstrap and Student-t approximations to the true distribution of the test statistic frequently used for testing hypotheses on the underlying slope parameter. In contrast to Student's t, the results suggest that the bootstrap can accurately estimate p values and power in this model in sample sizes as small as 5–10.     

Preliminary test almost unbiased two-parameter estimators with student’s t errors and conflicting test statistics

Abstract

In this paper, we consider the preliminary test approach to the estimation of the regression parameter in a multiple regression model under multicollinearity situation. The preliminary test almost unbiased two-parameter estimators based on the Wald, the Likelihood ratio, and the Lagrangian multiplier tests are given, when it is suspected that the regression parameter may be restricted to a subspace and the regression error is distributed with multivariate Student’s t errors. The bias and quadratic risk of the proposed estimators are derived and compared. Furthermore, a Monte Carlo simulation is provided to illustrate some of the theoretical results.     

A Revisit to the Behrens–Fisher Problem: Comparison of Five Test Methods

We revisit the well-known Behrens–Fisher problem and apply a newly developed ‘Computational Approach Test’ (CAT) to test the equality of two population means where the populations are assumed to be normal with unknown and possibly unequal variances. An advantage of the CAT is that it does not require the explicit knowledge of the sampling distribution of the test statistic. The CAT is then compared with three widely accepted tests—Welch–Satterthwaite test (WST), Cochran–Cox test (CCT), ‘Generalized p-value’ test (GPT)—and a recently suggested test based on the jackknife procedure, called Singh–Saxena–Srivastava test (SSST). Further, model robustness of these five tests are studied when the data actually came from t-distributions, but wrongly perceived as normal ones. Our detailed study based on a comprehensive simulation indicate some interesting results including the facts that the GPT is quite conservative, and the SSST is not as good as it has been claimed in the literature. To the best of our knowledge, the trends observed in our study have not been reported earlier in the existing literature.     

Mixture of linear mixed models using multivariate t distribution

Linear mixed models are widely used when multiple correlated measurements are made on each unit of interest. In many applications, the units may form several distinct clusters, and such heterogeneity can be more appropriately modelled by a finite mixture linear mixed model. The classical estimation approach, in which both the random effects and the error parts are assumed to follow normal distribution, is sensitive to outliers, and failure to accommodate outliers may greatly jeopardize the model estimation and inference. We propose a new mixture linear mixed model using multivariate t distribution. For each mixture component, we assume the response and the random effects jointly follow a multivariate t distribution, to conveniently robustify the estimation procedure. An efficient expectation conditional maximization algorithm is developed for conducting maximum likelihood estimation. The degrees of freedom parameters of the t distributions are chosen data adaptively, for achieving flexible trade-off between estimation robustness and efficiency. Simulation studies and an application on analysing lung growth longitudinal data showcase the efficacy of the proposed approach.     

Tests of regression coefficients under ridge regression models

This paper investigates two “non-exact” t-type tests, t( k₂) and t(k₂), of the individual coefficients of a linear regression model, based on two ordinary ridge estimators. The reported results are built on a simulation study covering 84 different models. For models with large standard errors, the ridge-based t-tests have correct levels with considerable gain in powers over those of the least squares t-test, t(0). For models with small standard errors, t(k₁) is found to be liberal and is not safe to use while, t(k₂) is found to slightly exceed the nominal level in few cases. When tie two ridge tests art: not winners, the results indicate that they don't loose much against t(0).     

Cumulative/dynamic ROC curve estimation

ABSTRACT

Receiver operating-characteristic (ROC) curve is a popular graphical method frequently used in order to study the diagnostic capacity of continuous (bio)markers. When the considered outcome is a time-dependent variable, the direct generalization is known as cumulative/dynamic ROC curve. For a fixed point of time, t, one subject is allocated into the positive group if the event happens before t and into the negative group if the event is not happened at t. The presence of censored subject, which can not be directly assigned into a group, is the main handicap of this approach. The proposed cumulative/dynamic ROC curve estimator assigns a probability to belong to the negative (positive) group to the subjects censored previously to t. The performance of the resulting estimator is studied from Monte Carlo simulations. Some real-world applications are reported. Results suggest that the new estimators provide a good approximation to the real cumulative/dynamic ROC curve.     

C65. On resolving statistical hypotheses

A number of procedures for testing adequacy of polynomial approximations to growth curves based on Rao’s test for additional information, Grizzle and Allen’s test or univariate t-tests were compared using data simulated from quadratic models. Quadratic models were indicated as adequately fitting the data in 95.10± 0.10 percent of analyses when the degree of the approximating polynomial was determined by the lowest-order significant coefficient (P = 0.05) that was followed by two successive nonsignificant ones according to separate t-tests. Procedures based on the Grizzle and Allen test and modifications of it indicated quadratic models in 86.70 ± 0.41 to 95.34 ±0.21 percent of analyses depending on error structure, variance and the number of coefficients analysed together. The t-test would be preferred in practice as its performance did not depend on the error structure or variance.