The effect of number of clusters and cluster size on statistical power and Type I error rates when testing random effects variance components in multilevel linear and logistic regression models

When using multilevel regression models that incorporate cluster-specific random effects, the Wald and the likelihood ratio (LR) tests are used for testing the null hypothesis that the variance of the random effects distribution is equal to zero. We conducted a series of Monte Carlo simulations to examine the effect of the number of clusters and the number of subjects per cluster on the statistical power to detect a non-null random effects variance and to compare the empirical type I error rates of the Wald and LR tests. Statistical power increased with increasing number of clusters and number of subjects per cluster. Statistical power was greater for the LR test than for the Wald test. These results applied to both the linear and logistic regressions, but were more pronounced for the latter. The use of the LR test is preferable to the use of the Wald test.     

Robust Small Sample Inference for Generalised Estimating Equations: An Application of the Anova‐type Test

Asymptotically, the Wald‐type test for generalised estimating equations (GEE) models can control the type I error rate at the nominal level. However in small sample studies, it may lead to inflated type I error rates. Even with currently available small sample corrections for the GEE Wald‐type test, the type I error rate inflation is still serious when the tested contrast is multidimensional. This paper extends the ANOVA‐type test for heteroscedastic factorial designs to GEE and shows that the proposed ANOVA‐type test can also control the type I error rate at the nominal level in small sample studies while still maintaining robustness with respect to mis‐specification of the working correlation matrix. Differences of inference between the Wald‐type test and the proposed test are observed in a two‐way repeated measures ANOVA model for a diet‐induced obesity study and a two‐way repeated measures logistic regression for a collagen‐induced arthritis study. Simulation studies confirm that the proposed test has better control of the type I error rate than the Wald‐type test in small sample repeated measures models. Additional simulation studies further show that the proposed test can even achieve larger power than the Wald‐type test in some cases of the large sample repeated measures ANOVA models that were investigated.     

A comment on sample size calculations for binomial confidence intervals

In this article we examine sample size calculations for a binomial proportion based on the confidence interval width of the Agresti–Coull, Wald and Wilson Score intervals. We pointed out that the commonly used methods based on known and fixed standard errors cannot guarantee the desired confidence interval width given a hypothesized proportion. Therefore, a new adjusted sample size calculation method was introduced, which is based on the conditional expectation of the width of the confidence interval given the hypothesized proportion. With the reduced sample size, the coverage probability can still maintain at the nominal level and is very competitive to the converge probability for the original sample size.     

Testing for multivariate heteroscedasticity

In this article, we propose a testing technique for multivariate heteroscedasticity, which is expressed as a test of linear restrictions in a multivariate regression model. Four test statistics with known asymptotical null distributions are suggested, namely the Wald, Lagrange multiplier (LM), likelihood ratio (LR) and the multivariate Rao F-test. The critical values for the statistics are determined by their asymptotic null distributions, but bootstrapped critical values are also used. The size, power and robustness of the tests are examined in a Monte Carlo experiment. Our main finding is that all the tests limit their nominal sizes asymptotically, but some of them have superior small sample properties. These are the F, LM and bootstrapped versions of Wald and LR tests.     

Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions

For interval estimation of a proportion, coverage probabilities tend to be too large for “exact” confidence intervals based on inverting the binomial test and too small for the interval based on inverting the Wald large-sample normal test (i.e., sample proportion ± z-score × estimated standard error). Wilson's suggestion of inverting the related score test with null rather than estimated standard error yields coverage probabilities close to nominal confidence levels, even for very small sample sizes. The 95% score interval has similar behavior as the adjusted Wald interval obtained after adding two “successes” and two “failures” to the sample. In elementary courses, with the score and adjusted Wald methods it is unnecessary to provide students with awkward sample size guidelines.     

Preliminary test Liu-type estimators based on W,LR, and LM test statistics in a regression model

This article discusses the preliminary test approach for the regression parameter in multiple regression model. The preliminary test Liu-type estimators based on the Wald (W), Likelihood ratio (LR), and Lagrangian multiplier(LM) tests are presented, when it is supposed that the regression parameter may be restricted to a subspace. We also give the bias and mean squared error of the proposed estimators and the superior of the proposed estimators is also discussed.     

Empirical Study of Six Tests for Equality of Populations with Zero-Inflated Continuous Distributions

We evaluated the properties of six statistical methods for testing equality among populations with zero-inflated continuous distributions. These tests are based on likelihood ratio (LR), Wald, central limit theorem (CLT), modified CLT (MCLT), parametric jackknife (PJ), and nonparametric jackknife (NPJ) statistics. We investigated their statistical properties using simulated data from mixed distributions with an unknown portion of non zero observations that have an underlying gamma, exponential, or log-normal density function and the remaining portion that are excessive zeros. The 6 statistical tests are compared in terms of their empirical Type I errors and powers estimated through 10,000 repeated simulated samples for carefully selected configurations of parameters. The LR, Wald, and PJ tests are preferred tests since their empirical Type I errors were close to the preset nominal 0.05 level and each demonstrated good power for rejecting null hypotheses when the sample sizes are at least 125 in each group. The NPJ test had unacceptable empirical Type I errors because it rejected far too often while the CLT and MCLT tests had low testing powers in some cases. Therefore, these three tests are not recommended for general use but the LR, Wald, and PJ tests all performed well in large sample applications.     

More on the Preliminary Test Estimator in Almost Unbiased Liu Regression

This article is concerned with the parameter estimation in linear regression model when it is suspected that the regression coefficients are the subspace of the equality restrictions. The objective of this article is to introduce the preliminary test almost unbiased Liu estimators (PTAULE) based on the Wald (W), the likelihood ratio (LR), and the Lagrangian multiplier (LM) tests and compare the proposed estimators in the sense of the quadratic bias and mean square error (MSE) criterion.     

Improved small sample inference in the mixed linear model: Bartlett correction and adjusted likelihood

The mixed linear model is a popular method for analysing unbalanced repeated measurement data. The classical statistical tests for parameters in this model are based on asymptotic theory that is unreliable in the small samples that are often encountered in practice. For testing a given fixed effect parameter with a small sample, we develop and investigate refined likelihood ratio (LR) tests. The refinements considered are the Bartlett correction and use of the Cox–Reid adjusted likelihood; these are examined separately and in combination. We illustrate the various LR tests on an actual data set and compare them in two simulation studies. The conventional LR test yields type I error rates that are higher than nominal. The adjusted LR test yields rates that are lower than nominal, with absolute accuracy similar to that of the conventional LR test in the first simulation study and better in the second. The Bartlett correction substantially improves the accuracy of the type I error rates with either the conventional or the adjusted LR test. In many cases, error rates that are very close to nominal are achieved with the refined methods.     

A monte carlo study of the wald,lm, and lr tests in a heteroscedastic linear model

In this paper, we examine by Monte Carlo experiments the small sample properties of the W (Wald), LM (Lagrange Multiplier) and LR (Likelihood Ratio) tests for equality between sets of coefficients in two linear regressions under heteroscedasticity. The small sample properties of the size-corrected W, LM and LR tests proposed by Rothenberg (1984) are also examined and it is shown that the performances of the size-corrected W and LM tests are very good. Further, we examine the two-stage test which consists of a test for homoscedasticity followed by the Chow (1960) test if homoscedasticity is indicated or one of the W, LM or LR tests if heteroscedasticity should be assumed. It is shown that the pretest does not reduce much the bias in the size when the sizecorrected citical values are used in the W, LM and LR tests.     

Testing for panel cointegration in an error-correction framework with an application to the Fisher hypothesis

In this article, three innovative panel error-correction model (PECM) tests are proposed. These tests are based on the multivariate versions of the Wald (W), likelihood ratio (LR), and Lagrange multiplier (LM) tests. Using Monte Carlo simulations, the size and power of the tests are investigated when the error terms exhibit both cross-sectional dependence and independence. We find that the LM test is the best option when the error terms follow independent white-noise processes. However, in the more empirically relevant case of cross-sectional dependence, we conclude that the W test is the optimal choice. In contrast to previous studies, our method is general and does not rely on the strict assumption that a common factor causes the cross-sectional dependency. In an empirical application, our method is also demonstrated in terms of the Fisher effect—a hypothesis about the existence of which there is still no clear consensus. Based on our sample of the five Nordic countries we utilize our powerful test and discover evidence which, in contrast to most previous research, confirms the Fisher effect.     

Preliminary Test Estimators Induced by Three Large Sample Tests for Stochastic Constraints in a Regression Model with Multivariate Student-t Error

In this article, we consider the preliminary test approach to the estimation of the regression parameter in a multiple regression model with multivariate Student-t distribution. The preliminary test estimators (PTE) based on the Wald (W), Likelihood Ratio (LR), and Lagrangian Multiplier (LM) tests are given under the suspicion of stochastic constraints occurring. The bias, mean square error matr ix (MSEM), and weighted mean square error (WMSE) of the proposed estimators are derived and compared. The conditions of superiority of the proposed estimators are obtained. Finally, we conclude that the optimum choice of the level of significance becomes the traditional choice by using the W test.     

Comparison to control in logistic regression

We are interested in comparing logistic regressions for several test treatments or populations with a logistic regression for a standard treatment or population. The research was motivated by some real life problems, which are discussed as data examples. We propose a step-down likelihood ratio method for declaring differences between the test treatments or populations and the standard treatment or population. Competitors based on the sequentially rejective Bonferroni Wald statistic, sequentially rejective exact Wald statistic and Reiers?l's statistic are also discussed. It is shown that the proposed method asymptotically controls the probability of type I error. A Monte Carlo simulation shows that the proposed method performs well for relatively small sample sizes, outperforming its competitors.     

Confidence regions for two proportions from independent negative binomial distributions

The negative binomial distribution offers an alternative view to the binomial distribution for modeling count data. This alternative view is particularly useful when the probability of success is very small, because, unlike the fixed sampling scheme of the binomial distribution, the inverse sampling approach allows one to collect enough data in order to adequately estimate the proportion of success. However, despite work that has been done on the joint estimation of two binomial proportions from independent samples, there is little, if any, similar work for negative binomial proportions. In this paper, we construct and investigate three confidence regions for two negative binomial proportions based on three statistics: the Wald (W), score (S) and likelihood ratio (LR) statistics. For large-to-moderate sample sizes, this paper finds that all three regions have good coverage properties, with comparable average areas for large sample sizes but with the S method producing the smaller regions for moderate sample sizes. In the small sample case, the LR method has good coverage properties, but often at the expense of comparatively larger areas. Finally, we apply these three regions to some real data for the joint estimation of liver damage rates in patients taking one of two drugs.     

Statistical inference of risk difference in K correlated 2×2 tables with structural zero

K correlated 2×2 tables with structural zero are commonly encountered in infectious disease studies. A hypothesis test for risk difference is considered in K independent 2×2 tables with structural zero in this paper. Score statistic, likelihood ratio statistic and Wald‐type statistic are proposed to test the hypothesis on the basis of stratified data and pooled data. Sample size formulae are derived for controlling a pre‐specified power or a pre‐determined confidence interval width. Our empirical results show that score statistic and likelihood ratio statistic behave better than Wald‐type statistic in terms of type I error rate and coverage probability, sample sizes based on stratified test are smaller than those based on the pooled test in the same design. A real example is used to illustrate the proposed methodologies. Copyright © 2009 John Wiley & Sons, Ltd.     

A simple investigation of the Granger-causality test in integrated-cointegrated VAR systems

The size and power of various generalization tests for the Granger-causality in integrated-cointegrated VAR systems are considered. By using Monte Carlo methods, properties of eight versions of the test are studied in two different forms, the standard form and the modified form by Dolado & Lütkepohl (1996) in a study confined to properties of the Wald test only. In their study as well as in ours, both the standard and the modified Wald tests are shown to perform badly especially in small samples. We find, however, that the corrected LR tests exhibit correct size even in small samples. The power of the test is higher when the true VAR(2) model is estimated, and the modified test loses information by estimating the extra coefficients. The same is true when considering the power results in the VAR(3) model, and the power of the tests is somewhat lower than those in the VAR(2).     

One-way analysis of proportions for misclassified binomial data

We consider data with a nominal grouping variable and a binary response variable. The grouping variable is measured without error, but the response variable is measured using a fallible device subject to misclassification. To achieve model identifiability, we use the double-sampling scheme which requires obtaining a subsample of the original data or another independent sample. This sample is then classified by both the fallible device and another infallible device regarding the response variable. We propose two Wald tests for testing the association between the two variables and illustrate the test using traffic data. The Type-I error rate and power of the tests are examined using simulations and a modified Wald test is recommended.     

Sequential analysis methodology for a Poisson GLMM with applications to multicenter randomized clinical trials

Sequential analyses in clinical trials have ethical and economic advantages over fixed sample size methods. The sequential probability ratio test (SPRT) is a hypothesis testing procedure which evaluates data as it is collected. The original SPRT was developed by Wald for one-parameter families of distributions and later extended by Bartlett to handle the case of nuisance parameters. However, Bartlett's SPRT requires independent and identically distributed observations. In this paper we show that Bartlett's SPRT can be applied to generalized linear model (GLM) contexts. Then we propose an SPRT analysis methodology for a Poisson generalized linear mixed model (GLMM) that is suitable for our application to the design of a multicenter randomized clinical trial that compares two preventive treatments for surgical site infections. We validate the methodology with a simulation study that includes a comparison to Neyman–Pearson and Bayesian fixed sample size test designs and the Wald SPRT.     

Estimating Desired Sample Size for Simple Random Sampling of a Skewed Population

A simulation study was conducted to assess how well the necessary sample size to achieve a stipulated margin of error can be estimated prior to sampling. Our concern was particularly focused on performance when sampling from a very skewed distribution, which is a common feature of many biological, economic, and other populations. We examined two approaches for estimating sample size—one being the commonly used strategy aimed at regulating the average magnitude of the stipulated margin of error and the second being a previously proposed strategy to control the tolerance probability with which the stipulated margin of error is exceeded. Results of the simulation revealed that (1) skewness does not much affect the average estimated sample size but can greatly extend the range of estimated sample sizes; and (2) skewness does reduce the effectiveness of Kupper and Hafner's sample size estimator, yet its effectiveness is negatively impacted less by skewness directly, and to a much greater degree by the common practice of estimating the population variance via a pilot sampling from the skewed population. Nonetheless, the simulations suggest that estimating sample size to control the probability with which the desired margin of error is achieved is a worthwhile alternative to the usual sample size formula that controls the average width of the confidence interval only.     

Asymptotic relative efficiency of wald tests in measurement error models

In this paper, asymptotic relative efficiency (ARE) of Wald tests for the Tweedie class of models with log-linear mean, is considered when the aux¬iliary variable is measured with error. Wald test statistics based on the naive maximum likelihood estimator and on a consistent estimator which is obtained by using Nakarnura's (1990) corrected score function approach are defined. As shown analytically, the Wald statistics based on the naive and corrected score function estimators are asymptotically equivalents in terms of ARE. On the other hand, the asymptotic relative efficiency of the naive and corrected Wald statistic with respect to the Wald statistic based on the true covariate equals to the square of the correlation between the unobserved and the observed co-variate. A small scale numerical Monte Carlo study and an example illustrate the small sample size situation.