Parametric Bootstrap Tests for Unbalanced Three-factor Nested Designs under Heteroscedasticity

In this article, we consider the three-factor unbalanced nested design model without the assumption of equal error variance. For the problem of testing “main effects” of the three factors, we propose a parametric bootstrap (PB) approach and compare it with the existing generalized F (GF) test. The Type I error rates of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the generalized F-test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of the random effect variance component in a three-factor mixed effects nested model under unequal error variances.     

A Parametric Bootstrap Test for Two-Way ANOVA Model Without Interaction Under Heteroscedasticity

In this article we consider the two-way ANOVA model without interaction under heteroscedasticity. For the problem of testing equal effects of factors, we propose a parametric bootstrap (PB) approach and compare it with existing the generalized F (GF) test. The Type I error rates and powers of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the GF test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of random effect variance component in a two-way mixed-effects model under unequal error variances.     

A parametric bootstrap solution to the MANOVA under heteroscedasticity

In this article, we consider the problem of comparing several multivariate normal mean vectors when the covariance matrices are unknown and arbitrary positive definite matrices. We propose a parametric bootstrap (PB) approach and develop an approximation to the distribution of the PB pivotal quantity for comparing two mean vectors. This approximate test is shown to be the same as the invariant test given in [Krishnamoorthy and Yu, Modified Nel and Van der Merwe test for the multivariate Behrens–Fisher problem, Stat. Probab. Lett. 66 (2004), pp. 161–169] for the multivariate Behrens–Fisher problem. Furthermore, we compare the PB test with two existing invariant tests via Monte Carlo simulation. Our simulation studies show that the PB test controls Type I error rates very satisfactorily, whereas other tests are liberal especially when the number of means to be compared is moderate and/or sample sizes are small. The tests are illustrated using an example.     

A Parametric Bootstrap Solution to Two-way MANOVA without Interaction under Heteroscedasticity: Fixed and Mixed Models

In this article, we propose a parametric bootstrap (PB) test for heteroscedastic two-way multivariate analysis of variance without Interaction. For the problem of testing equal main effects of factors, we obtain a PB approach and compare it with existing modified Brown–Forsythe (MBF) test and approximate Hotelling T² (AHT) test by an extensive simulation study. The PB test is a symmetric function in samples, and does not depend on the chosen weights used to define the parameters uniquely. Simulation results indicate that the PB test performs satisfactorily for various cell sizes and parameter configurations when the homogeneity assumption is seriously violated, and tends to outperform the AHT test for moderate or larger samples in terms of power and controlling size. The MBF test, the AHT test, and the PB test have similar robustness to violations of underlying assumptions. It is also noted that the same PB test can be used to test the significance of random effect vector in a two-way multivariate mixed effects model with unequal cell covariance matrices.     

Testing the equality of several inverse Gaussian means under heterogeneity

Abstract

We consider the problem of testing the equality of several inverse Gaussian means when the scale parameters and sample sizes are possibly unequal. We propose four parametric bootstrap (PB) tests based on the uniformly minimum variance unbiased estimators of parameters. We also compare our proposed tests with the existing ones via an extensive simulation study in terms of controlling the Type I error rate and power performance. Simulation results show the merits of the PB tests.     

Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

Choosing the best pairwise comparisons of means from non-normal populations,with unequal variances,but equal sample sizes

A Monte Carlo simulation evaluated five pairwise multiple comparison procedures for controlling Type I error rates, any-pair power, and all-pairs power. Realistic conditions of non-normality were based on a previous survey. Variance ratios were varied from 1:1 to 64:1. Procedures evaluated included Tukey's honestly significant difference (HSD) preceded by an F test, the Hayter–Fisher, the Games–Howell preceded by an F test, the Pertiz with F tests, and the Peritz with Alexander–Govern tests. Tukey's procedure shows the greatest robustness in Type I error control. Any-pair power is generally best with one of the Peritz procedures. All-pairs power is best with the Pertiz F test procedure. However, Tukey's HSD preceded by the Alexander–Govern F test may provide the best combination for controlling Type I and power rates in a variety of conditions of non-normality and variance heterogeneity.     

Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models

ABSTRACT

In panel data models and other regressions with unobserved effects, fixed effects estimation is often paired with cluster-robust variance estimation (CRVE) to account for heteroscedasticity and un-modeled dependence among the errors. Although asymptotically consistent, CRVE can be biased downward when the number of clusters is small, leading to hypothesis tests with rejection rates that are too high. More accurate tests can be constructed using bias-reduced linearization (BRL), which corrects the CRVE based on a working model, in conjunction with a Satterthwaite approximation for t-tests. We propose a generalization of BRL that can be applied in models with arbitrary sets of fixed effects, where the original BRL method is undefined, and describe how to apply the method when the regression is estimated after absorbing the fixed effects. We also propose a small-sample test for multiple-parameter hypotheses, which generalizes the Satterthwaite approximation for t-tests. In simulations covering a wide range of scenarios, we find that the conventional cluster-robust Wald test can severely over-reject while the proposed small-sample test maintains Type I error close to nominal levels. The proposed methods are implemented in an R package called clubSandwich. This article has online supplementary materials.     

Effects of the generalized Box–Cox transformation on Type I error rate and power of Hotelling's T 2

Most multivariate statistical techniques rely on the assumption of multivariate normality. The effects of nonnormality on multivariate tests are assumed to be negligible when variance–covariance matrices and sample sizes are equal. Therefore, in practice, investigators usually do not attempt to assess multivariate normality. In this simulation study, the effects of skewed and leptokurtic multivariate data on the Type I error and power of Hotelling's T ² were examined by manipulating distribution, sample size, and variance–covariance matrix. The empirical Type I error rate and power of Hotelling's T ² were calculated before and after the application of generalized Box–Cox transformation. The findings demonstrated that even when variance–covariance matrices and sample sizes are equal, small to moderate changes in power still can be observed.     

On the Sampling Distributions of the Estimated Process Loss Indices with Asymmetric Tolerances

The inverse Gaussian distribution provides a flexible model for analyzing positive, right-skewed data. The generalized variable test for equality of several inverse Gaussian means with unknown and arbitrary variances has satisfactory Type-I error rate when the number of samples (k) is small (Tian, 2006). However, the Type-I error rate tends to be inflated when k goes up. In this article, we propose a parametric bootstrap (PB) approach for this problem. Simulation results show that the proposed test performs very satisfactorily regardless of the number of samples and sample sizes. This method is illustrated by an example.     

A Geometric Examination of Linear Model Assumptions

From a geometric perspective, linear model theory relies on a single assumption, that (‘corrected’) data vector directions are uniformly distributed in Euclidean space. We use this perspective to explore pictorially the effects of violations of the traditional assumptions (normality, independence and homogeneity of variance) on the Type I error rate. First, for several non‐normal distributions we draw geometric pictures and carry out simulations to show how the effects of non‐normality diminish with increased parent distribution symmetry and continuity, and increased sample size. Second, we explore the effects of dependencies on Type I error rate. Third, we use simulation and geometry to investigate the effect of heterogeneity of variance on Type I error rate. We conclude, in a fresh way, that independence and homogeneity of variance are more important assumptions than normality. The practical implication is that statisticians and authors of statistical computing packages need to pay more attention to the correctness of these assumptions than to normality.     

Akaike Information Criterion for Selecting Variables in the Nested Error Regression Model

The Akaike Information Criterion (AIC) is developed for selecting the variables of the nested error regression model where an unobservable random effect is present. Using the idea of decomposing the likelihood into two parts of “within” and “between” analysis of variance, we derive the AIC when the number of groups is large and the ratio of the variances of the random effects and the random errors is an unknown parameter. The proposed AIC is compared, using simulation, with Mallows' C _p , Akaike's AIC, and Sugiura's exact AIC. Based on the rates of selecting the true model, it is shown that the proposed AIC performs better.     

A parametric bootstrap approach for two-way error component regression models

In this article, the two-way error component regression model is considered. For the nonhomogenous linear hypothesis testing of regression coefficients, a parametric bootstrap (PB) approach is proposed. Simulation results indicate that the PB test, regardless of the sample sizes, maintains the Type I error rates very well and outperforms the existing generalized variable test, which may far exceed the intended significance level when the sample sizes are small or moderate. Real data examples illustrate the proposed approach work quite satisfactorily.     

Selecting the Optimal Transformation of a Continuous Covariate in Cox's Regression: Implications for Hypothesis Testing

ABSTRACT

Background: Many exposures in epidemiological studies have nonlinear effects and the problem is to choose an appropriate functional relationship between such exposures and the outcome. One common approach is to investigate several parametric transformations of the covariate of interest, and to select a posteriori the function that fits the data the best. However, such approach may result in an inflated Type I error. Methods: Through a simulation study, we generated data from Cox's models with different transformations of a single continuous covariate. We investigated the Type I error rate and the power of the likelihood ratio test (LRT) corresponding to three different procedures that considered the same set of parametric dose-response functions. The first unconditional approach did not involve any model selection, while the second conditional approach was based on a posteriori selection of the parametric function. The proposed third approach was similar to the second except that it used a corrected critical value for the LRT to ensure a correct Type I error. Results: The Type I error rate of the second approach was two times higher than the nominal size. For simple monotone dose-response, the corrected test had similar power as the unconditional approach, while for non monotone, dose-response, it had a higher power. A real-life application that focused on the effect of body mass index on the risk of coronary heart disease death, illustrated the advantage of the proposed approach. Conclusion: Our results confirm that a posteriori selecting the functional form of the dose-response induces a Type I error inflation. The corrected procedure, which can be applied in a wide range of situations, may provide a good trade-off between Type I error and power.     

Parametric bootstrap inferences for panel data models

This article presents parametric bootstrap (PB) approaches for hypothesis testing and interval estimation for the regression coefficients and the variance components of panel data regression models with complete panels. The PB pivot variables are proposed based on sufficient statistics of the parameters. On the other hand, we also derive generalized inferences and improved generalized inferences for variance components in this article. Some simulation results are presented to compare the performance of the PB approaches with the generalized inferences. Our studies show that the PB approaches perform satisfactorily for various sample sizes and parameter configurations, and the performance of PB approaches is mostly the same as that of generalized inferences with respect to the expected lengths and powers. The PB inferences have almost exact coverage probabilities and Type I error rates. Furthermore, the PB procedure can be simply carried out by a few simulation steps, and the derivation is easier to understand and to be extended to the incomplete panels. Finally, the proposed approaches are illustrated by using a real data example.     

Adjusted homogeneity tests of odds ratios when data are clustered

Three modified tests for homogeneity of the odds ratio for a series of 2 × 2 tables are studied when the data are clustered. In the case of clustered data, the standard tests for homogeneity of odds ratios ignore the variance inflation caused by positive correlation among responses of subjects within the same cluster, and therefore have inflated Type I error. The modified tests adjust for the variance inflation in the three existing standard tests: Breslow–Day, Tarone and the conditional score test. The degree of clustering effect is measured by the intracluster correlation coefficient, ρ. A variance correction factor derived from ρ is then applied to the variance estimator in the standard tests of homogeneity of the odds ratio. The proposed tests are an application of the variance adjustment method commonly used in correlated data analysis and are shown to maintain the nominal significance level in a simulation study. Copyright © 2004 John Wiley & Sons, Ltd.     

On group sequential tests for data in unequally sized groups and with unknown variance

We present a method to generalise the scope of application of group sequential tests designed for equally sized groups of normal observations with known variance. Preserving the significance levels against which standardised statistics are compared leads to tests for unequally grouped data which maintain Type I error probabilities to a high degree of accuracy. The same approach can be followed when observations have unknown variance by setting critical values for Studentised statistics at percentiles of the appropriate t-distributions. This significance level approach is equally applicable to group sequential one-sided tests and two-sided tests, possibly with early stopping permitted to accept the null hypothesis. In applications to equivalence testing, tests are required to maintain a specified power, rather than Type I error rate: such tests can be constructed by defining the standardised test statistics used in the significance level approach with respect to appropriately chosen hypotheses.     

A nonparametric approach to detecting the difference of survival medians

Some nonparametric methods have been proposed to compare survival medians. Most of them are based on the asymptotic null distribution to estimate the p-value. However, for small to moderate sample sizes, those tests may have inflated Type I error rate, which makes their application limited. In this article, we proposed a new nonparametric test that uses bootstrap to estimate the sample mean and variance of the median. Through comprehensive simulation, we show that the proposed approach can control Type I error rates well. A real data application is used to illustrate the use of the new test.     

Parametric bootstrap inferences for the growth curve models with intraclass correlation structure

This paper presents parametric bootstrap (PB) approaches for hypothesis testing and interval estimation of the fixed effects and the variance component in the growth curve models with intraclass correlation structure. The PB pivot variables are proposed based on the sufficient statistics of the parameters. Some simulation results are presented to compare the performance of the proposed approaches with the generalized inferences. Our studies show that the PB approaches perform satisfactorily for various cell sizes and parameter configurations, and tends to outperform the generalized inferences with respect to the coverage probabilities and powers. The PB approaches not only have almost exact coverage probabilities and Type I error rates, but also have the shorter expected lengths and the higher powers. Furthermore, the PB procedure can be simply carried out by a few simulation steps. Finally, the proposed approaches are illustrated by using a real data example.     

Proportional subclass numbers in two-factor ANOVA

The between-within split of total sum of squares in one-way analysis of variance (ANOVA) is intuitively appealing and computationally simple, whether balanced or not. In the balanced two-factor setting, the same heuristic and computations apply to analyse treatment sum of squares into main effects and interaction effects sums of squares. Accomplishing the same in unbalanced settings is more difficult, requiring development of tests of general linear hypotheses. However, textbooks treat unbalanced settings with proportional subclasss numbers (psn) as essentially equivalent to balanced settings. It is shown here that, while psn permit an ANOVA-like partition of sums of squares, test statistics for main effects of the two factors generally test the wrong hypotheses when the model includes interaction effects.