ANOVA for unbalanced data: Use Type II instead of Type III sums of squares

Methods for analyzing unbalanced factorial designs can be traced back to Yates (1934). Today, most major statistical programs perform, by default, unbalanced ANOVA based on Type III sums of squares (Yates's weighted squares of means). As criticized by Nelder and Lane (1995), this analysis is founded on unrealistic models—models with interactions, but without all corresponding main effects. The Type II analysis (Yates's method of fitting constants) is usually not preferred because of the underlying assumption of no interactions. This argument is, however, also founded on unrealistic models. Furthermore, by considering the power of the two methods, it is clear that Type II is preferable.     

Interpretation of the four types of analysis of variance tables in sas

Various computational methods exist for generating sums of squares in an analysis of variance table. When the ANOVA design is balanced, most of these computational methods will produce equivalent sums of squares for testing the significance of the ANOVA model parameters. However, when the design is unbalanced, as is frequently the case in practice, these sums of squares depend on the computational method used.- The basic reason for the difference in these sums of squares is that different hypotheses are being tested. The purpose of this paper is to describe these hypotheses in terms of population or cell means. A numerical example is given for the two factor model with interaction. The hypotheses that are tested by the four computational methods of the SAS general linear model procedure are specified.

Although the ultimate choice of hypotheses should be made by the researcher before conducting the experiment, this paper

PENDLETON,VON TRESS,AND BREMER

presents the following guidelines in selecting these hypotheses:

When the design is balanced, all of the SAS procedures will agree.

In unbalanced ANOVA designs when there are no missing cells. SAS Type III should be used. SAS Type III tests an unweighted hypothesis about cell means. SAS Types I and II test hypotheses that are functions of the ceil frequencies. These frequencies are often merely arti¬facts of the experimental process and not reflective of any underlying frequencies in the population.

When there are missing cells, i.e. no observations for some factor level combinations. Type IV should be used with caution. SAS Type IV tests hypotheses which depend     

The Two-Scale Plot: An Exploratory Display of Data with Heterogeneous Variances

The history of the analysis of unbalanced factorial designs is traced from Yates's original papers (Yates 1933, 1934) to the beginning of the computational revolution in the 1960s. Emphasis is placed on putting the methods proposed during this period in perspective in view of our present understanding.     

On the analysis of unbalanced two-level supersaturated designs via generalized linear models

Supersaturated designs (SSDs) are factorial designs in which the number of experimental runs is smaller than the number of parameters to be estimated in the model. While most of the literature on SSDs has focused on balanced designs, the construction and analysis of unbalanced designs has not been developed to a great extent. Recent studies discuss the possible advantages of relaxing the balance requirement in construction or data analysis of SSDs, and that unbalanced designs compare favorably to balanced designs for several optimality criteria and for the way in which the data are analyzed. Moreover, the effect analysis framework of unbalanced SSDs until now is restricted to the central assumption that experimental data come from a linear model. In this article, we consider unbalanced SSDs for data analysis under the assumption of generalized linear models (GLMs), revealing that unbalanced SSDs perform well despite the unbalance property. The examination of Type I and Type II error rates through an extensive simulation study indicates that the proposed method works satisfactorily.     

Analyzing Designed Experiments with Multiple Responses

This paper is an overview of a unified framework for analyzing designed experiments with univariate or multivariate responses. Both categorical and continuous design variables are considered. To handle unbalanced data, we introduce the so-called Type II* sums of squares. This means that the results are independent of the scale chosen for continuous design variables. Furthermore, it does not matter whether two-level variables are coded as categorical or continuous. Overall testing of all responses is done by 50-50 MANOVA, which handles several highly correlated responses. Univariate p-values for each response are adjusted by using rotation testing. To illustrate multivariate effects, mean values and mean predictions are illustrated in a principal component score plot or directly as curves. For the unbalanced cases, we introduce a new variant of adjusted means, which are independent to the coding of two-level variables. The methodology is exemplified by case studies from cheese and fish pudding production.     

Sweeping,alignment and the analysis of unbalanced data

The terms sweeping and alignment refer to the same process. Sweeping/alignment is used by data analysts as a technique for describing the effects of a model factor (e.g., treatments in a randomized block design) after the effects of nuisance parameters (e.g., blocks) have been removed from the data. In this paper sweeping/alignment is used as the basis for developing tests of factors in unbalanced experimental design models. Formulas are presented for treatment effects in randomized block designs with missing observations, and for interaction and main effects in unbalanced two-way factorial designs with empty cells.     

Estimation of tail parameters under type i censoring

This paper is a continuation of previous work concerning the estimation of tail-parameters under Type II censoring (Weissman 1978). The same estimation problem is considered here, this truip under Type I censoring. A sample of size n is censored below aE a given level x₀it is assumed that che underlying distriibution .function (df)belogs to the domain of attraction of a known extreme-value distribution and that K - K(x_o) , the number of observed values, remains finite as on - ∞ . We offer here estimators, which are asymptotically maximum likelihood estimators (MLE's), for quantiles associated with the tail of F such as location and scale parameters, quantiles and F(x) itself (for x in the tail). The results are applied to two illustrative examples.     

Robustness and Corrections for Sample Size Adaptation Strategies Based on Effect Size Estimation

A study on the robustness of the adaptation of the sample size for a phase III trial on the basis of existing phase II data is presented—when phase III is lower than phase II effect size. A criterion of clinical relevance for phase II results is applied in order to launch phase III, where data from phase II cannot be included in statistical analysis. The adaptation consists in adopting the conservative approach to sample size estimation, which takes into account the variability of phase II data. Some conservative sample size estimation strategies, Bayesian and frequentist, are compared with the calibrated optimal γ conservative strategy (viz. COS) which is the best performer when phase II and phase III effect sizes are equal. The Overall Power (OP) of these strategies and the mean square error (MSE) of their sample size estimators are computed under different scenarios, in the presence of the structural bias due to lower phase III effect size, for evaluating the robustness of the strategies. When the structural bias is quite small (i.e., the ratio of phase III to phase II effect size is greater than 0.8), and when some operating conditions for applying sample size estimation hold, COS can still provide acceptable results for planning phase III trials, even if in bias absence the OP was higher.

Main results concern the introduction of a correction, which affects just sample size estimates and not launch probabilities, for balancing the structural bias. In particular, the correction is based on a postulation of the structural bias; hence, it is more intuitive and easier to use than those based on the modification of Type I or/and Type II errors. A comparison of corrected conservative sample size estimation strategies is performed in the presence of a quite small bias. When the postulated correction is right, COS provides good OP and the lowest MSE. Moreover, the OPs of COS are even higher than those observed without bias, thanks to higher launch probability and a similar estimation performance. The structural bias can therefore be exploited for improving sample size estimation performances. When the postulated correction is smaller than necessary, COS is still the best performer, and it also works well. A higher than necessary correction should be avoided.     

Theory of optimal blocking of nonregular factorial designs

The authors introduce the notion of split generalized wordlength pattern (GWP), i.e., treatment GWP and block GWP, for a blocked nonregular factorial design. They generalize the minimum aberration criterion to suit this type of design. Connections between factorial design theory and coding theory allow them to obtain combinatorial identities that govern the relationship between the split GWP of a blocked factorial design and that of its blocked consulting design. These identities work for regular and nonregular designs. Furthermore, the authors establish general rules for identifying generalized minimum aberration (GMA) blocked designs through their blocked consulting designs. Finally they tabulate and compare some GMA blocked designs from Hall's orthogonal array OA(16,2¹5,2) of type III.     

Parametric Bootstrap Tests for Unbalanced Three-factor Nested Designs under Heteroscedasticity

In this article, we consider the three-factor unbalanced nested design model without the assumption of equal error variance. For the problem of testing “main effects” of the three factors, we propose a parametric bootstrap (PB) approach and compare it with the existing generalized F (GF) test. The Type I error rates of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the generalized F-test. The PB test performs very satisfactorily even for small samples while the GF test exhibits poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of the random effect variance component in a three-factor mixed effects nested model under unequal error variances.     

Applications of asymptotic confidence intervals with continuity corrections for asymmetric comparisons in noninferiority trials

A large‐sample problem of illustrating noninferiority of an experimental treatment over a referent treatment for binary outcomes is considered. The methods of illustrating noninferiority involve constructing the lower two‐sided confidence bound for the difference between binomial proportions corresponding to the experimental and referent treatments and comparing it with the negative value of the noninferiority margin. The three considered methods, Anbar, Falk–Koch, and Reduced Falk–Koch, handle the comparison in an asymmetric way, that is, only the referent proportion out of the two, experimental and referent, is directly involved in the expression for the variance of the difference between two sample proportions. Five continuity corrections (including zero) are considered with respect to each approach. The key properties of the corresponding methods are evaluated via simulations. First, the uncorrected two‐sided confidence intervals can, potentially, have smaller coverage probability than the nominal level even for moderately large sample sizes, for example, 150 per group. Next, the 15 testing methods are discussed in terms of their Type I error rate and power. In the settings with a relatively small referent proportion (about 0.4 or smaller), the Anbar approach with Yates’ continuity correction is recommended for balanced designs and the Falk–Koch method with Yates’ correction is recommended for unbalanced designs. For relatively moderate (about 0.6) and large (about 0.8 or greater) referent proportion, the uncorrected Reduced Falk–Koch method is recommended, although in this case, all methods tend to be over‐conservative. These results are expected to be used in the design stage of a noninferiority study when asymmetric comparisons are envisioned. Copyright © 2013 John Wiley & Sons, Ltd.     

Parametric bootstrap tests for unbalanced nested designs under heteroscedasticity

In this article, we consider the two-factor unbalanced nested design model without the assumption of equal error variance. For the problem of testing ‘main effects’ of both factors, we propose a parametric bootstrap (PB) approach and compare it with the existing generalized F (GF) test. The Type I error rates of the tests are evaluated using Monte Carlo simulation. Our studies show that the PB test performs better than the GF test. The PB test performs very satisfactorily even for small samples while the GF test exhibit poor Type I error properties when the number of factorial combinations or treatments goes up. It is also noted that the same tests can be used to test the significance of the random effect variance component in a two-factor mixed effects nested model under unequal error variances.     

Mixed two- and four-level fractional factorial split-plot designs with clear effects

Mixed-level designs have become widely used in the practical experiments. When the levels of some factors are difficult to be changed or controlled, fractional factorial split-plot (FFSP) designs are often used. It is highly to know when a mixed-level FFSP design with resolution III or IV has clear effects. This paper investigates the conditions of a resolution III or IV FFSP design with both two-level and four-level factors to have various clear factorial effects, including two types of main effects and three types of two-factor interaction components. The structures of such designs are shown and illustrated with examples.     

A Comparison of Estimation Methods on the Coverage Probability of Satterthwaite Confidence Intervals for Assay Precision with Unbalanced Data

Construction of closed-form confidence intervals on linear combinations of variance components were developed generically for balanced data and studied mainly for one-way and two-way random effects analysis of variance models. The Satterthwaite approach is easily generalized to unbalanced data and modified to increase its coverage probability. They are applied on measures of assay precision in combination with (restricted) maximum likelihood and Henderson III Type 1 and 3 estimation. Simulations of interlaboratory studies with unbalanced data and with small sample sizes do not show superiority of any of the possible combinations of estimation methods and Satterthwaite approaches on three measures of assay precision. However, the modified Satterthwaite approach with Henderson III Type 3 estimation is often preferred above the other combinations.     

Generalisation of Mood's Test to Factorial Designs to Detect Heterogeneity of Variances

Mood's test, which is a relatively old test (and the oldest non‐parametric test among those tests in its class) for determining heterogeneity of variance, is still being widely used in different areas such as biometry, biostatistics and medicine. Although it is a popular test, it is not suitable for use on a two‐way factorial design. In this paper, Mood's test is generalised to the 2 × 2 factorial design setting and its performance is compared with that of Klotz's test. The power and robustness of these tests are examined in detail by means of a simulation study with 10,000 replications. Based on the simulation results, the generalised Mood's and Klotz's tests can especially be recommended in settings in which the parent distribution is symmetric. As an example application we analyse data from a multi‐factor agricultural system that involves chilli peppers, nematodes and yellow nutsedge. This example dataset suggests that the performance of the generalised Mood test is in agreement with that of the generalised Klotz's test.     

An optimality criterion for supersaturated designs with quantitative factors

A supersaturated design (SSD) is a factorial design in which the degrees of freedom for all its main effects exceed the total number of distinct factorial level-combinations (runs) of the design. Designs with quantitative factors, in which level permutation within one or more factors could result in different geometrical structures, are very different from designs with nominal ones which have been treated as traditional designs. In this paper, a new criterion is proposed for SSDs with quantitative factors. Comparison and analysis for this new criterion are made. It is shown that the proposed criterion has a high efficiency in discriminating geometrically nonisomorphic designs and an advantage in computation.     

Group screening method for the statistical analysis of -optimal mixed-level supersaturated designs

In this paper, we propose the application of group screening methods for analyzing data using E(f_NOD)-optimal mixed-level supersaturated designs possessing the equal occurrence property. Supersaturated designs are a large class of factorial designs which can be used for screening out the important factors from a large set of potentially active variables. The huge advantage of these designs is that they reduce the experimental cost drastically, but their critical disadvantage is the high degree of confounding among factorial effects. Based on the idea of the group screening methods, the f factors are sub-divided into g “group-factors”. The “group-factors” are then studied using the penalized likelihood statistical analysis methods at a factorial design with orthogonal or near-orthogonal columns. All factors in groups found to have a large effect are then studied in a second stage of experiments. A comparison of the Type I and Type II error rates of various estimation methods via simulation experiments is performed. The results are presented in tables and discussion follows.     

Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

On the number of 2n-p fractional factorials of resolution III

An algorithm is specified and demonstrated which will compute the total number of ways a 2ⁿ factorial design may be partitioned into 2^p mutually exclusive 2^n-p fractional factorial designs, each having resolution III. The results of its application to all designs possessing resolution III fractions for n=5,…,20 are also given.     

Expected Mean Squares for Hierarchical Factorial Layouts with Population Imbalance

We introduce an analysis of variance usable for two-factor hierarchical models where observations are incompletely sampled from unbalanced populations of finite effects. Our new approach enables unbiased estimation of the variance components for this type of model and allows hypothesis testing to identify significant effects/sub-class effects. An explanation of how these results can be generalized to factorial layouts with more than two factors is given.