Sample size determination in step-down and step-up multiple tests for comparing treatments with a control

We address the problem of sample size determination in multiple comparisons of k treatments with a control for step-down and step-up testing, assuming normal data and homogeneous variances. We define power as the probability of correctly rejecting all hypotheses for which the treatment vs. control difference exceeds a specified value. Our paper supplements papers by Hayter and Tamhane (J. Statist. Plann. Inference 27 (1991) 271–290) who solved the problem for one-sided comparisons using the step-down procedure and by Liu (J. Statist. Plann. Inference 62 (1997b) 255–261) who considered the two-sided case using the single-step method. We provide expressions that allow computer evaluation of the power and necessary sample sizes for one- and two-sided tests using either step-down or step-up procedures. Tables are given from which sample sizes to guarantee a specified power can be determined.     

Asymptotic comparison of the critical values of step-down and step-up multiple comparison procedures

We consider the problem of comparing step-down and step-up multiple test procedures for testing n hypotheses when independent p-values or independent test statistics are available. The defining critical values of these procedures for independent test statistics are asymptotically equal, which yields a theoretical argument for the numerical observation that the step-up procedure is mostly more powerful than the step-down procedure. The main aim of this paper is to quantify the differences between the critical values more precisely. As a by-product we also obtain more information about the gain when we consider two subsequent steps of these procedures. Moreover, we investigate how liberal the step-up procedure becomes when the step-up critical values are replaced by their step-down counterparts or by more refined approximate values. The results for independent p-values are the basis for obtaining corresponding results when independent real-valued test statistics are at hand. It turns out that the differences of step-down and step-up critical values as well as the differences between subsequent steps tend to zero for many distributions, except for heavy-tailed distributions. The Cauchy distribution yields an example where the critical values of both procedures are nearly linearly increasing in n.     

Multivariate analysis with an autoregressive covariance model

This paper addresses the problem of testing the multivariate linear hypothesis when the errors follow an antedependence model (Gabriel, 1961, 1962). Antedependence can be formulated as a nonstationary autoregressive model of general order. Three test statistics are derived that provide analogs to three commonly used MANOVA statistics: Wilks' Lambda, the Lawley-Hotelling Trace, and Pillai's Trace. Formulas are given for each of these statistics that show how they can be obtained From any statistical computing package that calculates the usual MANOVA statistics. These antedependent statistics would be appropriate in analyzing certain multivariate data sets in which repeated measurements are taken on the same subjects over a period of time.     

A note on connectedness in fixed effects manova and gmanova models withmissing cells

Murray and Smith (1985) and Hocking (1985) give a generalized definition and test of connectedness in the case of missing cells using the univariate cell-means model with linear restrictions on the cell-means. The test of connectedness is here extended to multivariate fixed effects models, including the usual MANOVA model with linear restrictions, the MANOVA model with double linear restrictions, and the GMANOVA model.     

A generalized step-up-down multiple test procedure

A generalization of step-up and step-down multiple test procedures is proposed. This step-up-down procedure is useful when the objective is to reject a specified minimum number, q, out of a family of k hypotheses. If this basic objective is met at the first step, then it proceeds in a step-down manner to see if more than q hypotheses can be rejected. Otherwise it proceeds in a step-up manner to see if some number less than q hypotheses can be rejected. The usual step-down procedure is the special case where q = 1, and the usual step-up procedure is the special case where q = k. Analytical and numerical comparisons between the powers of the step-up-down procedures with different choices of q are made to see how these powers depend on the actual number of false hypotheses. Examples of application include comparing the efficacy of a treatment to a control for multiple endpoints and testing the sensitivity of a clinical trial for comparing the efficacy of a new treatment with a set of standard treatments.     

Controlling the familywise error rate with plug-in estimator for the proportion of true null hypotheses

Summary.  Estimation of the number or proportion of true null hypotheses in multiple-testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjøtvoll. Among others, they proposed to use plug-in estimates for the proportion of true null hypotheses in multiple-test procedures to improve the power. We investigate the problem of controlling the familywise error rate FWER when such estimators are used as plug-in estimators in single-step or step-down multiple-test procedures. First we investigate the case of independent p -values under the null hypotheses and show that a suitable choice of plug-in estimates leads to control of FWER in single-step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step-down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug-in step-down procedures are not recommended here. For dependent p -values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.     

STEPWISE TESTS WHEN THE TEST STATISTICS ARE INDEPENDENT

Consider the problem of simultaneously testing a nonhierarchical finite family of hypotheses based on independent test statistics. A general stepwise test is defined, of which the well known step-down and step-up tests are special cases. The step-up test is shown to dominate the other stepwise tests, including the step-down test, for situations of practical importance. When testing against two-sided alternatives, it is pointed out that if the step-up test is augmented to include directional decisions then the augmented step-up test controls the type I and III familywise error jointly at the original level q. The definition of the adjusted p values for the step-up test is justified. The results are illustrated by a numerical example.     

A step-down heteroscedastic multiple comparison procedure

All-pairs power in a one-way ANOVA is the probability of detecting all true differences between pairs of means. Ramsey (1978) found that for normal distributions having equal variances, step-down multiple comparison procedures can have substantially more all-pairs power than single-step procedures, such as Tukey’s HSD, when equal sample sizes are randomly sampled from each group. This paper suggests a step-down procedure for the case of unequal variances and compares it to Dunnett's T3 technique. The new procedure is similar in spirit to one of the heteroscedastic procedures described by Hochberg and Tamhane (1987), but it has certain advantages that are discussed in the paper. Included are results on unequal sample sizes.     

A mixture of the manova and gmanova models

A mixture of the MANOVA and GMANOVA models is presented. The expected value of the response matrix in this model is the sum of two matrix components. The first component represents the GMANOVA portion and the second component represents the MANOVA portion. Maximum likelihood estimators are derived for the parameters in this model, and goodness-of-fit tests are constructed for fuller models via the likelihood ration criterion. Finally, likelihood ration tests for general liinear hypotheses are developed and a numerical example is presented.     

A monte carlo study on the robustness of four MANOVA criterion tests

Four MANOVA tests (Wilk's Lambda, Roy's Largest Root Test, the Hotelling-Lawley Trace and the Pillai-Bartlett Trace) were studied when restricted sample data were drawn from normal populations. Robustness was compared by examining bias at critical points and fluctuations in the standard error of the empirical distributions. The Wilk's Lambda statistic was found to be the least affected by the restricted sampling.     

Testing equality of mean vectors in a one-way MANOVA with monotone missing data

In this study, testing the equality of mean vectors in a one-way multivariate analysis of variance (MANOVA) is considered when each dataset has a monotone pattern of missing observations. The likelihood ratio test (LRT) statistic in a one-way MANOVA with monotone missing data is given. Furthermore, the modified test (MT) statistic based on likelihood ratio (LR) and the modified LRT (MLRT) statistic with monotone missing data are proposed using the decomposition of the LR and an asymptotic expansion for each decomposed LR. The accuracy of the approximation for the Chi-square distribution is investigated using a Monte Carlo simulation. Finally, an example is given to illustrate the methods.     

The Cgmanova Model

The Completely General MANOVA (CGMANOVA) model may be used to analyze many complex designs including the GMANOVA, EGMANOVA, MSUR models, the multivariate seemingly unrelated growth curve model, and numerous other designs that do not have closed form solutions using the likelihood ratio method. In this paper we review the theory of the CGMANOVA model and compare 11near model likelihood test results with the Wald statistic.     

Regularised Manova for High‐Dimensional Data

The traditional and readily available multivariate analysis of variance (MANOVA) tests such as Wilks' Lambda and the Pillai–Bartlett trace start to suffer from low power as the number of variables approaches the sample size. Moreover, when the number of variables exceeds the number of available observations, these statistics are not available for use. Ridge regularisation of the covariance matrix has been proposed to allow the use of MANOVA in high‐dimensional situations and to increase its power when the sample size approaches the number of variables. In this paper two forms of ridge regression are compared to each other and to a novel approach based on lasso regularisation, as well as to more traditional approaches based on principal components and the Moore‐Penrose generalised inverse. The performance of the different methods is explored via an extensive simulation study. All the regularised methods perform well; the best method varies across the different scenarios, with margins of victory being relatively modest. We examine a data set of soil compaction profiles at various positions relative to a ridgetop, and illustrate how our results can be used to inform the selection of a regularisation method.     

Scheffés mixed model for multivariate repeated measures:a relative efficiency evaluation

Scheffé’s mixed model, generalized for application to multivariate repeated measures, is known as the multivariate mixed model (MMM). The primary advantages the MMM are (1) the minimum sample size required to conduct an analysis is smaller than for competing procedures and (2) for certain covariance structures, the MMM analysis is more powerful than its competitors. The primary disadvantage is that the MMM makes a very restrictive covariance assumption; namely multivariate sphericity. This paper shows, first, that even minor departures from multivariate sphericity inflate the size of MMM based tests. Accordingly, MMM analyses, as computed in release 4.0 of SPSS MANOVA (SPSS Inc., 1990), can not be recommended unless it is known that multivariate sphericity is satisfied. Second, it is shown that a new Box-type (Box, 1954) Δ-corrected MMM test adequately controls test size unless departure from multivariate sphericity is severe or the covariance matrix departs substantially from a multiplicative-Kronecker structure. Third, power functions of adjusted MMM tests for selected covariance and noncentrality structures are compared to those of doubly multivariate methods that do not require multivariate sphericity. Based on relative efficiency evaluations, the adjusted MMM analyses described in this paper can be recommended only when sample sizes are very small or there is reason to believe that multivariate sphericity is nearly satisfied. Neither the e-adjusted analysis suggested in the SPSS MANOVA output (release 4.0) nor the adjusted analysis suggested by Boik (1988) can be recommended at all.     

On a gmanova model likelihood ratio test criterion

For the generalized MANOVA (GMANOVA) model of Potthoff and Roy (1964), X = BξA + E, Khatri (1966) derives the likelihood ratio test criterion for test-ing the composite double linear null hypothesis CξV = 0, C,V known. This criterion plays an important role in statistics, and several authors have recently studied its further properties. However, Khatri's (1966) de-reviation of the distribution of this criterion is involved. By noting that the GMANOVA model is re-stricted MANOVA model, this paper presents an alter-native simple derivation of the distribution of this criterion. The derivation is based on the generalized Sverdrup's lemma, Kabe (1965).     

On the maximal invariance of manova step down procedure statistics

Subbaiah and Mudhol kar (1978) remark the general mu1tivariate linear hypothesis testing step down procedure statistics do not appear to be maximal invariants under nonsingular lower triangular matrix transformations of the original variates. This paper proves the maximal invariance of these statistics. The invariance results are essential to study the power functions of the step down procedures for MANOVA problems. An example is given to show that such power function studies are very involved.     

A note on the manova model with double linear restrictions

In this note we show how one may construct goodness-of-fit tests to test hypotheses for the restricted MANOVA and GMANOVA models using the multivariate seemingly unrelated regression (MSUR) model.     

Robust nonparametrics in mixed-MANOCOVA models

In robust and nonparametric MANOVA models, the basic assumptions of independence, homoscedasticity and multinormality of the error components have been relaxed to a certain extent. In mixed-effects MANOVA models, the random effects components (due to concomitant variates) rest on the linearity of the regression function and some other distributional homogeneity conditions that may not hold universally, and avoidance of such regularity conditions generally introduce complications. Some of these difficulties are eliminated here through a conditional functional estimation approach, and in this setup, improved estimation of the fixed effects parameters is presented in a unified manner. Robustness and efficacy of these nonparametric procedures are appraised, and the picture is compared with their parametric as well as semiparametric counterparts.