A step-down heteroscedastic multiple comparison procedure

All-pairs power in a one-way ANOVA is the probability of detecting all true differences between pairs of means. Ramsey (1978) found that for normal distributions having equal variances, step-down multiple comparison procedures can have substantially more all-pairs power than single-step procedures, such as Tukey’s HSD, when equal sample sizes are randomly sampled from each group. This paper suggests a step-down procedure for the case of unequal variances and compares it to Dunnett's T3 technique. The new procedure is similar in spirit to one of the heteroscedastic procedures described by Hochberg and Tamhane (1987), but it has certain advantages that are discussed in the paper. Included are results on unequal sample sizes.     

Sample size determination in step-down and step-up multiple tests for comparing treatments with a control

We address the problem of sample size determination in multiple comparisons of k treatments with a control for step-down and step-up testing, assuming normal data and homogeneous variances. We define power as the probability of correctly rejecting all hypotheses for which the treatment vs. control difference exceeds a specified value. Our paper supplements papers by Hayter and Tamhane (J. Statist. Plann. Inference 27 (1991) 271–290) who solved the problem for one-sided comparisons using the step-down procedure and by Liu (J. Statist. Plann. Inference 62 (1997b) 255–261) who considered the two-sided case using the single-step method. We provide expressions that allow computer evaluation of the power and necessary sample sizes for one- and two-sided tests using either step-down or step-up procedures. Tables are given from which sample sizes to guarantee a specified power can be determined.     

Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures

Many multiple testing procedures (MTPs) are available today, and their number is growing. Also available are many type I error rates: the family-wise error rate (FWER), the false discovery rate, the proportion of false positives, and others. Most MTPs are designed to control a specific type I error rate, and it is hard to compare different procedures. We approach the problem by studying the exact level at which threshold step-down (TSD) procedures (an important class of MTPs exemplified by the classic Holm procedure) control the generalized FWER   defined as the probability of k$k$ or more false rejections. We find that level explicitly for any TSD procedure and any k$k$. No assumptions are made about the dependency structure of the p$p$-values of the individual tests. We derive from our formula a criterion for unimprovability   of a procedure in the class of TSD procedures controlling the generalized FWER at a given level. In turn, this criterion implies that for each k$k$ the number of such unimprovable procedures is finite and is greater than one if k>1$k>1$. Consequently, in this case the most rejective procedure in the above class does not exist.     

New critical values for the generalized t and generalized rank-sum procedures

Student's t test as well as Wilcoxon's rank-sum test may be inefficient in situations where treatments bring about changes in both location and scale. In order to rectify this situation, O'Brien (1988, Journal of the American Statistical Association 83, 52-61) has proposed two new statistics, the generalized t and generalized rank-sum procedures, which may be much more powerful than their traditional counterparts in such situations. Recently, however, Blair and Morel (1991, Statistics in Medicine in press) have shown that referencing these new statistics to standard F tables as recommended by O'Brien results in inflations of Type I errors. This paper provides tables of critical values which do not produce such inflations. Use of these new critical values results in Type I error rates near nominal levels for the generalized t statistic and slightly conservative rates for the generalized rank-sum test. In addition to the critical values, some new power results are given for the generalized tests.     

Nonparametric multiple comparison procedures for unbalanced two-way layouts

In this paper, we consider nonparametric multiple comparison procedures for unbalanced two-way factorial designs under a pure nonparametric framework. For multiple comparisons of treatments versus a control concerning the main effects or the simple factor effects, the limiting distribution of the associated rank statistics is proven to satisfy the multivariate totally positive of order two condition. Hence, asymptotically the proposed Hochberg procedure strongly controls the familywise type I error rate for the simultaneous testing of the individual hypotheses. In addition, we propose to employ Shaffer's modified version of Holm's stepdown procedure to perform simultaneous tests on all pairwise comparisons regarding the main or simple factor effects and to perform simultaneous tests on all interaction effects. The logical constraints in the corresponding hypothesis families are utilized to sharpen the rejective thresholds and improve the power of the tests.     

Two stage multiple comparison procedures based on the studentized range

Stein’s (1945) two sample approach and Tukey’s T-Method of multiple comparisons (see e.g. Miller, 1966, Ch. 2) are combined to obtain fixed width simultaneous confidence intervals and simultaneous test procedures of predetermined Type I and Type II error levels, for all contrasts, in a one way layout. The necessary constants for implementing the two stage procedure are obtained under a least favorable configuration of the parameters. This provides the required protection of the null and alternative hypotheses under any configuration of parameters. A table is provided for some selected designs and error levels and an example is given to illustrate certain features of the new procedure.     

Nonparametric multiple comparison procedures for unbalanced one-way factorial designs

In this paper, we present several nonparametric multiple comparison (MC) procedures for unbalanced one-way factorial designs. The nonparametric hypotheses are formulated by using normalized distribution functions and the comparisons are carried out on the basis of the relative treatment effects. The proposed test statistics take the form of linear pseudo rank statistics and the asymptotic joint distribution of the pseudo rank statistics for testing treatments versus control satisfies the multivariate totally positive of order two condition irrespective of the correlations among the rank statistics. Therefore, in the context of MCs of treatments versus control, the nonparametric Simes test is validated for the global testing of the intersection hypothesis. For simultaneous testing of individual hypotheses, the nonparametric Hochberg stepup procedure strongly controls the familywise type I error rate asymptotically. With regard to all pairwise comparisons, we generalize various single-step and stagewise procedures to perform comparisons on the relative treatment effects. To further compare with normal theory counterparts, the asymptotic relative efficiencies of the nonparametric MC procedures with respect to the parametric MC procedures are derived under a sequence of Pitman alternatives in a nonparametric location shift model for unbalanced one-way layouts. Monte Carlo simulations are conducted to demonstrate the validity and power of the proposed nonparametric MC procedures.     

Some multiple comparison procedures for comparing treatments within several groups

Multiple comparison procedures are extended to designs consisting of several groups, where the treatment means are to be compared within each group. This may arise in two-factor experiments, with a significant interaction term, when one is interested in comparing the levels of one factor at each level of the other factor. A general approach is presented for deriving the distributions and calculating critical points, following three papers which dealt with two specific procedures. These points are used for constructing simultaneous confidence intervals over some restricted set of contrasts among treatment means in each of the groups. Tables of critical values are provided for two procedures and an application is demonstrated. Some extensions are presented for the case of possible different sets of contrasts and also for unequal variances in the various groups.     

Step-up and step-down methods for testing multiple hypotheses in sequential experiments

Sequential methods are developed for testing multiple hypotheses, resulting in a statistical decision for each individual test and controlling the familywise error rate and the familywise power in the strong sense. Extending the ideas of step-up and step-down methods for multiple comparisons to sequential designs, the new techniques improve over the Bonferroni and closed testing methods proposed earlier by a substantial reduction of the expected sample size.     

An evaluation of some nonparametric multiple comparison procedures by Monte Carlo methods

Computer simulation techniques were employed to investigate the Type I and Type II error rates (experiment-wise and comparison-wise) of three nonparametric multiple comparison procedures. Three different underlying distributions were considered. It was found that the nonparametric analog to Fisher’s LSD (a Kruskal-Wallis test, followed by pairwise Mann-Whitney U tests if a significant overall effect is detected) appeared to be superior to the Nemenyi-Dunn and Steel-Dwass procedures, because of the extreme conservatism of these latter methods.     

New one-stage multiple comparison procedures with the average for exponential location parameters under heteroscedasticity

ABSTRACT

In this paper, we present new one-stage multiple comparison procedures with the average for location parameters of two-parameter exponential distributions under heteroscedasticity by modifying the existing one proposed by Wu [One stage multiple comparisons with the average for exponential location parameters under heteroscedasticity. Comput Stat Data Anal. 2013;68:352–360] with unequal sample sizes. A simulation study is done and the results show that the proposed procedures have shorter confidence length with coverage probabilities closer to the nominal ones. At last, an example of comparing the survival days of patients for four categories of lung cancer is given to demonstrate the proposed procedures.     

Improved algorithms for closed step-down procedures to test heterogeneity of all subsets of means

A common problem in analysis of variance is testing for heterogeneity of different subsets of the full set of k population means. A step-down procedure tests a given subset of p means only after rejecting homogeneity for all sets that contain it. The Peritz and Gabriel closed procedure rejects homogeneity for the subset if every partition of the k means that includes the subset includes some rejected set. The Begun and Gabriel closure algorithm reduces computations, but the number of tests still increases exponentially with respect to the number of complementary means, m=k−p. We propose a new algorithm that tests only the m−1 pairs of adjacent ordered complementary sample means. Our algorithm may be used with analyses of variance test statistics in balanced and unbalanced designs, and with Studentized ranges except in extremely unbalanced designs. Seaman, Levin, and Serlin proposed a more powerful closure criterion that cannot exploit the Begun and Gabriel algorithm. We propose a new algorithm in this case as well.     

Asymptotic calculations of functions of expected values and covariances of order statistics

Suppose m and V are respectively the vector of expected values and the covariance matrix of the order statistics of a sample of size n from a continuous distribution F. A method is presented to calculate asymptotic values of functions of m and V ^–1, for distributions F which are sufficiently regular. Values are given for the normal, logistic, and extreme-value distributions; also, for completeness, for the uniform and exponential distributions, although for these other methods must be used.     

Tables of critical values of the normal probability paper test

The present paper relates to Holm's normal probability paper test for simple hypotheses. It is shown how to obtain the statistics for the one-sided and two-sided tests using normal probability paper. Tables of critical values of these test statistics are given for practical use of the test; the equations for calculating the distribution functions of the test statistics are listed. Two different methods lead to these equations; but details are beyond the scope of the present paper and the interested reader is referred to another paper.     

Asymptotic distributions of order statistics and record values arising from the class of beta-generated distributions

In this paper, we show that both the class of beta-generated distributions G_F and its base distribution F belong to the same domain of maximal (or minimal or upper record value or lower record value) attraction. Moreover, it is shown that the weak convergence of any non-extreme order statistic (central or intermediate order statistic), based on a base distribution F, to a non-degenerate limit type implies the weak convergence of G_F to a non-degenerate limit type. The relations between the two limit types are deduced.     

Tables of critical values for two group interim analyses

Tables of critical values are given, which can be used to execute interim analyses in clinical trials involving two groups when the joint distribution of the test statistics can be approximated by a multivariate normal distribution. Critical values are given for both the one and two interim analyses cases for a variety of partitions of α and correlation structures. Results of power calculations are presented, which reflect the effects of both the correlation structure and partitions of α.Several examples are given, which illustrate how to apply the tables to a variety of experiments