Multiple Comparison Procedures: The Practical Solution

A practicing statistician looks at the multiple comparison controversy and related issues through the eyes of the users. The concept of consistency is introduced and discussed in relation to five of the more common multiple comparison procedures. All of the procedures are found to be inconsistent except the simplest procedure, the unrestricted least significant difference (LSD) procedure (or multiple t test). For this and other reasons the unrestricted LSD procedure is recommended for general use, with the proviso that it should be viewed as a hypothesis generator rather than as a method for simultaneous hypothesis generation and testing. The implications for Scheffé's test for general contrasts are also discussed, and a new recommendation is made.     

Editor's Report

There are two common methods for statistical inference on 2 × 2 contingency tables. One is the widely taught Pearson chi-square test, which uses the well-known χ²statistic. The chi-square test is appropriate for large sample inference, and it is equivalent to the Z-test that uses the difference between the two sample proportions for the 2 × 2 case. Another method is Fisher’s exact test, which evaluates the likelihood of each table with the same marginal totals. This article mathematically justifies that these two methods for determining extreme do not completely agree with each other. Our analysis obtains one-sided and two-sided conditions under which a disagreement in determining extreme between the two tests could occur. We also address the question whether or not their discrepancy in determining extreme would make them draw different conclusions when testing homogeneity or independence. Our examination of the two tests casts light on which test should be trusted when the two tests draw different conclusions.     

Tests for Symmetry Based on the One-Sample Wilcoxon Signed Rank Statistic

ABSTRACT

The one-sample Wilcoxon signed rank test was originally designed to test for a specified median, under the assumption that the distribution is symmetric, but it can also serve as a test for symmetry if the median is known. In this article we derive the Wilcoxon statistic as the first component of Pearson's X ² statistic for independence in a particularly constructed contingency table. The second and third components are new test statistics for symmetry. In the second part of the article, the Wilcoxon test is extended so that symmetry around the median and symmetry in the tails can be examined seperately. A trimming proportion is used to split the observations in the tails from those around the median. We further extend the method so that no arbitrary choice for the trimming proportion has to be made. Finally, the new tests are compared to other tests for symmetry in a simulation study. It is concluded that our tests often have substantially greater powers than most other tests.     

Optimum Critical Value for Pre-Test Estimator

In this paper, methods are proposed in finding the robust design in both Taguchi and Standard setups when a signal factor is present. The robust design is a set of level combinations of control factors so that the effect of controllable noise factors on response is minimum. Both univariate and multivariate methods are used in finding the influential noise factors for the determination of robust designs.     

Testing for Unit Root Against Stationarity Using the Likelihood Ratio Test

In a first-order autoregressive model with drift, we derive the likelihood ratio test for a unit root against the stationary alternative. We also derive the test in a state space model with trend. Finite sample and asymptotic critical values are obtained by Monte Carlo simulations. A simulation study investigates the power performance of the likelihood ratio test and we also examine how a bias correction of the test affects the results.     

Simulation-Based Tests that Can Use Any Number of Simulations

Conventional procedures for Monte Carlo and bootstrap tests require that B, the number of simulations, satisfy a specific relationship with the level of the test. Otherwise, a test that would instead be exact will either overreject or underreject for finite B. We present expressions for the rejection frequencies associated with existing procedures and propose a new procedure that yields exact Monte Carlo tests for any positive value of B. This procedure, which can also be used for bootstrap tests, is likely to be most useful when simulation is expensive.     

Performance of the One-Sample Goodness-of-Fit P–P-Plot Length Test

This article studies the performance of the one-sample goodness-of-fit test which is based on the length of the P–P-plot initially introduced in a similar context by Reschenhofer and Bomze (1991 Reschenhofer , E. , Bomze , I. M. ( 1991 ). Length tests for goodness-of-fit . Biometrika 78 : 207 – 216 . [Google Scholar]). The distributional properties of the length test are revised empirically via simulations. In the Monte Carlo power study that follows the length test is shown empirically to have high power under various alternatives considered relative to members of the Cramér–von Mises family of goodness-of-fit tests, and the Kolmogorov–Smirnov test.     

An Adaptive Test for the Two-Sample Location Problem Based on U-Statistics

For the two-sample location problem with continuous data we consider a general class of tests, all members of it are based on U-statistics. The asymptotic efficacies are investigated in detail. We construct an adaptive test where all statistics involved are suitably chosen U-statistics. It is shown that the proposed adaptive test has good asymptotic and finite sample power properties.     

Notes on Test Homogeneity of the Odds Ratio in Matched Pairs Under Stratified Sampling: A Monte Carlo Evaluation

Using Monte Carlo simulation, we compare the performance of five asymptotic test procedures and a randomized permutation test procedure for testing the homogeneity of odds ratio under the stratified matched-pair design. We note that the weighted-least-square test procedure is liberal, while Pearson's goodness-of-fit (PGF) test procedure with the continuity correction is conservative. We note that PGF without the continuity correction, the conditional likelihood ratio test procedure, and the randomized permutation test procedure can generally perform well with respect to Type I error. We use the data taken from a case–control study regarding the endometrial cancer incidence published elsewhere to illustrate the use of these test procedures.     

Simulation Study on Improved Shapiro–Wilk Tests for Normality

The article concerns tests for normality based on the Shapiro–Wilk W statistic. The constants in the test statistic are recalculated as those given in Shapiro and Wilk are incorrect. The empirical significance levels and power of improved tests have been evaluated in simulation study and compared to original ones. The improved tests were also applied to the multivariate case. In this case, we consider two implementations of the W statistic, the first one proposed by Srivastava and Hui and the other by Hanusz and Tarasinska. Empirical size of tests and their power have been compared to the Henze–Zirkler test.