THE POWER OF STUDENT'S t TEST: CAN A NON-SIMILAR TEST DO BETTER?

Lehmann & Stein (1948) proved the existence of non-similar tests which can be more powerful than best similar tests. They used Student's problem of testing for a non-zero mean given a random sample from the normal distribution with unknown variance as an example. This raises the question: should we use a non-similar test instead of Student's t test? Questions like this can be answered by comparing the power of the test with the power envelope. This paper discusses the difficulties involved in computing power envelopes. It reports an empirical comparison of the power of the t test and the power envelope and finds that the two are almost identical especially for sample sizes greater than 20. These findings suggest that, as well as being uniformly most powerful (UMP) within the class of similar tests, Student's t test is approximately UMP within the class of all tests. For practical purposes it might also be regarded as UMP when moderate or large sample sizes are involved.     

Maximum Average-Power (MAP) Tests

The objective of this article is to propose and study frequentist tests that have maximum average power, averaging with respect to some specified weight function. First, some relationships between these tests, called maximum average-power (MAP) tests, and most powerful or uniformly most powerful tests are presented. Second, the existence of a maximum average-power test for any hypothesis testing problem is shown. Third, an MAP test for any hypothesis testing problem with a simple null hypothesis is constructed, including some interesting classical examples. Fourth, an MAP test for a hypothesis testing problem with a composite null hypothesis is discussed. From any one-parameter exponential family, a commonly used UMPU test is shown to be also an MAP test with respect to a rich class of weight functions. Finally, some remarks are given to conclude the article.     

The approximation of an unbiased test with a small level for the bioequivalence problem

In the bioequivalence problem. Brown. Hwang and Munk (1997) constructed an unbiased level a test and other tests which are uniformly more powerful than the two one-sided tests procedures when a iscomparatively larger. In this paper, for a small level, an unbiased test is shown to be approxirnately constructeQ lor tnis prooiem oy using tneir Metnog. ine numerical construction is also given.     

An exact two-sample test based on the baumgartner-weiss-schindler statistic and a modification of lepage's test

It is shown that the nonparametric two-saniDle test recently proposed by Baumgartner, WeiB, Schindler (1998, Biometrics, 54, 1129-1135) does not control the type I error rate in case of small sample sizes. We investigate the exact permutation test based on their statistic and demonstrate that this test is almost not conservative. Comparing exact tests, the procedure based on the new statistic has a less conservative size and is, according to simulation results, more powerful than the often employed Wilcoxon test. Furthermore, the new test is also powerful with regard to less restrictive settings than the location-shift model. For example, the test can detect location-scale alternatives. Therefore, we use the test to create a powerful modification of the nonparametric location-scale test according to Lepage (1971, Biometrika, 58, 213-217). Selected critical values for the proposed tests are given.     

A modified harmonic mean test procedure for variance components

A complete class of tests of variance components is characterized within the class of tests statistics of the form of a ratio of a linear combination of chi-squared random variables to an independent chi-squared random variable. This result is used in the context of general unbalanced mixed models to show that the harmonic mean method results in an inadmissible test of the random treatment effects. The harmonic mean procedure is then modified in such a way that the modified test uniformly dominates the original test. Two competitive tests are the LMP (locally most powerful) and Wald's tests, which have optimal power properties against small and large alternatives, respectively. A Monte Carlo simulation study reveals that the modified test outperforms both the LMP and Wald's tests in badly unbalanced designs and that it is a viable alternative in less unbalanced designs.     

The two-stage δ-corrected Kolmogorov-Smirnov test

The delta-corrected Kolmogorov-Smirnov test has been shown to be uniformly more powerful than the classical Kolmogorov-Smirnov test for small to moderate sample sizes. However, the delta-corrected test consists of two tests, leading to a slight inflation of the experimentwise type I error rate. The critical values of the delta-corrected test are adjusted to take into account the two-stage nature of the test, ensuring an experimentwise error rate at the nominal level. A power study confirms that the resulting so-called two-stage delta-corrected test is uniformly more powerful than the classical Kolmogorov-Smirnov test, with power improvements of up to 46 percentage points.     

Optimal record-based statistical procedures for the two-parameter exponential distribution

There are a number of situations in which the experimental data observed are record statistics. In this paper, optimal confidence intervals as well as uniformly most powerful (MP) tests for one-sided alternatives are developed. Since a uniformly MP test for a two-sided alternative does not exist, generalized likelihood ratio and uniformly unbiased and invariant tests are derived for the two parameters of the exponential distribution based on record data. For illustrative purposes, a data set on the times between consecutive telephone calls to a company's switchboard is analysed using the proposed procedures. Finally, some open problems in this direction are pointed out.     

A comparative study of the &-corrected Kolmogorov-Smirnov test

The delta-corrected Kolmogorov-Smirnov test has been shown to be uniformly more powerful than the classical Kolmogorov-Smirnov test. The power of the delta-corrected Kolmogorov-Smimov test is compared to six other goodness of fit tests based on the empirical distribution function using 10 000 Monte Carlo samples. Also, how the delta-corrected Kolmogorov-Smirnov test is conducted is illustrated.     

Partial Round-Robin Comparisons with Perfect Rankings

In teaching the development of uniformly most powerful unbiased (UMPU) tests, one rarely discusses the performance of alternative biased tests. It is shown, through the comparison of two independent Bernoulli proportions, that a biased test (the Z test) can be more powerful than the UMPU test (Fisher's exact test—randomized) in a large region of the alternative parameter space. A more general example is also given.     

A more powerful unconditional exact test of homogeneity for 2 × c contingency table analysis

The classical unconditional exact p-value test can be used to compare two multinomial distributions with small samples. This general hypothesis requires parameter estimation under the null which makes the test severely conservative. Similar property has been observed for Fisher's exact test with Barnard and Boschloo providing distinct adjustments that produce more powerful testing approaches. In this study, we develop a novel adjustment for the conservativeness of the unconditional multinomial exact p-value test that produces nominal type I error rate and increased power in comparison to all alternative approaches. We used a large simulation study to empirically estimate the 5th percentiles of the distributions of the p-values of the exact test over a range of scenarios and implemented a regression model to predict the values for two-sample multinomial settings. Our results show that the new test is uniformly more powerful than Fisher's, Barnard's, and Boschloo's tests with gains in power as large as several hundred percent in certain scenarios. Lastly, we provide a real-life data example where the unadjusted unconditional exact test wrongly fails to reject the null hypothesis and the corrected unconditional exact test rejects the null appropriately.     

Testing the equality of several means when the population variances are unequal

A comparative study is made of three tests, developed by James (1951), Welch (1951) and Brown & Forsythe (1974). James presented two methods of which only one is considered in this paper. It is shown that this method gives better control over the size than the other two tests. None of these methods is uniformly more powerful than the other two. In some cases the tests of James and Welch reject a false null hypothesis more often than the test of Brown & Forsythe, but there are also situations in which it is the other way around.

We conclude that for implementation in a statistical software package the very complicated test of James is the most attractive. A practical disadvantage of this method can be overcome by a minor modification.     

Tests for Equality of Parameters of Two Exponential Distributions with Common Known Coefficient of Variation

ABSTRACT

This article considers the problem of testing equality of parameters of two exponential distributions having common known coefficient of variation, both under unconditional and conditional setup. Unconditional tests based on BLUE'S and LRT are considered. Using the Conditionality Principle of Fisher, an UMP conditional test for one-sided alternative is derived by conditioning on an ancillary. This test is seen to be uniformly more powerful than unconditional tests in certain given ranges of ancillary. Simulation studies on the power functions of the tests are done for this purpose.     

A comparison of some conditional and unconditional exact tests for 2x2 contingency tables

In 1935, R.A. Fisher published his well-known “exact” test for 2x2 contingency tables. This test is based on the conditional distribution of a cell entry when the rows and columns marginal totals are held fixed. Tocher (1950) and Lehmann (1959) showed that Fisher s test, when supplemented by randomization, is uniformly most powerful among all the unbiased tests UMPU). However, since all the practical tests for 2x2 tables are nonrandomized - and therefore biased the UMPU test is not necessarily more powerful than other tests of the same or lower size. Inthis work, the two-sided Fisher exact test and the UMPU test are compared with six nonrandomized unconditional exact tests with respect to their power. In both the two-binomial and double dichotomy models, the UMPU test is often less powerful than some of the unconditional tests of the same (or even lower) size. Thus, the assertion that the Tocher-Lehmann modification of Fisher's conditional test is the optimal test for 2x2 tables is unjustified.     

Testing Variability in the Two-Sample Case

A number of robust methods for testing variability have been reported in previous literature. An examination of these procedures for a wide variety of populations confirms their general robustness. Shoemaker's improvement of the F test extends that test use to a realistic variety of population shapes. However, a combination of the Brown–Forsythe and O'Brien methods based on testing kurtosis is shown to be conservative for a wide range of sample sizes and population distributions. The composite test is also shown to be more powerful in most conditions than other conservative procedures.     

CONSTRUCTING MORE POWERFUL EXACT TESTS OF EQUIVALENCE FROM BINARY MATCHED PAIRS

In randomized clinical trials, it is often necessary to demonstrate that a new medical treatment does not substantially differ from a standard reference treatment. Formal testing of such ‘equivalence hypotheses’ is typically done by combining two one‐sided tests (TOST). A quite different strand of research has demonstrated that replacing nuisance parameters with a null estimate produces P‐values that are close to exact ( Lloyd 2008a ) and that maximizing over the residual dependence on the nuisance parameter produces P‐values that are exact and optimal within a class ( Röhmel & Mansmann 1999 ; Lloyd 2008a ). The three procedures – TOST, estimation and maximization of a nuisance parameter – can each be expressed as a transformation of an approximate P‐value. In this paper, we point out that TOST‐based P‐values will generally be conservative, even if based on exact and optimal one‐sided tests. This conservatism is avoided by applying the three transforms in a certain order – estimation followed by TOST followed by maximization. We compare this procedure with existing alternatives through a numerical study of binary matched pairs where the two treatments are compared by the difference of response rates. The resulting tests are uniformly more powerful than the considered competitors, although the difference in power can range from very small to moderate.     

Distance Testing for Selecting the Best Population

Consider testing the null hypothesis that a given population has location parameter greater than or equal to the largest location parameter of k competing populations. This paper generalizes tests proposed by Gupta and Bartholomew by considering tests based on p -distances from the parameter estimate to the null parameter space. It is shown that all tests are equivalent when k →∞ for a class of distributions that includes the normal and the uniform. The paper proposes the use of adaptive quantiles. Under suitable assumptions the resulting tests are asymptotically equivalent to the uniformly most powerful test for the case that the location parameters of all but one of the populations are known. The increase in power obtained by using adaptive tests is confirmed by a simulation study.     

Efficient Estimators for Regressing Regression Coefficients

The Chow test compares regressions developed from two samples from possibly different populations. Its use has traditionally been recommended only when the number of observations in one of the samples does not exceed the number of predictor variables. It is shown that when that condition is not satisfied, the test remains uniformly most powerful (UMP) among a certain class of tests against an important class of alternatives.     

Exact tests for matched pairs:studies with covariates

In prospective or retrospective studies with matched pairs one often wishes to control for covariates other than those used in the matching process.Large sample procedures assuming a logistic model are available for this problem.The present paper presents some exact permutation tests which are uniformly most powerful unbiased within a large class of tests.     

Nonparametric tests for the mean of a non-negative population

We construct level-α tests for testing the null hypothesis that the mean of a non-negative population falls below a prespecified nominal value. These tests make no assumption about the distribution function other than that it be supported on [0,∞). Simple tests are derived based on either the sample mean or the sample product. The nonparametric likelihood ratio test is also discussed in this context. We also derive the uniformly most powerful monotone (UMP) tests for special cases.     

TEST FOR TREATMENT EFFECT BASED ON BINARY DATA WITH RANDOM SAMPLE SIZES

The problem of testing for treatment effect based on binary response data is considered, assuming that the sample size for each experimental unit and treatment combination is random. It is assumed that the sample size follows a distribution that belongs to a parametric family. The uniformly most powerful unbiased tests, which are equivalent to the likelihood ratio tests, are obtained when the probability of the sample size being zero is positive. For the situation where the sample sizes are always positive, the likelihood ratio tests are derived. These test procedures, which are unconditional on the random sample sizes, are useful even when the random sample sizes are not observed. Some examples are presented as illustration.