THE POWER OF STUDENT'S t TEST: CAN A NON-SIMILAR TEST DO BETTER?

Lehmann & Stein (1948) proved the existence of non-similar tests which can be more powerful than best similar tests. They used Student's problem of testing for a non-zero mean given a random sample from the normal distribution with unknown variance as an example. This raises the question: should we use a non-similar test instead of Student's t test? Questions like this can be answered by comparing the power of the test with the power envelope. This paper discusses the difficulties involved in computing power envelopes. It reports an empirical comparison of the power of the t test and the power envelope and finds that the two are almost identical especially for sample sizes greater than 20. These findings suggest that, as well as being uniformly most powerful (UMP) within the class of similar tests, Student's t test is approximately UMP within the class of all tests. For practical purposes it might also be regarded as UMP when moderate or large sample sizes are involved.     

On the procedure of hodges and lehmann for testing composite hypotheses of the mean of a normal distribution with unknown variance

In 1954 Hodges and Lehmann gave a test procedure for testing the hypothesis that the mean of an identically independently normally distributed random sample with unknown variance is contained within a certain interval [μ1, μ2]. The test is similar on the boundary of the zero-hypothesis and superior in power to the composite t-test usually applied to this problem. However Hodges and Lehmann could prove the unbiasedness of their test only for the special case that the sample consists of two elements. From numerical computations they guessed that unbiasedness would be valid for arbitrary sample sizes. This question is discussed here and partially answered.     

A robust I-sample analysis of means type randomization test for variances for unbalanced designs

Two analysis of means type randomization tests for testing the equality of I variances for unbalanced designs are presented. Randomization techniques for testing statistical hypotheses can be used when parametric tests are inappropriate. Suppose that I independent samples have been collected. Randomization tests are based on shuffles or rearrangements of the (combined) sample. Putting each of the I samples ‘in a bowl’ forms the combined sample. Drawing samples ‘from the bowl’ forms a shuffle. Shuffles can be made with replacement (bootstrap shuffling) or without replacement (permutation shuffling). The tests that are presented offer two advantages. They are robust to non-normality and they allow the user to graphically present the results via a decision chart similar to a Shewhart control chart. A Monte Carlo study is used to verify that the permutation version of the tests exhibit excellent power when compared to other robust tests. The Monte Carlo study also identifies circumstances under which the popular Levene's test fails.     

Union-Intersection testing for outliers in multivariate normal data

The union-intersection approach to multivariate test construction is used to develop an alternative to Wilks' likelihood ratio test statistic for testing for two or more outliers in multivariate normal data. It is shown that critical values of both statistics are poorly approximated by Bonferroni bounds. Simulated critical values are presented for both statistics for significance levels 1% and 5%, for sample sizes 10(5)30, 40, 50, 75 and 100 for 2, 3, 4 and 5 dimensions. A power comparison of the two tests in the slippage of the mean model for generating outliers indicates that the union-intersection test is the more powerful when the slippages are close to collinear. Although Wilks' test remains the preference for general use, the union-intersection test could be valuable when such special structure in the data is suspected.     

A Robust Test for Non-nested Hypotheses

We propose a robust version of Cox-type test statistics for the choice between two non-nested hypotheses. We first show that the influence of small amounts of contamination in the data on the test decision can be very large. Secondly, we build a robust test statistic by using the results on robust parametric tests that are available in the literature and show that the level of the robust test is stable. Finally, we show numerically not only the robustness of this new test statistic but also that its asymptotic distribution is a good approximation of its sample distribution, unlike for the classical test statistic. We apply our results to the choice between a Pareto and an exponential distribution as well as between two competing regressors in the simple linear regression model without intercept.     

Hausman-type tests for individual and time effects in the panel regression model with incomplete data

By comparing estimators of the variance of idiosyncratic error at different robust levels, two Hausman-type test statistics are respectively constructed for the existence of individual and time effects in the panel regression model with incomplete data. The resultant test statistics have several desired properties. Firstly, they are robust to the presence of one effect when the other is tested. Secondly, they are immune to the non-normal distribution of the disturbances since the distributional conditions are not needed in the construction of the statistics. Thirdly, they have more robust performances than the main competitors in the literature when the covariates are correlated with the effects. Additionally, they are very simple and have no heavy computational burden. Joint tests for both of the two effects are also discussed. Monte Carlo evidence shows that the proposed tests have desired finite sample properties, and a real data analysis gives further support.     

Robust tests for equality of two population means under the normal model

A robust procedure is developed for testing the equality of means in the two sample normal model. This is based on the weighted likelihood estimators of Basu et al. (1993). When the normal model is true the tests proposed have the same asymptotic power as the two sample Student's t-statistic in the equal variance case. However, when the normality assumptions are only approximately true the proposed tests can be substantially more powerful than the classical tests. In a Monte Carlo study for the equal variance case under various outlier models the proposed test using Hellinger distance based weighted likelihood estimator compared favorably with the classical test as well as the robust test proposed by Tiku (1980).     

Generalized spectral tests for serial dependence

Two tests for serial dependence are proposed using a generalized spectral theory in combination with the empirical distribution function. The tests are generalizations of the Cramér-von Mises and Kolmogorov-Smirnov tests based on the standardized spectral distribution function. They do not involve the choice of a lag order, and they are consistent against all types of pairwise serial dependence, including those with zero autocorrelation. They also require no moment condition and are distribution free under serial independence. A simulation study compares the finite sample performances of the new tests and some closely related tests. The asymptotic distribution theory works well in finite samples. The generalized Cramér-von Mises test has good power against a variety of dependent alternatives and dominates the generalized Kolmogorov-Smirnov test. A local power analysis explains some important stylized facts on the power of the tests based on the empirical distribution function.     

A TEST FOR THE NONPARAMETRIC TWO SAMPLE SCALE PROBLEM

We propose a distribution-free test for the nonparametric two sample scale problem. Unlike the other tests for this problem, we do not assume that the two distribution functions have a common median. We assume that they have a common quantile of order a (not necessarily 1/2). The test statistic is a modification of the Sukhatme statistic for the scale problem and the Wilcoxon-Mann-Whitney statistic for stochastic dominance. It is shown that the new test is uniformly more efficient (in the Pitman sense) than the Sukhatme test and has very good efficiency when compared to the Mood test.     

Comparing the Survival of Two Groups with an Intermediate Clinical Event

Consider a subject entered on a clinicaltrial in which the major endpoint is a time metric such as deathor time to reach a well defined event. During the observationalperiod the subject may experience an intermediate clinical event.The intermediate clinical event may induce a change in the survivaldistribution. We consider models for the one and two sample problem.The model for the one sample problem enables one to test if theoccurrence of the intermediate event changed the survival distribution.This models provides a way of carrying out non-randomized clinicaltrial to determine if a therapy has benefit. The two sample problemconsiders testing if the probability distributions, with andwithout an intermediate event, are the same. Statistical testsare derived using a semi-Markov or a time dependent mixture model.Simulation studies are carried out to compare these new procedureswith the log rank, stratified log rank and landmark tests. Thenew tests appear to have uniformly greater power than these competitortests. The methods are applied to a randomized clinical trialcarried out by the Aids Clinical Trial Group (ACTG) which comparedlow versus high doses of zidovudine (AZT).     

Exact distribution-free tests for equality of several linear models

Exact tests for the equality of several linear models are developed using permutation techniques. Two cases of the linear model, characterized by either stochastic or nonstochastic predictors, are considered: the linear regression model (LRM) and the general linear model (GLM). A general class of test statistics using the volume of simplexes as the basic unit of analysis is proposed for this problem. The resulting class of statistics is shown to be a natural generalization of the multi-response permutation procedure (MRPP) test statistics which have been shown to comprise many of the statistics used in both parametric and nonparametric analysis of the standard g—sample problem. In the LRM case, exact moments of all orders are derived for the permutation distribution of any test statistic in the general class. Moment-based approximation of significance levels is shown to be computationally feasible in the simple LRM.     

A nonparametric test of conditional autoregressive heteroscedasticity for threshold autoregressive models

Threshold autoregressive models are widely used in time‐series applications. When building or using such a model, it is important to know whether conditional heteroscedasticity exists. The authors propose a nonparametric test of this hypothesis. They develop the large‐sample theory of a test of nonlinear conditional heteroscedasticity adapted to nonlinear autoregressive models and study its finite‐sample properties through simulations. They also provide percentage points for carrying out this test, which is found to have very good power overall.     

Robust I-sample analysis of means type randomization tests for variances

Four Analysis of Means (ANOM) type randomization tests for testing the equality of I variances are presented. Randomization techniques for testing statistical hypotheses can be used when parametric tests are inappropriate. Suppose that I independent samples have been collected. Randomization tests are based on shuffles or rearrangements of the (combined) sample. Putting each of the I samples "in a bowl" forms the combined sample. Drawing samples "from the bowl" forms a shuffle. Shuffles can be made with replacement (bootstrap shuffling) or without replacement (permutation shuffling). The tests that are presented offer two advantages. They are robust to non-normality and they allow the user to graphically present the results via a decision chart similar to a Shewhart control chart. The decision chart facilitates easy assessment of both statistical and practical significance. A Monte Carlo study is used to identify robust randomization tests that exhibit excellent power when compared to other robust tests.     

The Asymptotic Power Of Jonckheere-Type Tests For Ordered Alternatives

For the c -sample location problem with ordered alternatives, the test proposed by Barlow et al . (1972 p. 184) is an appropriate one under the model of normality. For non-normal data, however, there are rank tests which have higher power than the test of Barlow et al ., e.g. the Jonckheere test or so-called Jonckheere-type tests recently introduced and studied by Büning & Kössler (1996). In this paper the asymptotic power of the Jonckheere-type tests is computed by using results of Hájek (1968) which may be considered as extensions of the theorem of Chernoff & Savage (1958). Power studies via Monte Carlo simulation show that the asymptotic power values provide a good approximation to the finite ones even for moderate sample sizes.     

On lachin'S formulae for sample sizes of survival tests

Lachin [1981] and Lachin and Foulkes [1986] consider two groups of identically independently exponentially distributed random variables and four models of data sampling. The test problem they treat is to decide whether the two distributions are identical (null-hypothesis H⁰) or not (alternative hypothesis H¹). Basing the test on maximum-likelihood estimators and their asymptotic normal densities they obtain formulae for the group sizes necessary to yield asymptotic tests with guaranteed power under a prescribed level for specified hypotheses. It is intuitively reasonable to expect the sizes decrease the more the hypotheses differ. It the distance betwen H⁰ and H¹ is measured by the difference of the exponential parameters this assumption time or the deviation of the exponential parameter ratio from unity is the measure larger distances between the hypotheses do not necessarily lead to smaller sample sizes.     

A two sample ordered alternative test for means and variances

A two sample test of likelihood ratio type is proposed, assuming normal distribution theory, for testing the hypothesis that two samples come from identical normal populations versus the alternative that the populations are normal but vary in mean value and variance with one population having a smaller mean and smaller variance than the other. The small sample and large sample distribution of the proposed statistic are derived assuming normality. Some computations are presented which show the speed of convergence of small sample critical values to their asymptotic counterparts. Comparisons of local power of the proposed test are made with several potential competing tests. Asymptotics for the test statistic are derived when underlying distributions are not necessarily normal.     

Testing hypotheses about medical test accuracy: considerations for design and inference

Developing new medical tests and identifying single biomarkers or panels of biomarkers with superior accuracy over existing classifiers promotes lifelong health of individuals and populations. Before a medical test can be routinely used in clinical practice, its accuracy within diseased and non-diseased populations must be rigorously evaluated. We introduce a method for sample size determination for studies designed to test hypotheses about medical test or biomarker sensitivity and specificity. We show how a sample size can be determined to guard against making type I and/or type II errors by calculating Bayes factors from multiple data sets simulated under null and/or alternative models. The approach can be implemented across a variety of study designs, including investigations into one test or two conditionally independent or dependent tests. We focus on a general setting that involves non-identifiable models for data when true disease status is unavailable due to the nonexistence of or undesirable side effects from a perfectly accurate (i.e. ‘gold standard’) test; special cases of the general method apply to identifiable models with or without gold-standard data. Calculation of Bayes factors is performed by incorporating prior information for model parameters (e.g. sensitivity, specificity, and disease prevalence) and augmenting the observed test-outcome data with unobserved latent data on disease status to facilitate Gibbs sampling from posterior distributions. We illustrate our methods using a thorough simulation study and an application to toxoplasmosis.     

Significance levels and confidence intervals for permutation tests

A computational algorithm is given which calculates exact significance levels of a wide class of permutation tests in the one and two sample problems. This class includes the permutation test based on the means, locally most powerful permutation tests and linear rank tests. When a shift model is assumed confidence intervals can also be obtained. Approximate methods, based on asymptotic expansions, are also presented.     

Likelihood ratio tests based on subglobal optimization: A power comparison in exponential mixture models

The paper compares several versions of the likelihood ratio test for exponential homogeneity against mixtures of two exponentials. They are based on different implementations of the likelihood maximization algorithm. We show that global maximization of the likelihood is not appropriate to obtain a good power of the LR test. A simple starting strategy for the EM algorithm, which under the null hypothesis often fails to find the global maximum, results in a rather powerful test. On the other hand, a multiple starting strategy that comes close to global maximization under both the null and the alternative hypotheses leads to inferior power.     

On the Ghoudi,Khoudraji, and Rivest test for extreme‐value dependence

Ghoudi, Khoudraji & Rivest [The Canadian Journal of Statistics 1998;26:187–197] showed how to test whether the dependence structure of a pair of continuous random variables is characterized by an extreme‐value copula. The test is based on a U‐statistic whose finite‐ and large‐sample variance are determined by the present authors. They propose estimates of this variance which they compare to the jackknife estimate of Ghoudi, Khoudraji & Rivest ( 1998 ) through simulations. They study the finite‐sample and asymptotic power of the test under various alternatives. They illustrate their approach using financial and geological data. The Canadian Journal of Statistics © 2009 Statistical Society of Canada