A comparison of several adaptive tests for paired data

In the last few years, two adaptive tests for paired data have been proposed. One test proposed by Freidlin et al. [On the use of the Shapiro–Wilk test in two-stage adaptive inference for paired data from moderate to very heavy tailed distributions, Biom. J. 45 (2003), pp. 887–900] is a two-stage procedure that uses a selection statistic to determine which of three rank scores to use in the computation of the test statistic. Another statistic, proposed by O'Gorman [Applied Adaptive Statistical Methods: Tests of Significance and Confidence Intervals, Society for Industrial and Applied Mathematics, Philadelphia, 2004], uses a weighted t-test with the weights determined by the data. These two methods, and an earlier rank-based adaptive test proposed by Randles and Hogg [Adaptive Distribution-free Tests, Commun. Stat. 2 (1973), pp. 337–356], are compared with the t-test and to Wilcoxon's signed-rank test. For sample sizes between 15 and 50, the results show that the adaptive test proposed by Freidlin et al. and the adaptive test proposed by O'Gorman have higher power than the other tests over a range of moderate to long-tailed symmetric distributions. The results also show that the test proposed by O'Gorman has greater power than the other tests for short-tailed distributions. For sample sizes greater than 50 and for small sample sizes the adaptive test proposed by O'Gorman has the highest power for most distributions.     

Linear hypothesis testing for weighted functional data with applications

In socioeconomic areas, functional observations may be collected with weights, called weighted functional data. In this paper, we deal with a general linear hypothesis testing (GLHT) problem in the framework of functional analysis of variance with weighted functional data. With weights taken into account, we obtain unbiased and consistent estimators of the group mean and covariance functions. For the GLHT problem, we obtain a pointwise F-test statistic and build two global tests, respectively, via integrating the pointwise F-test statistic or taking its supremum over an interval of interest. The asymptotic distributions of test statistics under the null and some local alternatives are derived. Methods for approximating their null distributions are discussed. An application of the proposed methods to density function data is also presented. Intensive simulation studies and two real data examples show that the proposed tests outperform the existing competitors substantially in terms of size control and power.     

Comparison of ANOVA-F and ANOM tests with regard to type I error rate and test power

A Monte Carlo simulation was conducted to compare the type I error rate and test power of the analysis of means (ANOM) test to the one-way analysis of variance F-test (ANOVA-F). Simulation results showed that as long as the homogeneity of the variance assumption was satisfied, regardless of the shape of the distribution, number of group and the combination of observations, both ANOVA-F and ANOM test have displayed similar type I error rates. However, both tests have been negatively affected from the heterogeneity of the variances. This case became more obvious when the variance ratios increased. The test power values of both tests changed with respect to the effect size (Δ), variance ratio and sample size combinations. As long as the variances are homogeneous, ANOVA-F and ANOM test have similar powers except unbalanced cases. Under unbalanced conditions, the ANOVA-F was observed to be powerful than the ANOM-test. On the other hand, an increase in total number of observations caused the power values of ANOVA-F and ANOM test approach to each other. The relations between effect size (Δ) and the variance ratios affected the test power, especially when the sample sizes are not equal. As ANOVA-F has become to be superior in some of the experimental conditions being considered, ANOM is superior in the others. However, generally, when the populations with large mean have larger variances as well, ANOM test has been seen to be superior. On the other hand, when the populations with large mean have small variances, generally, ANOVA-F has observed to be superior. The situation became clearer when the number of the groups is 4 or 5.     

A DISTRIBUTION-FREE TEST FOR TWO-WAY LAYOUT1

Many nonparametric tests have been proposed for the hypothesis of no row (treatment) effect in a one-way layout design. Examples of such tests are Kruskal-Wallis H-test, Bhapkar's (1961) V-test and Deshpande's (1965) L-test. However not many tests are available for testing the same hypothesis in a two-way layout design without interaction. Perhaps the only “established” test is the one due to Friedman (1937). However, it applies to the case of one observation per cell only. In this paper, a new distribution-free test is proposed for the hypothesis of row effect in a two-way layout design. It applies to the case of several observations per cell, not necessarily equal. The asymptotic efficiency of the proposed test relative to other tests is studied.     

A new adaptive test for paired data for small to moderate sample sizes

When carrying out data analysis, a practitioner has to decide on a suitable test for hypothesis testing, and as such, would look for a test that has a high relative power. Tests for paired data tests are usually conducted using t-test, Wilcoxon signed-rank test or the sign test. Some adaptive tests have also been suggested in the literature by O'Gorman, who found that no single member of that family performed well for all sample sizes and different tail weights, and hence, he recommended that choice of a member of that family be made depending on both the sample size and the tail weight. In this paper, we propose a new adaptive test. Simulation studies for n=25 and n=50 show that it works well for nearly all tail weights ranging from the light-tailed beta and uniform distributions to t(4) distributions. More precisely, our test has both robustness of level (in keeping the empirical levels close to the nominal level) and efficiency of power. The results of our study contribute to the area of statistical inference.     

Analysis of binary data for the balanced one-way classification

A test is proposed for testing the equality of proportions based on the data available from a one-way classification having t treatment conditions and n binary observations per treatment. The test statistic B is a constant multiple of the F-statistic which results when the analysis of variance procedure for the one-way classification is applied to the data and, hence, is computationally simple. The statistic B from this binary analysis of variance (BIANOVA) is distributed asymptotically as a chi-square random variable. The proposed test is uniformly more powerful than either the F-test indicated above or the Pearson chi-square test; however, the attained empirical level of significance is frequently higher than for either of these competitors and usually higher than the stated level of significance for smaller values of n (say n ≤ 20).     

A new distribution-free k-sample test: Analysis of kernel density functionals

A novel distribution-free k-sample test of differences in location shifts based on the analysis of kernel density functional estimation is introduced and studied. The proposed test parallels one-way analysis of variance and the Kruskal–Wallis (KW) test aiming at testing locations of unknown distributions. In contrast to the rank (score)-transformed non-parametric approach, such as the KW test, the proposed F-test uses the measurement responses along with well-known kernel density estimation (KDE) to estimate the locations and construct the test statistic. A practical optimal bandwidth selection procedure is also provided. Our simulation studies and real data example indicate that the proposed analysis of kernel density functional estimate (ANDFE) test is superior to existing competitors for fat-tailed or heavy-tailed distributions when the k groups differ mainly in location rather than shape, especially with unbalanced data. ANDFE is also highly recommended when it is unclear whether test groups differ mainly in shape or location. The Canadian Journal of Statistics 48: 167–186; 2020 © 2019 Statistical Society of Canada     

Admissibility of Test Procedures for Linear Hypotheses in General Linear Model

This article shows that an F-test procedure is admissible for testing a linear hypothesis concerning one of the split mean vectors in a general linear model and an F-test procedure is also admissible for testing a linear hypothesis concerning another of the split mean vectors in the same model. These results are proved by showing that the critical functions of the tests are unique Bayes procedures with respect to proper prior distributions set in common for the null hypotheses and for the alternative ones, respectively.     

On the power of the t-test and some rank tests when outliers may be present

The power of some rank tests, used for testing the hypothesis of shift, is found when the underlying distributions contain outliers. The outliers are assumed to occur as the result of mixing two normal distributions with common variance. A small sample case shows how the scores for the rank tests are found and the exact power is computed for each of these rank tests. A Monte Carlo study provides an estimate of the power of the usual two sample t-test.     

Combining the t test and Wilcoxon's rank-sum test

In the two-sample location-shift problem, Student's t test or Wilcoxon's rank-sum test are commonly applied. The latter test can be more powerful for non-normal data. Here, we propose to combine the two tests within a maximum test. We show that the constructed maximum test controls the type I error rate and has good power characteristics for a variety of distributions; its power is close to that of the more powerful of the two tests. Thus, irrespective of the distribution, the maximum test stabilizes the power. To carry out the maximum test is a more powerful strategy than selecting one of the single tests. The proposed test is applied to data of a clinical trial.     

Simultaneous tests for finite families of hypotheses

A procedure is proposed whereby R test statistics F=(F₁F₂…F_r)together with "randomly generated critical points" (C₁C₂…C_r) may be used to construct a simultaneous test for

a family containing R hypotheses. This procedure provides simultaneous tests having an exact prescribed type I error rate; the procedure does not require the distribution of F to be known. The simultaneous test is illustrated for making all pairwise comparisons in a one-way ANOVA model.     

Heterogeneity of variance and biased hypothesis tests

This study examined the influence of heterogeneity of variance on Type I error rates and power of the independent-samples Student's t-test of equality of means on samples of scores from normal and 10 non-normal distributions. The same test of equality of means was performed on corresponding rank-transformed scores. For many non-normal distributions, both versions produced anomalous power functions, resulting partly from the fact that the hypothesis test was biased, so that under some conditions, the probability of rejecting H ₀ decreased as the difference between means increased. In all cases where bias occurred, the t-test on ranks exhibited substantially greater bias than the t-test on scores. This anomalous result was independent of the more familiar changes in Type I error rates and power attributable to unequal sample sizes combined with unequal variances.     

Levene type tests for the ratio of two scales

Tests for the equality of variances are of interest in many areas such as quality control, agricultural production systems, experimental education, pharmacology, biology, as well as a preliminary to the analysis of variance, dose–response modelling or discriminant analysis. The literature is vast. Traditional non-parametric tests are due to Mood, Miller and Ansari–Bradley. A test which usually stands out in terms of power and robustness against non-normality is the W50 Brown and Forsythe [Robust tests for the equality of variances, J. Am. Stat. Assoc. 69 (1974), pp. 364–367] modification of the Levene test [Robust tests for equality of variances, in Contributions to Probability and Statistics, I. Olkin, ed., Stanford University Press, Stanford, 1960, pp. 278–292]. This paper deals with the two-sample scale problem and in particular with Levene type tests. We consider 10 Levene type tests: the W50, the M50 and L50 tests [G. Pan, On a Levene type test for equality of two variances, J. Stat. Comput. Simul. 63 (1999), pp. 59–71], the R-test [R.G. O'Brien, A general ANOVA method for robust tests of additive models for variances, J. Am. Stat. Assoc. 74 (1979), pp. 877–880], as well as the bootstrap and permutation versions of the W50, L50 and R tests. We consider also the F-test, the modified Fligner and Killeen [Distribution-free two-sample tests for scale, J. Am. Stat. Assoc. 71 (1976), pp. 210–213] test, an adaptive test due to Hall and Padmanabhan [Adaptive inference for the two-sample scale problem, Technometrics 23 (1997), pp. 351–361] and the two tests due to Shoemaker [Tests for differences in dispersion based on quantiles, Am. Stat. 49(2) (1995), pp. 179–182; Interquantile tests for dispersion in skewed distributions, Commun. Stat. Simul. Comput. 28 (1999), pp. 189–205]. The aim is to identify the effective methods for detecting scale differences. Our study is different with respect to the other ones since it is focused on resampling versions of the Levene type tests, and many tests considered here have not ever been proposed and/or compared. The computationally simplest test found robust is W50. Higher power, while preserving robustness, is achieved by considering the resampling version of Levene type tests like the permutation R-test (recommended for normal- and light-tailed distributions) and the bootstrap L50 test (recommended for heavy-tailed and skewed distributions). Among non-Levene type tests, the best one is the adaptive test due to Hall and Padmanabhan.     

Testing the equality of the variances of two linear models

Testing the equality of variances of two linear models with common β-parameter is considered. A test based on least squares residuals (ASR test) is proposed, and it is shown that this test is invariant under the group of scale and translation changes. For some special cases, it is also proved that this test has a monotone power function. Finding the exact critical values of this test is not easy; an approximation is given to facilitate the computation of these. The powers of the BLUS test, the F-test and the new test are computed for various alternatives and compared in a particular case. The proposed test seems to be locally more powerful than the alternative tests.     

The performance of tests when observations have different variances

It is common to test if there is an effect due to a treatment. The commonly used tests have the assumption that the observations differ in location, and that their variances are the same over the groups. Different variances can arise if the observations being analyzed are means of different numbers of observations on individuals or slopes of growth curves with missing data. This study is concerned with cases in which the unequal variances are known, or known to a constant of proportionality. It examines the performance of the ttest, the Mann–Whitney–Wilcoxon Rank Sum test, the Median test, and the Van der Waerden test under these conditions. The t-test based on the weighted means is the likelihood ratio test under normality and has the usual optimality properties. The other tests are compared to it. One may align and scale the observations by subtracting the mean and dividing by the standard deviation of each point. This leads to other, analogous test statistics based on these adjusted observations. These statistics are also compared. Finally, the regression scores tests are compared to the other procedures.     

A simulation study on the influence of ties on uniform scores test for circular data

Uniform scores test is a rank-based method that tests the homogeneity of k-populations in circular data problems. The influence of ties on the uniform scores test has been emphasized by several authors in several articles and books. Moreover, it is suggested that the uniform scores test should be used with caution if ties are present in the data. This paper investigates the influence of ties on the uniform scores test by computing the power of the test using average, randomization, permutation, minimum, and maximum methods to break ties. Monte Carlo simulation is performed to compute the power of the test under several scenarios such as having 5% or 10% of ties and tie group structures in the data. The simulation study shows no significant difference among the methods under the existence of ties but the test loses its power when there are many ties or complicated group structures. Thus, randomization or average methods are equally powerful to break ties when applying uniform scores test. Also, it can be concluded that k-sample uniform scores test can be used safely without sacrificing the power if there are only less than 5% of ties or at most two groups of a few ties.     

Two separate effects of variance heterogeneity on the validity and power of significance tests of location

Heterogeneity of variances of treatment groups influences the validity and power of significance tests of location in two distinct ways. First, if sample sizes are unequal, the Type I error rate and power are depressed if a larger variance is associated with a larger sample size, and elevated if a larger variance is associated with a smaller sample size. This well-established effect, which occurs in t and F tests, and to a lesser degree in nonparametric rank tests, results from unequal contributions of pooled estimates of error variance in the computation of test statistics. It is observed in samples from normal distributions, as well as non-normal distributions of various shapes. Second, transformation of scores from skewed distributions with unequal variances to ranks produces differences in the means of the ranks assigned to the respective groups, even if the means of the initial groups are equal, and a subsequent inflation of Type I error rates and power. This effect occurs for all sample sizes, equal and unequal. For the t test, the discrepancy diminishes, and for the Wilcoxon–Mann–Whitney test, it becomes larger, as sample size increases. The Welch separate-variance t test overcomes the first effect but not the second. Because of interaction of these separate effects, the validity and power of both parametric and nonparametric tests performed on samples of any size from unknown distributions with possibly unequal variances can be distorted in unpredictable ways.     

A comparative study of tests for paired lifetime data

In this paper, we investigate different procedures for testing the equality of two mean survival times in paired lifetime studies. We consider Owen’s M-test and Q-test, a likelihood ratio test, the paired t-test, the Wilcoxon signed rank test and a permutation test based on log-transformed survival times in the comparative study. We also consider the paired t-test, the Wilcoxon signed rank test and a permutation test based on original survival times for the sake of comparison. The size and power characteristics of these tests are studied by means of Monte Carlo simulations under a frailty Weibull model. For less skewed marginal distributions, the Wilcoxon signed rank test based on original survival times is found to be desirable. Otherwise, the M-test and the likelihood ratio test are the best choices in terms of power. In general, one can choose a test procedure based on information about the correlation between the two survival times and the skewness of the marginal survival distributions.     

An adaptive two-sample location-scale test of lepage type for symmetric distributions

For the two-sample location and scale problem we propose an adaptive test which is based on so called Lepage type tests. The well known test of Lepage (1971) is a combination of the Wilcoxon test for location alternatives and the Ansari-Bradley test for scale alternatives and it behaves well for symmetric and medium-tailed distributions. For the cae of short-, medium- and long-tailed distributions we replace the Wilcoxon test and the .Ansari-Bradley test by suitable other two-sample tests for location and scale, respectively, in oder to get higher power than the classical Lepage test for such distribotions. These tests here are called Lepage type tests. in practice, however, we generally have no clear idea about the distribution having generated our data. Thus, an adaptive test should be applied which takes the the given data set inio consideration. The proposed adaptive test is based on the concept of Hogg (1974), i.e., first, to classify the unknown symmetric distribution function with respect to a measure for tailweight and second, to apply an appropriate Lepage type test for this classified type of distribution. We compare the adaptive test with the three Lepage type tests in the adaptive scheme and with the classical Lepage test as well as with other parametric and nonparametric tests. The power comparison is carried out via Monte Carlo simulation. It is shown that the adaptive test is the best one for the broad class of distributions considered.     

One-way ANOVA with Unequal Variances

We consider the one-way ANOVA problem of testing the equality of several normal means when the variances are not assumed to be equal. This is a generalization of the Behrens-Fisher problem, but even in this special case there is no exact test and the actual size of any test depends on the values of the nuisance parameters. Therefore, controlling the actual size of the test is of main concern. In this article, we first consider a test using the concept of generalized p-value. Extensive simulation studies show that the actual size of this test does not exceed the nominal level, for practically all values of the nuisance parameters, but the test is not too conservative either, in the sense that the actual size of the test can be very close to the nominal level for some values of the nuisance parameters. We then use this test to propose a simple F-test, which has similar properties but avoids the computations associated with generalized p-values. Because of its simplicity, both conceptually as well as computationally, this F-test may be more useful in practice, since one-way ANOVA is widely used by practitioners who may not be familiar with the generalized p-value and its computational aspects.