A Discussion of Permutation Tests Conditional to Observed Responses in Unreplicated 2 M Full Factorial Designs

ABSTRACT

In this article we present a new solution to test for effects in unreplicated two-level factorial designs. The proposed test statistic, in case the error components are normally distributed, follows an F random variable, though our attention is on its nonparametric permutation version. The proposed procedure does not require any transformation of data such as residualization and it is exact for each effect and distribution-free. Our main aim is to discuss a permutation solution conditional to the original vector of responses. We give two versions of the same nonparametric testing procedure in order to control both the individual error rate and the experiment-wise error rate. A power comparison with Loughin and Noble's test is provided in the case of a unreplicated 2⁴ full factorial design.     

2<Superscript><Emphasis Type="Italic">m</Emphasis></Superscript>4<Superscript>1</Superscript> designs with minimum aberration or weak minimum aberration

For measuring the goodness of 2 ^m 4¹ designs, Wu and Zhang (1993) proposed the minimum aberration (MA) criterion. MA 2 ^m 4¹ designs have been constructed using the idea of complementary designs when the number of two-level factors, m, exceeds n/2, where n is the total number of runs. In this paper, the structures of MA 2 ^m 4¹ designs are obtained when m>5n/16. Based on these structures, some methods are developed for constructing MA 2 ^m 4¹ designs for 5n/16<m<n/2 as well as for n/2≤m<n. When m≤5n/16, there is no general method for constructing MA 2 ^m 4¹ designs. In this case, we obtain lower bounds for A ₃₀ and A ₃₁, where A ₃₀ and A ₃₁ are the numbers of type 0 and type 1 words with length three respectively. And a method for constructing weak minimum aberration (WMA) 2 ^m 4¹ designs (A ₃₀ and A ₃₁ achieving the lower bounds) is demonstrated. Some MA or WMA 2 ^m 4¹ designs with 32 or 64 runs are tabulated for practical use, which supplement the tables in Wu and Zhang (1993), Zhang and Shao (2001) and Mukerjee and Wu (2001).     

A comparative study of tests for paired lifetime data

In this paper, we investigate different procedures for testing the equality of two mean survival times in paired lifetime studies. We consider Owen’s M-test and Q-test, a likelihood ratio test, the paired t-test, the Wilcoxon signed rank test and a permutation test based on log-transformed survival times in the comparative study. We also consider the paired t-test, the Wilcoxon signed rank test and a permutation test based on original survival times for the sake of comparison. The size and power characteristics of these tests are studied by means of Monte Carlo simulations under a frailty Weibull model. For less skewed marginal distributions, the Wilcoxon signed rank test based on original survival times is found to be desirable. Otherwise, the M-test and the likelihood ratio test are the best choices in terms of power. In general, one can choose a test procedure based on information about the correlation between the two survival times and the skewness of the marginal survival distributions.     

Saddlepoint <Emphasis Type="Italic">p</Emphasis>-values and confidence intervals for a class of two sample permutation tests for current status and panel count data

The current status and panel count data frequently arise from cancer and tumorigenicity studies when events currently occur. A common and widely used class of two sample tests, for current status and panel count data, is the permutation class. We manipulate the double saddlepoint method to calculate the exact mid-p-values of the underlying permutation distributions of this class of tests. Permutation simulations are replaced by analytical saddlepoint computations which provide extremely accurate mid-p-values that are exact for most practical purposes and almost always more accurate than normal approximations. The method is illustrated using two real tumorigenicity panel count data. To compare the saddlepoint approximation with the normal asymptotic approximation, a simulation study is conducted. The speed and accuracy of the saddlepoint method facilitate the calculation of the confidence interval for the treatment effect. The inversion of the mid-p-values to calculate the confidence interval for the mean rate of development of the recurrent event is discussed.     

Accurate ARL computation for EWMA-S<Superscript>2</Superscript> control charts

Originally, the exponentially weighted moving average (EWMA) control chart was developed for detecting changes in the process mean. The average run length (ARL) became the most popular performance measure for schemes with this objective. When monitoring the mean of independent and normally distributed observations the ARL can be determined with high precision. Nowadays, EWMA control charts are also used for monitoring the variance. Charts based on the sample variance S² are an appropriate choice. The usage of ARL evaluation techniques known from mean monitoring charts, however, is difficult. The most accurate method—solving a Fredholm integral equation with the Nyström method—fails due to an improper kernel in the case of chi-squared distributions. Here, we exploit the collocation method and the product Nyström method. These methods are compared to Markov chain based approaches. We see that collocation leads to higher accuracy than currently established methods.     

Construction and Nonparametric Testing of Orthogonal Arrays through Algebraic Strata and Inequivalent Permutation Matrices

In this article we use a new methodology, based on algebraic strata, to generate the class of all the orthogonal arrays of given size and strength. From this class we extract all the non isomorphic orthogonal arrays. Then, using all these non isomorphic orthogonal arrays, we suggest a method based on the inequivalent matrices permutations testing procedures Basso et al. (2004 Basso , D. , Evangelaras , H. , Koukouvinos , C. , Salmaso , L. ( 2004 ). Nonparametric testing for main effects on inequivalent designs. Proc. 7th Int. Workshop Model-Oriented Design Anal. Heeze, Netherlands, June 14–18 . [Google Scholar]) in order to obtain separate permutation tests for the effects in unreplicated mixed level fractional factorial designs. In order to validate the proposed method we perform a Monte Carlo simulation study and find out that the permutation tests appear to be a valid solution for testing effects, in particular when the usual normality assumptions cannot be justified.     

Distribution of extremes of<Emphasis Type="Italic">r</Emphasis><Superscript>th</Superscript> concomitant from the Morgenstern family

In the present paper the distribution theory of maximum and minimum of ther ^th concomitants from k independent subgroups each of same size m from the Morgenstern family is investigated. Some applications of the results in estimation of the scale parameter of a marginal variable in the bivariate uniform distribution and a selection problem are discussed.     

Testing fuzzy hypotheses based on vague observations: a <Emphasis Type="Italic">p</Emphasis>-value approach

This paper deals with the problem of testing statistical hypotheses when both the hypotheses and data are fuzzy. To this end, we first introduce the concept of fuzzy p-value and then develop an approach for testing fuzzy hypotheses by comparing a fuzzy p-value and a fuzzy significance level. Numerical examples are provided to illustrate the approach for different cases.     

A method for testing interaction in unreplicated two-way tables: using all pairwise interaction contrasts

Several methods exist for testing interaction in unreplicated two-way layouts. Some are based on specifying a functional form for the interaction term and perform well provided that the functional form is appropriate. Other methods do not require such a functional form to be specified but only test for the presence of non-additivity and do not provide a suitable estimate of error variance for a non-additive model. This paper presents a method for testing for interaction in unreplicated two-way tables that is based on testing all pairwise interaction contrasts. This method (i) is easy to implement, (ii) does not assume a functional form for the interaction term, (iii) can find a sub-table of data which may be free from interaction and to base the estimate of unknown error variance, and (iv) can be used for incomplete two-way layouts. The proposed method is illustrated using examples and its power is investigated via simulation studies. Simulation results show that the proposed method is competitive with existing methods for testing for interaction in unreplicated two-way layouts.     

A robust permutation test for the concordance correlation coefficient

In this work, we developed a robust permutation test for the concordance correlation coefficient (ρ_c) for testing the general hypothesis H₀ : ρ_c = ρ_c(0). The proposed test is based on an appropriately studentized statistic. Theoretically, the test is proven to be asymptotically valid in the general setting when two paired variables are uncorrelated but dependent. This desired property was demonstrated across a range of distributional assumptions and sample sizes in simulation studies, where the test exhibits robust type I error control in all settings tested, even when the sample size is small. We demonstrated the application of this test in two real world examples across cardiac output measurements and endocardiographic imaging.     

Properties of a randomization test for multifactor comparisons of groups

Multivariate hypothesis testing in studies of vegetation is likely to be hindered by unrealistic assumptions when based on conventional statistical methods. This can be overcome by randomization tests. In this paper, the accuracy and power of a MANOVA randomization test are evaluated for one and two factors with interaction with simulated data from three distributions. The randomization test is based on the partitioning of sum of squares computed from Euclidean distances. In one-factor designs, sample size and variance inequality were evaluated. The results showed a high level of accuracy. The power curve was higher with normal distribution, lower with uniform, intermediate with lognormal and was sensitive to variance inequality. In two-factor designs, three methods of permutations and two statistics were compared. The results showed that permutation of the residuals with F _pseudo is accurate and can give good power for testing the interaction and restricted permutation for testing main factors.     

A modified two-sample t-test based on permutation method for large-scale data

In large-scale data, for example, analyzing microarray data, which includes hypothesis testing for equality of means in order to discover differentially expressed genes, often deals with a large number of features versus a few number of replicates. Furthermore, some genes are differentially expressed and some others not. Thus, a usual permutation method, which is applied facing these situations, estimates the p-value poorly. This is because two types of genes are mixed. To overcome this obstacle, the null permutation samples are suggested in the literatures. We propose a modified uniformly most powerful unbiased test for testing the null hypothesis.     

Minimal augmented square designs and their extensions

Minimal square designs are proposed and compared. All treatment contrasts in both designs are estimable under the existence of two-way heterogeneity. That is, all designs are treatment-connected. Extended treatment-connected designs are generated by adding one column to minimal treatment-connected square designs. The extended designs not only have lower variances in paired comparisons of unreplicated treatments but also provide necessary degrees of freedom to estimate the process error. (M,S)-optimal extended designs are constructed systematically. Both square designs and their extensions have large numbers of unreplicated treatments.     

A Class of Permutation Tests for the Equality of Two Marginal Survival Functions Using Paired Censored Data

We studied several test statistics for testing the equality of marginal survival functions of paired censored data. The null distribution of the test statistics was approximated by permutation. These tests do not require explicit modeling or estimation of the within-pair correlation, accommodate both paired data and singletons, and the computation is straightforward with most statistical software. Numerical studies showed that these tests have competitive size and power performance. One test statistic has higher power than previously published test statistics when the two survival functions under comparison cross. We illustrate use of these tests in a propensity score matched dataset.     

Une application des méthodes séquentielles aux tests de permutations

To carry out a permutation test we have to examine the n! permutations of the observations. In order to make the permutation test feasible, Dwass (1957) proposed to examine only a sample of these permutations. With the help of sequential methods, we obtain a test which is never less efficient than that proposed by Dwass or the permutation test itself, in the sense that it is as powerful and never requires more permutations to make a decision. In practice, we can expect to gain much efficiency.     

Multivariate two‐sample permutation tests for trials with multiple time‐to‐event outcomes

Clinical trials involving multiple time‐to‐event outcomes are increasingly common. In this paper, permutation tests for testing for group differences in multivariate time‐to‐event data are proposed. Unlike other two‐sample tests for multivariate survival data, the proposed tests attain the nominal type I error rate. A simulation study shows that the proposed tests outperform their competitors when the degree of censored observations is sufficiently high. When the degree of censoring is low, it is seen that naive tests such as Hotelling's T² outperform tests tailored to survival data. Computational and practical aspects of the proposed tests are discussed, and their use is illustrated by analyses of three publicly available datasets. Implementations of the proposed tests are available in an accompanying R package.     

Probability generating function of GPED<Subscript>2</Subscript>

The Probability generating function of a random variable which has Generalized Polya Eggenberger Distribution of the second kind (GPED ₂) is obtained. The probability density function of the range R, in random sampling from a uniform distribution on (k, l) and exponential distribution with parameter λ is obtained, when the sample size is a random variable from GPED ₂. The results of Bazargan-Lari (2004) follow as special cases.     

On Hotelling’s T2 test in a special paired sample case

In a special paired sample case, Hotelling’s T² test based on the differences of the paired random vectors is the likelihood ratio test for testing the hypothesis that the paired random vectors have the same mean; with respect to a special group of affine linear transformations it is the uniformly most powerful invariant test for the general alternative of a difference in mean. We present an elementary straightforward proof of this result. The likelihood ratio test for testing the hypothesis that the covariance structure is of the assumed special form is derived and discussed. Applications to real data are given.     

A robust I-sample analysis of means type randomization test for variances for unbalanced designs

Two analysis of means type randomization tests for testing the equality of I variances for unbalanced designs are presented. Randomization techniques for testing statistical hypotheses can be used when parametric tests are inappropriate. Suppose that I independent samples have been collected. Randomization tests are based on shuffles or rearrangements of the (combined) sample. Putting each of the I samples ‘in a bowl’ forms the combined sample. Drawing samples ‘from the bowl’ forms a shuffle. Shuffles can be made with replacement (bootstrap shuffling) or without replacement (permutation shuffling). The tests that are presented offer two advantages. They are robust to non-normality and they allow the user to graphically present the results via a decision chart similar to a Shewhart control chart. A Monte Carlo study is used to verify that the permutation version of the tests exhibit excellent power when compared to other robust tests. The Monte Carlo study also identifies circumstances under which the popular Levene's test fails.     

MMCTest—A Safe Algorithm for Implementing Multiple Monte Carlo Tests

Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest , a sequential algorithm that gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal p‐values. The method can be used with a class of multiple testing procedures that include the Benjamini and Hochberg false discovery rate procedure and the Bonferroni correction controlling the familywise error rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses that can already be decided as being rejected or non‐rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non‐rejected and the undecided hypotheses. A simulation study motivated by actual biological data shows that MMCTest is usable in practice and that, despite the additional guarantee, it can be computationally more efficient than other methods.