A more powerful unconditional exact test of homogeneity for 2 × c contingency table analysis

The classical unconditional exact p-value test can be used to compare two multinomial distributions with small samples. This general hypothesis requires parameter estimation under the null which makes the test severely conservative. Similar property has been observed for Fisher's exact test with Barnard and Boschloo providing distinct adjustments that produce more powerful testing approaches. In this study, we develop a novel adjustment for the conservativeness of the unconditional multinomial exact p-value test that produces nominal type I error rate and increased power in comparison to all alternative approaches. We used a large simulation study to empirically estimate the 5th percentiles of the distributions of the p-values of the exact test over a range of scenarios and implemented a regression model to predict the values for two-sample multinomial settings. Our results show that the new test is uniformly more powerful than Fisher's, Barnard's, and Boschloo's tests with gains in power as large as several hundred percent in certain scenarios. Lastly, we provide a real-life data example where the unadjusted unconditional exact test wrongly fails to reject the null hypothesis and the corrected unconditional exact test rejects the null appropriately.     

The exact unconditional z-pooled test for equality of two binomial probabilities: optimal choice of the Berger and Boos confidence coefficient

Exact unconditional tests for comparing two binomial probabilities are generally more powerful than conditional tests like Fisher's exact test. Their power can be further increased by the Berger and Boos confidence interval method, where a p-value is found by restricting the common binomial probability under H ₀ to a 1?γ confidence interval. We studied the average test power for the exact unconditional z-pooled test for a wide range of cases with balanced and unbalanced sample sizes, and significance levels 0.05 and 0.01. The detailed results are available online on the web. Among the values 10^?3, 10^?4, …, 10^?10, the value γ=10^?4 gave the highest power, or close to the highest power, in all the cases we looked at, and can be given as a general recommendation as an optimal γ.     

Testing the homogeneity of several two‐parameter populations

The authors extend Fisher's method of combining two independent test statistics to test homogeneity of several two‐parameter populations. They explore two procedures combining asymptotically independent test statistics: the first pools two likelihood ratio statistics and the other, score test statistics. They then give specific results to test homogeneity of several normal, negative binomial or beta‐binomial populations. Their simulations provide evidence that in this context, Fisher's method performs generally well, even when the statistics to be combined are only asymptotically independent. They are led to recommend Fisher's test based on score statistics, since the latter have simple forms, are easy to calculate, and have uniformly good level properties.     

Confidence Intervals on Sum of Variance Components in Unbalanced Two-Fold Nested Designs with Equal Subsampling

I hybrid significance test, which blends exact and asymptotic theory in a unique way, is presided as an alternative to Fisher's exact test for unordered rxc contingency tables. The hybrid test is almost equivlent to Fisher's exact test, but requires considerably less computational effort The accuracy of the hybrid p-value is not compromised by sparse contingency tables.     

Partial Round-Robin Comparisons with Perfect Rankings

In teaching the development of uniformly most powerful unbiased (UMPU) tests, one rarely discusses the performance of alternative biased tests. It is shown, through the comparison of two independent Bernoulli proportions, that a biased test (the Z test) can be more powerful than the UMPU test (Fisher's exact test—randomized) in a large region of the alternative parameter space. A more general example is also given.     

Exact computation of minimum sample size for estimation of binomial parameters

It is indicated by some researchers in the literature that it might be difficult to exactly determine the minimum sample size for the estimation of a binomial parameter with prescribed margin of error and confidence level. In this paper, we investigate such a very old but also extremely important problem and demonstrate that the difficulty for obtaining the exact solution is not insurmountable. Unlike the classical approximate sample size method based on the central limit theorem, we develop a new approach for computing the minimum sample size that does not require any approximation. Moreover, our approach overcomes the conservatism of existing rigorous sample size methods derived from Bernoulli's theorem or Chernoff-Hoeffding bound.Our computational machinery consists of two essential ingredients. First, we prove that the minimum of coverage probability with respect to a binomial parameter bounded in an interval is attained at a discrete set of finite many values of the binomial parameter. This allows for reducing infinite many evaluations of coverage probability to finite many evaluations. Second, a recursive bounding technique is developed to further improve the efficiency of computation.     

Exact Tests for 2 × 2 Contingency Tables

Fisher's exact test, difference in proportions, log odds ratio, Pearson's chi-squared, and likelihood ratio are compared as test statistics for testing independence of two dichotomous factors when the associated p values are computed by using the conditional distribution given the marginals. The statistics listed above that can be used for a one-sided alternative give identical p values. For a two-sided alternative, many of the above statistics lead to different p values. The p values are shown to differ only by which tables in the opposite tail from the observed table are considered more extreme than the observed table.     

A Nonrandomized,Nonconservative Version of the Fisher Exact Test

The Fisher exact test has been unjustly dismissed by some as ‘only conditional,’ whereas it is unconditionally the uniform most powerful test among all unbiased tests, tests of size α and with power greater than its nominal level of significance α. The problem with this truly optimal test is that it requires randomization at the critical value(s) to be of size α. Obviously, in practice, one does not want to conclude that ‘with probability x the we have a statistical significant result.’ Usually, the hypothesis is rejected only if the test statistic's outcome is more extreme than the critical value, reducing the actual size considerably.

The randomized unconditional Fisher exact is constructed (using Neyman–structure arguments) by deriving a conditional randomized test randomizing at critical values c(t) by probabilities γ(t), that both depend on the total number of successes T (the complete-sufficient statistic for the nuisance parameter—the common success probability) conditioned upon.

In this paper, the Fisher exact is approximated by deriving nonrandomized conditional tests with critical region including the critical value only if γ (t) > γ₀, for a fixed threshold value γ₀, such that the size of the unconditional modified test is for all value of the nuisance parameter—the common success probability—smaller, but as close as possible to α. It will be seen that this greatly improves the size of the test as compared with the conservative nonrandomized Fisher exact test.

Size, power, and p value comparison with the (virtual) randomized Fisher exact test, and the conservative nonrandomized Fisher exact, Pearson's chi-square test, with the more competitive mid-p value, the McDonald's modification, and Boschloo's modifications are performed under the assumption of two binomial samples.     

Dempster's combination rule and sufficient statistics

In his articles (1966-1968) concerning statistical inference based on lower and upper probabilities, Dempster refers to the connection between Fisher's fiducial argument and his own ideas of statistical inference. Dempster's main concern however focuses on the “Bayesian” aspects of his theory and not on an elaboration of the relation between Fisher's and his ideas. This article attempts to work out the connection between those two approaches and focuses primarily on the question, whether Dempster's combination rule, his upper and lower probabilty based on sufficient statistics and inference based on sufficient statistics in Fisher's sense are consistent. To be adequate to Fisher's reasoning, we deal with absolutely continuous, one parametric families of distributions.This is certainly not the usual assumption in context with Dempster's theory and implies a normative but straightforward definition concerning the underlying conditional distribution; this definition however is done in Dempster's spirit as can be seen from his articles, (1966, 1968,a,b). Under those assumptions it can be shown that - similar to Lindley's results concerning consistency in fiducial reasoning (1958) - the combination rule, Dempster's procedure based on sufficient statistics and fiducial inference by sufficient statistics agree iff the parametric family under consideration can be transformed to location parameter form.     

The analysis of multicentre clinical trials when there is heterogeneity between centres

Two-treatment multicentre clinical trials are very common in practice. In cases where a non-parametric analysis is appropriate, a rank-sum test for grouped data called the van Elteren test can be applied. As an alternative approach, one may apply a combination test such as Fisher's combination test or the inverse normal combination test (also called Liptak's method) in order to combine centre-specific P-values. If there are no ties and no differences between centres with regard to the groups’ sample sizes, the inverse normal combination test using centre-specific Wilcoxon rank-sum tests is equivalent to the van Elteren test. In this paper, the van Elteren test is compared with Fisher's combination test based on Wilcoxon rank-sum tests. Data from two multicentre trials as well as simulated data indicate that Fisher's combination of P-values is more powerful than the van Elteren test in realistic scenarios, i.e. when there are large differences between the centres’ P-values, some quantitative interaction between treatment and centre, and/or heterogeneity in variability. The combination approach opens the possibility of using statistics other than the rank sum, and it is also a suitable method for more complicated designs, e.g. when covariates such as age or gender are included in the analysis.     

New critical region tables for Fisher's exact test

The best-known non-asymptotic method for comparing two independent proportions is Fisher's exact text. The usual critical region (CR) tables for this test contain one or more of the following defects:they distinguish between rows and columns; they distinguish between the alternatives H = p1 < p2 and H = p1 > p2; they assume that the error for the two-tailed test is twice that of the one-tailed test; they do not use the optimal version of the test; they do not give both CRs for one and two tails at the same time. All this results in the unnecessary duplication of the space required for the tables, the construction of tables of low-powered methods, or the need to manipulate two different tables (one for the one-tailed test, the other for the two-tailed test). This paper presents CR tables which have been obtained from the most powerful version of Fisher's exact test and which occupy the minimum space possible. The tables, which are valid for one- or two-tailed tests, have levels of significance of 10%, 5% and 1% and values for N (the total size of both samples) of less than or equal to 40. This article shows how to calculate the P value in a specific problem, using the tables as a means of partial checking and as a preliminary step to determining the exact P value.     

A generalization of Fisher's exact test in p×q contingency tables using more concordant relations

A randomized test for pxq contingency tables to test independence against non-independence of two categorical random variables, a generalization of Fisher's exact test, is constructed. Based on the power of the test, subsets of the alternative region are identified.     

On the choice of transformations of the correlation coefficient with or without an outlier

Five transformations of the correlation coefficient, namely, Fisher's z, Nair's u, Sankaran's v, Ruben's y and Samiuddin's t are compared numerically using confidence intervals. Samiuddin's t_s transformation is close to the exact nominal confidence level for a small sample size ≤ 25 from a bivariate normal density. For a sample size > 25 both Samiuddin's t_s and Fisher's z can be used. In the presence of an outlier (on a minor axis), both Fisher's z and Samiuddin's t_s are not affected as long as |p| ≤ 0.3 but are seriously affected when |p&| > 0.3.     

Comments on “a generalization of fisher's exact test in pxq contingency tables using more concordant relations”

The purpose of this note is to criticize Nguyen (1985) for his account of the literature on the generalization of Fisher's exact test and to point out parallels with existing algorithms of the algorithm proposed by Nguyen. Subsequently we will briefly raise some questions on the methodology proposed by Nguyen.

Nguyen (1985) suggests that all literature on exact testing prior to Nguyen & Sampson (1985) is based on the “more probable” relation or Exact Probability Test (EPT) as a test statistic. This is not correct. Yates (1934 - Pearson's X²), Lewontin & Felsenstein (1965 - X²), Agresti & Wackerly (1977 - X², Kendall's tau, Kruskal & Goodman's gamma), Klotz (1966 - Wilcoxon), Klotz & Teng (1977 - Kruskall & Wallis' H), Larntz (1978 - X², loglike-lihood-ratio statistic G², Freeman & Tukey statistic), and several others have investigated exact tests with other statistics than the EPT. In fact, Bennett & Nakamura (1963) are incorrectly cited as they investigated both X² and G², rather than EPT. Also, Freeman & Halton (1951) are incorrectly cited for they generalized Fisher's exact test to pxq tables and not 2xq tables as stated. And they are even predated by Yates (1934) who extended the test to 2×3 tables.     

Sample Sizes in the Interval Estimation of the Correlation Coefficient

An expression is derived for the maximum length of the interval estimator of the correlation coefficient, p, under bivariate normal assumptions. The prespecification of this minimum attainable precision and the confidence level results in an expression for the sample size required. An approximate expression for the sample size is proposed and is numerically shown to be as good as or better than that based on the Fisher's Z transformation.     

Tests for detecting overdispersion in poisson models

Collings and Margolin(1985) developed a locally most powerful unbiased test for detecting negative binomial departures from a Poisson model, when the variance is a quadratic function of the mean. Kim and Park(1992) developed a locally most powerful unbiased test, when the variance is a linear function of the mean. It is found that a different mean-variance structure of a negative binomial derives a different locally optimal test statistic.

In this paper Collings and Margolin's and Kim and Park's results are unified and extended by developing a test for overdispersion in Poisson model against Katz family of distributions, Our setup has two extensions: First, Katz family of distributions is employed as an extension of the negative binomial distribution. Second, the mean-variance structure of the mixed Poisson model is given by σ² = μ+cμ^r for arbitrary but fixed r. We derive a local score test for testing H₀ : c = 0. Superiority of a new test is proved by the asymtotic relative efficiency as well as the simulation study.     

The Appropriateness of Some Common Procedures for Testing the Equality of Two Independent Binomial Populations

For testing the equality of two independent binomial populations the Fisher exact test and the chi-squared test with Yates's continuity correction are often suggested for small and intermediate size samples. The use of these tests is inappropriate in that they are extremely conservative. In this article we demonstrate that, even for small samples, the uncorrected chi-squared test (i.e., the Pearson chi-squared test) and the two-independent-sample t test are robust in that their actual significance levels are usually close to or smaller than the nominal levels. We encourage the use of these latter two tests.     

On Two Forms of Fisher's Measure of Information

ABSTRACT

Fisher's information number is the second moment of the “score function” where the derivative is with respect to x rather than Θ. It is Fisher's information for a location parameter, and also called shift-invariant Fisher information. In recent years, Fisher's information number has been frequently used in several places regardless of parameters of the distribution or of their nature. Is this number a nominal, standard, and typical measure of information? The Fisher information number is examined in light of the properties of classical statistical information theory. It has some properties analogous to those of Fisher's measure, but, in general, it does not have good properties if used as a measure of information when Θ is not a location parameter. Even in the case of location parameter, the regularity conditions must be satisfied. It does not possess the two fundamental properties of the mother information, namely the monotonicity and invariance under sufficient transformations. Thus the Fisher information number should not be used as a measure of information (except when Θ a location parameter). On the other hand, Fisher's information number, as a characteristic of a distribution f(x), has other interesting properties. As a byproduct of its superadditivity property a new coefficient of association is introduced.     

To adjust or not to adjust for baseline when analyzing repeated binary responses? The case of complete data when treatment comparison at study end is of interest

The benefits of adjusting for baseline covariates are not as straightforward with repeated binary responses as with continuous response variables. Therefore, in this study, we compared different methods for analyzing repeated binary data through simulations when the outcome at the study endpoint is of interest. Methods compared included chi‐square, Fisher's exact test, covariate adjusted/unadjusted logistic regression (Adj.logit/Unadj.logit), covariate adjusted/unadjusted generalized estimating equations (Adj.GEE/Unadj.GEE), covariate adjusted/unadjusted generalized linear mixed model (Adj.GLMM/Unadj.GLMM). All these methods preserved the type I error close to the nominal level. Covariate adjusted methods improved power compared with the unadjusted methods because of the increased treatment effect estimates, especially when the correlation between the baseline and outcome was strong, even though there was an apparent increase in standard errors. Results of the Chi‐squared test were identical to those for the unadjusted logistic regression. Fisher's exact test was the most conservative test regarding the type I error rate and also with the lowest power. Without missing data, there was no gain in using a repeated measures approach over a simple logistic regression at the final time point. Analysis of results from five phase III diabetes trials of the same compound was consistent with the simulation findings. Therefore, covariate adjusted analysis is recommended for repeated binary data when the study endpoint is of interest. Copyright © 2015 John Wiley & Sons, Ltd.     

On the Power of Bootstrapped Specification Tests

Abstract

Decisions based on econometric model estimates may not have the expected effect if the model is misspecified. Thus, specification tests should precede any analysis. Bierens' specification test is consistent and has optimality properties against some local alternatives. A shortcoming is that the test statistic is not distribution free, even asymptotically. This makes the test unfeasible. There have been many suggestions to circumvent this problem, including the use of upper bounds for the critical values. However, these suggestions lead to tests that lose power and optimality against local alternatives. In this paper we show that bootstrap methods allow us to recover power and optimality of Bierens' original test. Bootstrap also provides reliable p-values, which have a central role in Fisher's theory of hypothesis testing. The paper also includes a discussion of the properties of the bootstrap Nonlinear Least Squares Estimator under local alternatives.