Evaluating the Accuracy of Small P‐Values In Genetic Association Studies Using Edgeworth Expansions

The asymptotic distributions of many classical test statistics are normal. The resulting approximations are often accurate for commonly used significance levels, 0.05 or 0.01. In genome‐wide association studies, however, the significance level can be as low as 1×10⁻⁷, and the accuracy of the p‐values can be challenging. We study the accuracies of these small p‐values are using two‐term Edgeworth expansions for three commonly used test statistics in GWAS. These tests have nuisance parameters not defined under the null hypothesis but estimable. We derive results for this general form of testing statistics using Edgeworth expansions, and find that the commonly used score test, maximin efficiency robust test and the chi‐squared test are second order accurate in the presence of the nuisance parameter, justifying the use of the p‐values obtained from these tests in the genome‐wide association studies.     

A test statistic for weighted runs

A new test statistic based on runs of weighted deviations is introduced. Its use for observations sampled from independent normal distributions is worked out in detail. It supplements the classic χ² test which ignores the ordering of observations and provides additional sensitivity to local deviations from expectations. The exact distribution of the statistic in the non-parametric case is derived and an algorithm to compute p-values is presented. The computational complexity of the algorithm is derived employing a novel identity for integer partitions.     

Reconsideration of the F-test of the analysis of variance: the semi-bayesian significance test

The usual F-test of the analysis of variance is reconsidered within the Bayesian framework, In terms of predictive distributions, This leads to the notion of semi-Bayesian significance test, so called because it consists in only probabilizing the space of nuisance parameters, thus bringing a general principle for “eliminating” nuisance parameters, or more exactly incorporating information about these parameters. The approach is shown to extend the F-tests, by allowing the testing of hypotheses of non-zero effects.     

Randomization in a complete block design with few blocks

It is known that the distributions of statistics commonly used in experimental design (notably the F statistic) involve certain nuisance parameters which depend on the model and the design. Raudomization was a technique introduced by R.A. Fisher to eliminate some of these parameter and produce a usable distribution function. We will show that there is a close relationship between the analytical properties of the non-randomized distribution and the more combinatorial properties of the nuisance parameters. This relationship allows us to determine theoretically, and practically in some cases, how good an approximation the central F distribution is to the randomized distribution.     

Balanced fractional factorial designs of resolution 2l+1 for interesting effects orthogonal to some nuisance parameters: 2m1+m2 series

For 2^m₁+m₂ factorial designs, this paper investigates balanced fractional 2^m₁ factorial designs of resolution 2l+1 with some nuisance parameters concerning the second factors. They are derivable from partially balanced arrays and further permit estimation of the effects up to the l-factor interactions concerning the first factors orthogonally to the nuisance parameters.     

Robust Bayesian hypothesis testing in the presence of nuisance parameters

Robust Bayesian testing of point null hypotheses is considered for problems involving the presence of nuisance parameters. The robust Bayesian approach seeks answers that hold for a range of prior distributions. Three techniques for handling the nuisance parameter are studied and compared. They are (i) utilize a noninformative prior to integrate out the nuisance parameter; (ii) utilize a test statistic whose distribution does not depend on the nuisance parameter; and (iii) use a class of prior distributions for the nuisance parameter. These approaches are studied in two examples, the univariate normal model with unknown mean and variance, and a multivariate normal example.     

Some asymptotic results for hypothesis testing in multivariate failure time data

This paper presents a class of generalized Wald, generalized score and generalized likelihood ratio statistics for hypothesis testing and model selection for multivariate failure time data. These statistics are based on a marginal hazard model with a common baseline hazard function. The large sample distributions of these statistics are examined. It is shown that the proposed test statistics follow asymptotically a weighted sum of independent χ₁² distributions.     

Irreconcilability of P-value and Bayesian measure in two-sided hypothesis: Pareto distribution with the presence of nuisance parameter

Abstract

This article is concerned with the comparison of Bayesian and classical testing of a point null hypothesis for the Pareto distribution when there is a nuisance parameter. In the first stage, using a fixed prior distribution, the posterior probability is obtained and compared with the P-value. In the second case, lower bounds of the posterior probability of H₀, under a reasonable class of prior distributions, are compared with the P-value. It has been shown that even in the presence of nuisance parameters for the model, these two approaches can lead to different results in statistical inference.     

Testing with a nuisance parameter present only under the alternative: a score-based approach with application to segmented modelling

ABSTRACT

We introduce a score-type statistic to test for a non-zero regression coefficient when the relevant term involves a nuisance parameter present only under the alternative. Despite the non-regularity and complexity of the problem and unlike the previous approaches, the proposed test statistic does not require the nuisance to be estimated. It is simple to implement by relying on the conventional distributions, such as Normal or t, and it justified in the setting of probabilistic coherence. We focus on testing for the existence of a breakpoint in segmented regression, and illustrate the methodology with an analysis on data of DNA copy number aberrations and gene expression profiles from 97 breast cancer patients; moreover some simulations reveal that the proposed test is more powerful than its competitors previously discussed in literature.     

Local power properties of some asymptotic tests in symmetric linear regression models

In this paper we obtain asymptotic expansions, up to order n^−1/2 and under a sequence of Pitman alternatives, for the nonnull distribution functions of the likelihood ratio, Wald, score and gradient test statistics in the class of symmetric linear regression models. This is a wide class of models which encompasses the t model and several other symmetric distributions with longer-than normal tails. The asymptotic distributions of all four statistics are obtained for testing a subset of regression parameters. Furthermore, in order to compare the finite-sample performance of these tests in this class of models, Monte Carlo simulations are presented. An empirical application to a real data set is considered for illustrative purposes.     

Comparison of powers for the sphericity tests using both the asymptotic distribution and the bootstrap

The asymptotic distributions of two tests for sphericity:the locally most powerful invariant test and the likelihood ratio test are derived under the general alternaties ∑?σ² I. The powers of these two tests are then compared when the data are from a trivariate normal population. The bootstrap method is also used to obtain the powers and the powers obtained by this method agree with those from the asymptotic distributions.     

A simple nonparametric test for bivariate symmetry about a line

Over the years many researchers have dealt with testing the hypotheses of symmetry in univariate and multivariate distributions in the parametric and nonparametric setup. In a multivariate setup, there are several formulations of symmetry, for example, symmetry about an axis, joint symmetry, marginal symmetry, radial symmetry, symmetry about a known point, spherical symmetry, and elliptical symmetry among others. In this paper, for the bivariate case, we formulate a concept of symmetry about a straight line passing through the origin in a plane and accordingly develop a simple nonparametric test for testing the hypothesis of symmetry about a straight line. The proposed test is based on a measure of deviance between observed counts of bivariate samples in suitably defined pairs of sets. The exact null distribution and non-null distribution, for specified classes of alternatives, of the test statistics are obtained. The null distribution is tabulated for sample size from n=5 up to n=30. The null mean, null variance and the asymptotic null distributions of the proposed test statistics are also obtained. The empirical power of the proposed test is evaluated by simulating samples from the suitable class of bivariate distributions. The empirical findings suggest that the test performs reasonably well against various classes of asymmetric bivariate distributions. Further, it is advocated that the basic idea developed in this work can be easily adopted to test the hypotheses of exchangeability of bivariate random variables and also bivariate symmetry about a given axis which have been considered by several authors in the past.     

A tuning parameter free test for properties of space–time covariance functions

We propose a new nonparametric test to test for symmetry and separability of space–time covariance functions. Unlike the existing nonparametric tests, our test has the attractive convenience of being free of choosing any user-chosen number or smoothing parameter. The asymptotic null distributions of the test statistics are free of nuisance parameters and the critical values have been tabulated in the literature. From a practical point of view, our test is easy to implement and can be readily used by the practitioner. A Monte-Carlo experiment and real data analysis illustrate the finite sample performance of the new test.     

Using principal components to test normality of high-dimensional data

Many multivariate statistical procedures are based on the assumption of normality and different approaches have been proposed for testing this assumption. The vast majority of these tests, however, are exclusively designed for cases when the sample size n is larger than the dimension of the variable p, and the null distributions of their test statistics are usually derived under the asymptotic case when p is fixed and n increases. In this article, a test that utilizes principal components to test for nonnormality is proposed for cases when p/n → c. The power and size of the test are examined through Monte Carlo simulations, and it is argued that the test remains well behaved and consistent against most nonnormal distributions under this type of asymptotics.     

Asymptotic expansions of the null distributions of some test statistics for profile analysis in general distributions

This paper discusses asymptotic expansions for the null distributions of some test statistics for profile analysis under non-normality. It is known that the null distributions of these statistics converge to chi-square distribution under normality [Siotani, M., 1956. On the distributions of the Hotelling's T²$T^2$-statistics. Ann. Inst. Statist. Math. Tokyo 8, 1–14; Siotani, M., 1971. An asymptotic expansion of the non-null distributions of Hotelling's generalized T²$T^2$-statistic. Ann. Math. Statist. 42, 560–571]. We extend this result by obtaining asymptotic expansions under general distributions. Moreover, the effect of non-normality is also considered. In order to obtain all the results, we make use of matrix manipulations such as direct products and symmetric tensor, rather than usual elementwise tensor notation.     

Heteroscedastic ANOVA: old p values, new views

The generalization of the Behrens–Fisher problem to comparing more than two means from nonhomogeneous populations has attracted the attention of statisticians for many decades. Several approaches offer different approximations to the distribution of the test statistic. The question of statistical properties of these approximations is still alive. Here, we present a brief overview of several approaches suggested in the literature and implemented in software with a focus on investigating the accuracy of p values as well as their dependence on nuisance parameters and on the underlying assumption of normality. We illustrate by simulation the behavior of p values. In addition to the Satterthwaite–Fai–Cornelius test, the Kenward–Roger test, the simple ANOVA F test, the parametric bootstrap test, and the generalized F test will be briefly discussed.     

A multivariate rank test for comparing mass size distributions

Particle size analyses of a raw material are commonplace in the mineral processing industry. Knowledge of particle size distributions is crucial in planning milling operations to enable an optimum degree of liberation of valuable mineral phases, to minimize plant losses due to an excess of oversize or undersize material or to attain a size distribution that fits a contractual specification. The problem addressed in the present paper is how to test the equality of two or more underlying size distributions. A distinguishing feature of these size distributions is that they are not based on counts of individual particles. Rather, they are mass size distributions giving the fractions of the total mass of a sampled material lying in each of a number of size intervals. As such, the data are compositional in nature, using the terminology of Aitchison [1 Aitchison, J. 1986. “The Statistical Analysis of Compositional Data”. London: Chapman and Hall. [Crossref] , [Google Scholar]] that is, multivariate vectors the components of which add to 100%. In the literature, various versions of Hotelling's T ² have been used to compare matched pairs of such compositional data. In this paper, we propose a robust test procedure based on ranks as a competitor to Hotelling's T ². In contrast to the latter statistic, the power of the rank test is not unduly affected by the presence of outliers or of zeros among the data.     

On the asymptotic power and relative efficiency of the frequency X2 test

The asymptotic (Pitman) power of the X² test is investigated for particular classes of alternatives. A simple rule is introduced to identify ‘orthogonal alternatives’, for which the noncentrality parameter can be computed in a very simple way. In the sequel, restricted alternatives are considered and the ARE of the unrestricted test w.r.t. the restricted one is shown to depend only on the numbers of degrees of freedom. The concluding section discusses ‘undetectable alternatives’, i.e. alternatives for which the noncentrality vanishes.     

An adaptive test for the one-way layout

An adaptive test is proposed for the one-way layout. This test procedure uses the order statistics of the combined data to obtain estimates of percentiles, which are used to select an appropriate set of rank scores for the one-way test statistic. This test is designed to have reasonably high power over a range of distributions. The adaptive procedure proposed for a one-way layout is a generalization of an existing two-sample adaptive test procedure. In this Monte Carlo study, the power and significance level of the F-test, the Kruskal-Wallis test, the normal scores test, and the adaptive test were evaluated for the one-way layout. All tests maintained their significance level for data sets having at least 24 observations. The simulation results show that the adaptive test is more powerful than the other tests for skewed distributions if the total number of observations equals or exceeds 24. For data sets having at least 60 observations the adaptive test is also more powerful than the F-test for some symmetric distributions.     

A nonparametric test for similarity of marginals—With applications to the assessment of population bioequivalence

In this paper we suggest a completely nonparametric test for the assessment of similar marginals of a multivariate distribution function. This test is based on the asymptotic normality of Mallows distance between marginals. It is also shown that the n out of n bootstrap is weakly consistent, thus providing a theoretical justification to the work in Czado, C. and Munk, A. [2001. Bootstrap methods for the nonparametric assessment of population bioequivalence and similarity of distributions. J. Statist. Comput. Simulation 68, 243–280]. The test is extended to cross-over trials and is applied to the problem of population bioequivalence, where two formulations of a drug are shown to be similar up to a tolerable limit. This approach was investigated in small samples using bootstrap techniques in Czado, C., Munk, A. [2001. Bootstrap methods for the nonparametric assessment of population bioequivalence and similarity of distributions. J. Statist. Comput. Simulation 68, 243–280], showing that the bias corrected and accelerated bootstrap yields a very accurate and powerful finite sample correction. A data example is discussed.