False Discovery Control for Multiple Tests of Association Under General Dependence

Abstract.  We propose a confidence envelope for false discovery control when testing multiple hypotheses of association simultaneously. The method is valid under arbitrary and unknown dependence between the test statistics and allows for an exploratory approach when choosing suitable rejection regions while still retaining strong control over the proportion of false discoveries.     

Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing

Abstract.  A new multiple testing procedure, the generalized augmentation procedure (GAUGE), is introduced. The procedure is shown to control the false discovery exceedance and to be competitive in terms of power. It is also shown how to apply the idea of GAUGE to achieve control of other error measures. Extensions to dependence are discussed, together with a modification valid under arbitrary dependence. We present an application to an original study on prostate cancer and on a benchmark data set on colon cancer.     

More Powerful Control of the False Discovery Rate Under Dependence

In a breakthrough paper, Benjamini and Hochberg (J Roy Stat Soc Ser B 57:289–300, 1995) proposed a new error measure for multiple testing, the FDR; and developed a distribution-free procedure to control it under independence among the test statistics. In this paper we argue by extensive simulation and theoretical considerations that the assumption of independence is not needed. Along the lines of (Ann Stat 32:1035–1061, 2004b), we moreover provide a more powerful method, that exploits an estimator of the number of false nulls among the tests. We propose a whole family of iterative estimators that prove robust under dependence and independence between the test statistics. These estimators can be used to improve also classical multiple testing procedures, and in general to estimate the weight of a known component in a mixture distribution. Innovations are illustrated by simulations.     

Bayesian False Discovery Rate Wavelet Shrinkage: Theory and Applications

Statistical inference in the wavelet domain remains a vibrant area of contemporary statistical research because of desirable properties of wavelet representations and the need of scientific community to process, explore, and summarize massive data sets. Prime examples are biomedical, geophysical, and internet related data. We propose two new approaches to wavelet shrinkage/thresholding.

In the spirit of Efron and Tibshirani's recent work on local false discovery rate, we propose Bayesian Local False Discovery Rate (BLFDR), where the underlying model on wavelet coefficients does not assume known variances. This approach to wavelet shrinkage is shown to be connected with shrinkage based on Bayes factors. The second proposal, Bayesian False Discovery Rate (BaFDR), is based on ordering of posterior probabilities of hypotheses on true wavelets coefficients being null, in Bayesian testing of multiple hypotheses.

We demonstrate that both approaches result in competitive shrinkage methods by contrasting them to some popular shrinkage techniques.     

False Discovery Rate Control of Step‐Up‐Down Tests with Special Emphasis on the Asymptotically Optimal Rejection Curve

Abstract. This paper is concerned with exact control of the false discovery rate (FDR) for step‐up‐down (SUD) tests related to the asymptotically optimal rejection curve (AORC). Since the system of equations and/or constraints for critical values and FDRs is numerically extremely sensitive, existence and computation of valid solutions is a challenging problem. We derive explicit formulas for upper bounds of the FDR and show that under a well‐known monotonicity condition, control of the FDR by a step‐up procedure results in control of the FDR by a corresponding SUD procedure. Various methods for adjusting the AORC to achieve finite FDR control are investigated. Moreover, we introduce alternative FDR bounding curves and study their connection to rejection curves as well as the existence of critical values for exact FDR control with respect to the underlying FDR bounding curve. Finally, we propose an iterative method for the computation of critical values.     

On False Discovery and Non‐discovery Proportions of the Dynamic Adaptive Procedure

This paper studies the asymptotic behaviour of the false discovery and non‐discovery proportions of the dynamic adaptive procedure under some dependence structure. A Bahadur‐type representation of the cut point in simultaneously performing a large scale of tests is presented. The asymptotic bias decompositions of the false discovery and non‐discovery proportions are given under some dependence structure. In addition to existing literatures, we find that the randomness due to the dynamic selection of the tuning parameter in estimating the true null rate serves as a source of the approximation error in the Bahadur representation and enters into the asymptotic bias term of the false discovery proportion and those of the false non‐discovery proportion. The theory explains to some extent why some seemingly attractive dynamic adaptive procedures do not outperform the competing fixed adaptive procedures substantially in some situations. Simulations justify our theory and findings.     

Evaluation of false discovery rate and power via sample size in microarray studies

Microarray studies are now common for human, agricultural plant and animal studies. False discovery rate (FDR) is widely used in the analysis of large-scale microarray data to account for problems associated with multiple testing. A well-designed microarray study should have adequate statistical power to detect the differentially expressed (DE) genes, while keeping the FDR acceptably low. In this paper, we used a mixture model of expression responses involving DE genes and non-DE genes to analyse theoretical FDR and power for simple scenarios where it is assumed that each gene has equal error variance and the gene effects are independent. A simulation study was used to evaluate the empirical FDR and power for more complex scenarios with unequal error variance and gene dependence. Based on this approach, we present a general guide for sample size requirement at the experimental design stage for prospective microarray studies. This paper presented an approach to explicitly connect the sample size with FDR and power. While the methods have been developed in the context of one-sample microarray studies, they are readily applicable to two-sample, and could be adapted to multiple-sample studies.     

Bayesian Robustness of the Positive False Discovery Rate

The positive false discovery rate was introduced by Storey (2003 Storey , J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value. Ann. Statist. 31:2013–2035.[Crossref], [Web of Science ®] , [Google Scholar]) as an alternative to the family wise error rate for the case in which we are simultaneously testing a large amount of hypotheses. The positive false discovery rate has a very nice Bayesian interpretation (as it was shown by Storey, 2003 Storey , J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value. Ann. Statist. 31:2013–2035.[Crossref], [Web of Science ®] , [Google Scholar]) and its robustness is analyzed. The emphasis is on the ε-contamination class (one of the most used classes of priors for Bayesian robustness) and it is shown that robustness is not obtained when the basic prior concentrates the probability on the null hypothesis.     

A Bayesian False Discovery Rate for Multiple Testing

Case-control studies of genetic polymorphisms and gene-environment interactions are reporting large numbers of statistically significant associations, many of which are likely to be spurious. This problem reflects the low prior probability that any one null hypothesis is false, and the large number of test results reported for a given study. In a Bayesian approach to the low prior probabilities, Wacholder et al. (2004) suggest supplementing the p-value for a hypothesis with its posterior probability given the study data. In a frequentist approach to the test multiplicity problem, Benjamini & Hochberg (1995) propose a hypothesis-rejection rule that provides greater statistical power by controlling the false discovery rate rather than the family-wise error rate controlled by the Bonferroni correction. This paper defines a Bayes false discovery rate and proposes a Bayes-based rejection rule for controlling it. The method, which combines the Bayesian approach of Wacholder et al. with the frequentist approach of Benjamini & Hochberg, is used to evaluate the associations reported in a case-control study of breast cancer risk and genetic polymorphisms of genes involved in the repair of double-strand DNA breaks.     

Optimal False Discovery Rate Control with Kernel Density Estimation in a Microarray Experiment

Most of current false discovery rate (FDR) procedures in a microarray experiment assume restrictive dependence structures, resulting in being less reliable. FDR controlling procedure under suitable dependence structures based on Poisson distributional approximation is shown. Unlike other procedures, the distribution of false null hypotheses is estimated by using kernel density estimation allowing for dependent structures among the genes. Furthermore, we develop an FDR framework that minimizes the false nondiscovery rate (FNR) with a constraint on the controlled level of the FDR. The performance of the proposed FDR procedure is compared with that of other existing FDR controlling procedures, with an application to the microarray study of simulated data.     

False discovery rates for large-scale model checking under certain dependence

In many scientific fields, it is interesting and important to determine whether an observed data stream comes from a prespecified model or not, particularly when the number of data streams is of large scale, where multiple hypotheses testing is necessary. In this article, we consider large-scale model checking under certain dependence among different data streams observed at the same time. We propose a false discovery rate (FDR) control procedure to check those unusual data streams. Specifically, we derive an approximation of false discovery and construct a point estimate of FDR. Theoretical results show that, under some mild assumptions, our proposed estimate of FDR is simultaneously conservatively consistent with the true FDR, and hence it is an asymptotically strong control procedure. Simulation comparisons with some competing procedures show that our proposed FDR procedure behaves better in general settings. Application of our proposed FDR procedure is illustrated by the StarPlus fMRI data.     

Multiple testing using the posterior probabilities of directional alternatives,with application to genomic studies

This paper aims to connect Bayesian analysis and frequentist theory in the context of multiple comparisons. The authors show that when testing the equality of two sample means, the posterior probability of the one‐sided alternative hypothesis, defined as a half‐space, shares with the frequentist P‐value the property of uniformity under the null hypothesis. Ultimately, the posterior probability may thus be used in the same spirit as a P‐value in the Benjamini‐Hochberg procedure, or in any of its extensions.     

Power and stability comparisons of multiple testing procedures with false discovery rate control

High-throughput data analyses are widely used for examining differential gene expression, identifying single nucleotide polymorphisms, and detecting methylation loci. False discovery rate (FDR) has been considered a proper type I error rate to control for discovery-based high-throughput data analysis. Various multiple testing procedures have been proposed to control the FDR. The power and stability properties of some commonly used multiple testing procedures have not been extensively investigated yet, however. Simulation studies were conducted to compare power and stability properties of five widely used multiple testing procedures at different proportions of true discoveries for various sample sizes for both independent and dependent test statistics. Storey's two linear step-up procedures showed the best performance among all tested procedures considering FDR control, power, and variance of true discoveries. Leukaemia and ovarian cancer microarray studies were used to illustrate the power and stability characteristics of these five multiple testing procedures with FDR control.     

On the Screening of Large Numbers of Significance Tests

A brief review is given of procedures for the collective analysis of a large number of significance tests. A simple procedure previously supplied for isolating 'real' effects on the basis of a large number of significance tests is generalized to deal with two-sided tests and is also related more explicitly to the false discovery rate.     

Comparisons of estimators of the number of true null hypotheses and adaptive FDR procedures in multiplicity testing

Many exploratory studies such as microarray experiments require the simultaneous comparison of hundreds or thousands of genes. It is common to see that most genes in many microarray experiments are not expected to be differentially expressed. Under such a setting, a procedure that is designed to control the false discovery rate (FDR) is aimed at identifying as many potential differentially expressed genes as possible. The usual FDR controlling procedure is constructed based on the number of hypotheses. However, it can become very conservative when some of the alternative hypotheses are expected to be true. The power of a controlling procedure can be improved if the number of true null hypotheses (m ₀) instead of the number of hypotheses is incorporated in the procedure [Y. Benjamini and Y. Hochberg, On the adaptive control of the false discovery rate in multiple testing with independent statistics, J. Edu. Behav. Statist. 25(2000), pp. 60–83]. Nevertheless, m ₀ is unknown, and has to be estimated. The objective of this article is to evaluate some existing estimators of m ₀ and discuss the feasibility of these estimators in incorporating into FDR controlling procedures under various experimental settings. The results of simulations can help the investigator to choose an appropriate procedure to meet the requirement of the study.     

Evaluations of FWER-controlling methods in multiple hypothesis testing

Simultaneously testing a family of n null hypotheses can arise in many applications. A common problem in multiple hypothesis testing is to control Type-I error. The probability of at least one false rejection referred to as the familywise error rate (FWER) is one of the earliest error rate measures. Many FWER-controlling procedures have been proposed. The ability to control the FWER and achieve higher power is often used to evaluate the performance of a controlling procedure. However, when testing multiple hypotheses, FWER and power are not sufficient for evaluating controlling procedure’s performance. Furthermore, the performance of a controlling procedure is also governed by experimental parameters such as the number of hypotheses, sample size, the number of true null hypotheses and data structure. This paper evaluates, under various experimental settings, the performance of some FWER-controlling procedures in terms of five indices, the FWER, the false discovery rate, the false non-discovery rate, the sensitivity and the specificity. The results can provide guidance on how to select an appropriate FWER-controlling procedure to meet a study’s objective.     

On Efficient Estimators of the Proportion of True Null Hypotheses in a Multiple Testing Setup

We consider the problem of estimating the proportion θ of true null hypotheses in a multiple testing context. The setup is classically modelled through a semiparametric mixture with two components: a uniform distribution on interval [0,1] with prior probability θ and a non‐parametric density f . We discuss asymptotic efficiency results and establish that two different cases occur whether f vanishes on a non‐empty interval or not. In the first case, we exhibit estimators converging at a parametric rate, compute the optimal asymptotic variance and conjecture that no estimator is asymptotically efficient (i.e. attains the optimal asymptotic variance). In the second case, we prove that the quadratic risk of any estimator does not converge at a parametric rate. We illustrate those results on simulated data.     

A Note on the Relations Between Some Concepts of Positive Dependence

In this article, we provide some explicit examples showing that weak association in sequence is strictly weaker than weak association and strictly stronger than positive supermodular dependence. Furthermore, we show that strongly positive orthant dependence is strictly weaker than weakly association and strictly stronger than positive orthant dependence. Finally, we also show that positive supermodular dependence is not stronger than strongly positive orthant dependence.     

中国短期利率跳跃行为的实证研究

内容提要：通过在Vasicek模型中引入跳跃强度与宏观经济变量相关的跳跃成分,本文建立了一个更具一般性的跳跃－扩散动态利率期限结构模型,并对该模型的五种不同形式进行了实证比较与分析。借助于新模型和比较结果,本文对中国短期利率的跳跃行为进行了实证研究。结果表明：（1）短期利率不仅存在均值回复和扩散行为,还存在显著的跳跃行为;（2）短期利率的跳跃强度存在显著的正向水平效应和宏观经济效应 ,但水平效应比宏观经济效应更显著;（3）跳跃行为、跳跃强度的水平效应以及宏观经济效应在刻画利率动态行为时都是必要的,现有的跳跃－扩散模型不足以描述中国短期利率的动态行为特征;（4）随着跳跃、跳跃强度的宏观经济效应和水平效应的逐步引入,模型的拟合优度和预测能力逐步显著提高。