Efficient Stratified Testing Procedure for a False Discovery Rate

The false discovery rate (FDR) has become a popular error measure in the large-scale simultaneous testing. When data are collected from heterogenous sources and form grouped hypotheses testing, it may be beneficial to use the distinct feature of groups to conduct the multiple hypotheses testing. We propose a stratified testing procedure that uses different FDR levels according to the stratification features based on p-values. Our proposed method is easy to implement in practice. Simulations studies show that the proposed method produces more efficient testing results. The stratified testing procedure minimizes the overall false negative rate (FNR) level, while controlling the overall FDR. An example from a type II diabetes mice study further illustrates the practical advantages of this new approach.     

More Powerful Control of the False Discovery Rate Under Dependence

In a breakthrough paper, Benjamini and Hochberg (J Roy Stat Soc Ser B 57:289–300, 1995) proposed a new error measure for multiple testing, the FDR; and developed a distribution-free procedure to control it under independence among the test statistics. In this paper we argue by extensive simulation and theoretical considerations that the assumption of independence is not needed. Along the lines of (Ann Stat 32:1035–1061, 2004b), we moreover provide a more powerful method, that exploits an estimator of the number of false nulls among the tests. We propose a whole family of iterative estimators that prove robust under dependence and independence between the test statistics. These estimators can be used to improve also classical multiple testing procedures, and in general to estimate the weight of a known component in a mixture distribution. Innovations are illustrated by simulations.     

Bayesian False Discovery Rate Wavelet Shrinkage: Theory and Applications

Statistical inference in the wavelet domain remains a vibrant area of contemporary statistical research because of desirable properties of wavelet representations and the need of scientific community to process, explore, and summarize massive data sets. Prime examples are biomedical, geophysical, and internet related data. We propose two new approaches to wavelet shrinkage/thresholding.

In the spirit of Efron and Tibshirani's recent work on local false discovery rate, we propose Bayesian Local False Discovery Rate (BLFDR), where the underlying model on wavelet coefficients does not assume known variances. This approach to wavelet shrinkage is shown to be connected with shrinkage based on Bayes factors. The second proposal, Bayesian False Discovery Rate (BaFDR), is based on ordering of posterior probabilities of hypotheses on true wavelets coefficients being null, in Bayesian testing of multiple hypotheses.

We demonstrate that both approaches result in competitive shrinkage methods by contrasting them to some popular shrinkage techniques.     

Bayesian Robustness of the Positive False Discovery Rate

The positive false discovery rate was introduced by Storey (2003 Storey , J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value. Ann. Statist. 31:2013–2035.[Crossref], [Web of Science ®] , [Google Scholar]) as an alternative to the family wise error rate for the case in which we are simultaneously testing a large amount of hypotheses. The positive false discovery rate has a very nice Bayesian interpretation (as it was shown by Storey, 2003 Storey , J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value. Ann. Statist. 31:2013–2035.[Crossref], [Web of Science ®] , [Google Scholar]) and its robustness is analyzed. The emphasis is on the ε-contamination class (one of the most used classes of priors for Bayesian robustness) and it is shown that robustness is not obtained when the basic prior concentrates the probability on the null hypothesis.     

Optimal False Discovery Rate Control with Kernel Density Estimation in a Microarray Experiment

Most of current false discovery rate (FDR) procedures in a microarray experiment assume restrictive dependence structures, resulting in being less reliable. FDR controlling procedure under suitable dependence structures based on Poisson distributional approximation is shown. Unlike other procedures, the distribution of false null hypotheses is estimated by using kernel density estimation allowing for dependent structures among the genes. Furthermore, we develop an FDR framework that minimizes the false nondiscovery rate (FNR) with a constraint on the controlled level of the FDR. The performance of the proposed FDR procedure is compared with that of other existing FDR controlling procedures, with an application to the microarray study of simulated data.     

Positive False Discovery Rate Estimate in Step-Wise Variable Selection

Selecting predictors to optimize the outcome prediction is an important statistical method. However, it usually ignores the false positives in the selected predictors. In this article, we advocate a conventional stepwise forward variable selection method based on the predicted residual sum of squares, and develop a positive false discovery rate (pFDR) estimate for the selected predictor subset, and a local pFDR estimate to prioritize the selected predictors. This pFDR estimate takes account of the existence of non null predictors, and is proved to be asymptotically conservative. In addition, we propose two views of a variable selection process: an overall and an individual test. An interesting feature of the overall test is that its power of selecting non null predictors increases with the proportion of non null predictors among all candidate predictors. Data analysis is illustrated with an example, in which genetic and clinical predictors were selected to predict the cholesterol level change after four months of tamoxifen treatment, and pFDR was estimated. Our method's performance is evaluated through statistical simulations.     

A Comparison of Procedures for Controlling the False Discovery Rate in the Presence of Small Variance Genes: A Simulation Study

The Significance Analysis of Microarrays (SAM; Tusher et al., 2001 Tusher , V. G. , Tibshirani , R. , Chu , G. ( 2001 ). Significance analysis of microarrys applied to the ionizing radiation response . Proceedings of the National Academy of Sciences 98 : 5116 – 5121 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) method is widely used in analyzing gene expression data while controlling the FDR by using resampling-based procedure in the microarray setting. One of the main components of the SAM procedure is the adjustment of the test statistic. The introduction of the fudge factor to the test statistic aims at deflating the large value of test statistics due to the small standard error of gene-expression. Lin et al. (2008 Lin , D. , Shkedy , Z. , Burzykowski , T. , Göhlmann , H. W. H. , De Bondt , A. , Perera , T. , Geerts , T. , Bijnens , L. ( 2008 ). Significance analysis of microarray (SAM) for comparisons of several treatments with one control . Biometric Journal, MCP 50 ( 5 ): 801 – 823 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) pointed out that the fudge factor does not effectively improve the power and the control of the FDR as compared to the SAM procedure without the fudge factor in the presence of small variance genes. Motivated by the simulation results presented in Lin et al. (2008 Lin , D. , Shkedy , Z. , Burzykowski , T. , Göhlmann , H. W. H. , De Bondt , A. , Perera , T. , Geerts , T. , Bijnens , L. ( 2008 ). Significance analysis of microarray (SAM) for comparisons of several treatments with one control . Biometric Journal, MCP 50 ( 5 ): 801 – 823 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), in this article, we extend our study to compare several methods for choosing the fudge factor in the modified t-type test statistics and use simulation studies to investigate the power and the control of the FDR of the considered methods.     

Expected Power for the False Discovery Rate with Independence

The Benjamini–Hochberg procedure is widely used in multiple comparisons. Previous power results for this procedure have been based on simulations. This article produces theoretical expressions for expected power. To derive them, we make assumptions about the number of hypotheses being tested, which null hypotheses are true, which are false, and the distributions of the test statistics under each null and alternative. We use these assumptions to derive bounds for multiple dimensional rejection regions. With these bounds and a permanent based representation of the joint density function of the largest p-values, we use the law of total probability to derive the distribution of the total number of rejections. We derive the joint distribution of the total number of rejections and the number of rejections when the null hypothesis is true. We give an analytic expression for the expected power for a false discovery rate procedure that assumes the hypotheses are independent.     

A Bayesian False Discovery Rate for Multiple Testing

Case-control studies of genetic polymorphisms and gene-environment interactions are reporting large numbers of statistically significant associations, many of which are likely to be spurious. This problem reflects the low prior probability that any one null hypothesis is false, and the large number of test results reported for a given study. In a Bayesian approach to the low prior probabilities, Wacholder et al. (2004) suggest supplementing the p-value for a hypothesis with its posterior probability given the study data. In a frequentist approach to the test multiplicity problem, Benjamini & Hochberg (1995) propose a hypothesis-rejection rule that provides greater statistical power by controlling the false discovery rate rather than the family-wise error rate controlled by the Bonferroni correction. This paper defines a Bayes false discovery rate and proposes a Bayes-based rejection rule for controlling it. The method, which combines the Bayesian approach of Wacholder et al. with the frequentist approach of Benjamini & Hochberg, is used to evaluate the associations reported in a case-control study of breast cancer risk and genetic polymorphisms of genes involved in the repair of double-strand DNA breaks.     

A Process Monitoring Scheme Controlling False Discovery Rate

A new process monitoring scheme is proposed by using the Storey procedure for controlling the positive false discovery rate in multiple testing. For the 2-span control scheme, it is shown numerically that the proposed method performs better than X-bar chart in terms of the average run length. Some simulations are accomplished to evaluate the performance of the proposed scheme in terms of the average run length and the conditional expected delay. The results are compared with those of the existing monitoring schemes including the X-bar chart. The false discovery rate is also estimated and compared with the target control level.     

Some Results on the Control of the False Discovery Rate under Dependence

Abstract.  Controlling the false discovery rate (FDR) is a powerful approach to multiple testing, with procedures developed with applications in many areas. Dependence among the test statistics is a common problem, and many attempts have been made to extend the procedures. In this paper, we show that a certain degree of dependence is allowed among the test statistics, when the number of tests is large, with no need for any correction. We then suggest a way to conservatively estimate the proportion of false nulls, both under dependence and independence, and discuss the advantages of using such estimators when controlling the FDR.     

Bayesian decision theoretic approach to hypothesis problems with skewed alternatives

Many hypothesis problems in practice require the selection of the left side or the right side alternative when the null is rejected. For parametric models, this problem can be stated as _H0:θ=_θ0$H_0:\theta ={\theta }_0$vs.  _H−:θ<_θ0$H_{-}:\theta <{\theta }_0$ or _H+:θ>_θ0$H_{+}:\theta >{\theta }_0$. Frequentists use Type-III error (directional error) to develop statistical methodologies. This approach and other approaches considered in the literature do not take into account the situations where the selection of one side may be more important or when one side may be more probable than the other. This problem can be tackled by specifying a loss function and/or by specifying a hierarchical prior structure with allowing the skewness in the alternatives. Based on this, we develop a Bayesian decision theoretic methodology and show that the resulted Bayes rule perform better in the side of the alternatives which is more probable. The methodology can be also used in a frequentist's framework when it is desired to discover an alternative that is more important. We also consider the multiple hypotheses problem and develop new false discovery rates for the selection of the left and the right sides of alternatives. These discovery rates would be useful in the situations when one side of the alternatives are more important or more probable than the other.     

False Discovery Control for Multiple Tests of Association Under General Dependence

Abstract.  We propose a confidence envelope for false discovery control when testing multiple hypotheses of association simultaneously. The method is valid under arbitrary and unknown dependence between the test statistics and allows for an exploratory approach when choosing suitable rejection regions while still retaining strong control over the proportion of false discoveries.     

Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing

Abstract.  A new multiple testing procedure, the generalized augmentation procedure (GAUGE), is introduced. The procedure is shown to control the false discovery exceedance and to be competitive in terms of power. It is also shown how to apply the idea of GAUGE to achieve control of other error measures. Extensions to dependence are discussed, together with a modification valid under arbitrary dependence. We present an application to an original study on prostate cancer and on a benchmark data set on colon cancer.     

On False Discovery and Non‐discovery Proportions of the Dynamic Adaptive Procedure

This paper studies the asymptotic behaviour of the false discovery and non‐discovery proportions of the dynamic adaptive procedure under some dependence structure. A Bahadur‐type representation of the cut point in simultaneously performing a large scale of tests is presented. The asymptotic bias decompositions of the false discovery and non‐discovery proportions are given under some dependence structure. In addition to existing literatures, we find that the randomness due to the dynamic selection of the tuning parameter in estimating the true null rate serves as a source of the approximation error in the Bahadur representation and enters into the asymptotic bias term of the false discovery proportion and those of the false non‐discovery proportion. The theory explains to some extent why some seemingly attractive dynamic adaptive procedures do not outperform the competing fixed adaptive procedures substantially in some situations. Simulations justify our theory and findings.     

False Discovery Rate Control of Step‐Up‐Down Tests with Special Emphasis on the Asymptotically Optimal Rejection Curve

Abstract. This paper is concerned with exact control of the false discovery rate (FDR) for step‐up‐down (SUD) tests related to the asymptotically optimal rejection curve (AORC). Since the system of equations and/or constraints for critical values and FDRs is numerically extremely sensitive, existence and computation of valid solutions is a challenging problem. We derive explicit formulas for upper bounds of the FDR and show that under a well‐known monotonicity condition, control of the FDR by a step‐up procedure results in control of the FDR by a corresponding SUD procedure. Various methods for adjusting the AORC to achieve finite FDR control are investigated. Moreover, we introduce alternative FDR bounding curves and study their connection to rejection curves as well as the existence of critical values for exact FDR control with respect to the underlying FDR bounding curve. Finally, we propose an iterative method for the computation of critical values.     

A note on estimating the false discovery rate under mixture model

In this note, we focus on estimating the false discovery rate (FDR) of a multiple testing method with a common, non-random rejection threshold under a mixture model. We develop a new class of estimates of the FDR and prove that it is less conservatively biased than what is traditionally used. Numerical evidence is presented to show that the mean squared error (MSE) is also often smaller for the present class of estimates, especially in small-scale multiple testings. A similar class of estimates of the positive false discovery rate (pFDR) less conservatively biased than what is usually used is then proposed. When modified using our estimate of the pFDR and applied to a gene-expression data, Storey's q-value method identifies a few more significant genes than his original q-value method at certain thresholds. The BH like method developed by thresholding our estimate of the FDR is shown to control the FDR in situations where the p  -values have the same dependence structure as required by the BH method and, for lack of information about the proportion _π0${\pi }_0$ of true null hypotheses, it is reasonable to assume that _π0${\pi }_0$ is uniformly distributed over (0,1).     

A Convergence Result for a Class of Asymptotically Linear Estimators

This article provides the distribution of the last exit for strongly consistent estimators. Namely, we consider a small neighborhood of the (almost sure) limit and state the asymptotic distribution of the last time the estimator is outside this neighborhood. Such problems have been considered in the literature by various authors; this article extends these results in a semi-parametric frame. An application to adaptive estimation is provided.     

A new approach: Interrelated two-way clustering of gene expression data

The paper presents a new approach to interrelated two-way clustering of gene expression data. Clustering of genes has been effected using entropy and a correlation measure, whereas the samples have been clustered using the fuzzy C-means. The efficiency of this approach has been tested on two well known data sets: the colon cancer data set and the leukemia data set. Using this approach, we were able to identify the important co-regulated genes and group the samples efficiently at the same time.     

Bonferroni-type Plug-in Procedure Controlling Generalized Familywise Error Rate

Consider the multiple hypotheses testing problem controlling the generalized familywise error rate k-FWER, the probability of at least k false rejections. We propose a plug-in procedure based on the estimation of the number of true null hypotheses. Under the independence assumption of the p-values corresponding to the true null hypotheses, we first introduce the least favorable configuration (LFC) of k-FWER for Bonferroni-type plug-in procedure, then we construct a plug-in k-FWER-controlled procedure based on LFC. For dependent p-values, we establish the asymptotic k-FWER control under some mild conditions. Simulation studies suggest great improvement over generalized Bonferroni test and generalized Holm test.