Weighted Multiple Testing Correction for Correlated Binary Endpoints

Multiple binary endpoints often occur in clinical trials and are usually correlated. Many multiple testing adjustment methods have been proposed to control familywise type I error rates. However, most of them disregard the correlation among the endpoints, for example, the commonly used Bonferroni correction, Bonferroni fixed-sequence (BFS) procedure, and its extension, the alpha-exhaustive fallback (AEF). Extending BFS by taking into account correlations among endpoints, Huque and Alosh proposed a flexible fixed-sequence (FFS) testing method, but this FFS method faces computational difficulty when there are four or more endpoints and the power of the first hypothesis does not depend on the correlations among endpoints. In dealing with these issues, Xie proposed a weighted multiple testing correction (WMTC) for correlated continuous endpoints and showed that the proposed method can easily handle hundreds of endpoints by using the R package and has higher power for testing the first hypothesis compared with the FFS and AEF methods. Since WMTC depends on the joint distribution of the endpoints, it is not clear whether WMTC still keeps those advantages when correlated binary endpoints are used. In this article, we evaluated the statistical power of WMTC method for correlated binary endpoints in comparison with the FFS, the AEF, the prospective alpha allocation scheme (PAAS), and the weighted Holm-Bonferroni methods. Furthermore the WMTC method and others are illustrated on a real dataset examining the circumstance of homicide in New York City.     

Confidence intervals and statistical power of the ‘Validation’ ratio for surrogate or intermediate endpoints

There is currently much interest in the use of surrogate endpoints in clinical trials and intermediate endpoints in epidemiology. Freedman et al. [Statist. Med. 11 (1992) 167] proposed the use of a validation ratio for judging the evidence of the validity of a surrogate endpoint. The method involves calculation of a confidence interval for the ratio. In this paper, I compare through computer simulations the performance of Fieller's method with the delta method for this calculation. In typical situations, the numerator and denominator of the ratio are highly correlated. I find that the Fieller method is superior to the delta method in coverage properties and in statistical power of the validation test. In addition, the formula for predicting statistical power seems to be much more accurate for the Fieller method than for the delta method. The simulations show that the role of validation analysis is likely to be limited in evaluating the reliability of using surrogate endpoints in clinical trials; however, it is likely to be a useful tool in epidemiology for identifying intermediate endpoints.     

A flexible fixed-sequence testing method for hierarchically ordered correlated multiple endpoints in clinical trials

Statistical approaches for addressing multiplicity in clinical trials range from the very conservative (the Bonferroni method) to the least conservative the fixed sequence approach. Recently, several authors proposed methods that combine merits of the two extreme approaches. Wiens [2003. A fixed sequence Bonferroni procedure for testing multiple endpoints. Pharmaceutical Statist. 2003, 2, 211–215], for example, considered an extension of the Bonferroni approach where the type I error rate (α)$(\alpha )$ is allocated among the endpoints, however, testing proceeds in a pre-determined order allowing the type I error rate to be saved for later use as long as the null hypotheses are rejected. This leads to a higher power of the test in testing later null hypotheses. In this paper, we consider an extension of Wiens’ approach by taking into account correlations among endpoints for achieving higher flexibility in testing. We show strong control of the family-wise type I error rate for this extension and provide critical values and significance levels for testing up to three endpoints with equal correlations and show how to calculate them for other correlation structures. We also present results of a simulation experiment for comparing the power of the proposed method with those of Wiens’ and others. The results of this experiment show that the magnitude of the gain in power of the proposed method depends on the prospective ordering of testing of the endpoints, the magnitude of the treatment effects of the endpoints and the magnitude of correlation between endpoints. Finally, we consider applications of the proposed method for clinical trials with multiple time points and multiple doses, where correlations among endpoints frequently arise.     

Comparison of Selected Methods for Modeling of Multi-State Disease Progression Processes: A Simulation Study

Prognostic studies are essential to understand the role of particular prognostic factors and, thus, improve prognosis. In most studies, disease progression trajectories of individual patients may end up with one of mutually exclusive endpoints or can involve a sequence of different events.

One challenge in such studies concerns separating the effects of putative prognostic factors on these different endpoints and testing the differences between these effects.

In this article, we systematically evaluate and compare, through simulations, the performance of three alternative multivariable regression approaches in analyzing competing risks and multiple-event longitudinal data. The three approaches are: (1) fitting separate event-specific Cox's proportional hazards models; (2) the extension of Cox's model to competing risks proposed by Lunn and McNeil; and (3) Markov multi-state model.

The simulation design is based on a prognostic study of cancer progression, and several simulated scenarios help investigate different methodological issues relevant to the modeling of multiple-event processes of disease progression. The results highlight some practically important issues. Specifically, the decreased precision of the observed timing of intermediary (non fatal) events has a strong negative impact on the accuracy of regression coefficients estimated with either the Cox's or Lunn-McNeil models, while the Markov model appears to be quite robust, under the same circumstances. Furthermore, the tests based on both Markov and Lunn-McNeil models had similar power for detecting a difference between the effects of the same covariate on the hazards of two mutually exclusive events. The Markov approach yields also accurate Type I error rate and good empirical power for testing the hypothesis that the effect of a prognostic factor on changes after an intermediary event, which cannot be directly tested with the Lunn-McNeil method. Bootstrap-based standard errors improve the coverage rates for Markov model estimates. Overall, the results of our simulations validate Markov multi-state model for a wide range of data structures encountered in prognostic studies of disease progression, and may guide end users regarding the choice of model(s) most appropriate for their specific application.     

Testing equality of generalized variances of k multivariate normal populations

Generalized variance is a measure of dispersion of multivariate data. Comparison of dispersion of multivariate data is one of the favorite issues for multivariate quality control, generalized homogeneity of multidimensional scatter, etc. In this article, the problem of testing equality of generalized variances of k multivariate normal populations by using the Bartlett's modified likelihood ratio test (BMLRT) is proposed. Simulations to compare the Type I error rate and power of the BMLRT and the likelihood ratio test (LRT) methods are performed. These simulations show that the BMLRT method has a better chi-square approximation under the null hypothesis. Finally, a practical example is given.     

Statistical analysis of non-inferiority via non-zero risk difference in stratified matched-pair studies

A stratified study is often designed for adjusting several independent trials in modern medical research. We consider the problem of non-inferiority tests and sample size determinations for a nonzero risk difference in stratified matched-pair studies, and develop the likelihood ratio and Wald-type weighted statistics for testing a null hypothesis of non-zero risk difference for each stratum in stratified matched-pair studies on the basis of (1) the sample-based method and (2) the constrained maximum likelihood estimation (CMLE) method. Sample size formulae for the above proposed statistics are derived, and several choices of weights for Wald-type weighted statistics are considered. We evaluate the performance of the proposed tests according to type I error rates and empirical powers via simulation studies. Empirical results show that (1) the likelihood ratio and the Wald-type CMLE test based on harmonic means of the stratum-specific sample size (SSIZE) weight (the Cochran's test) behave satisfactorily in the sense that their significance levels are much closer to the prespecified nominal level; (2) the likelihood ratio test is better than Nam's [2006. Non-inferiority of new procedure to standard procedure in stratified matched-pair design. Biometrical J. 48, 966–977] score test; (3) the sample sizes obtained by using SSIZE weight are smaller than other weighted statistics in general; (4) the Cochran's test statistic is generally much better than other weighted statistics with CMLE method. A real example from a clinical laboratory study is used to illustrate the proposed methodologies.     

On optimal tests for general interval-hypotheses

Optimal statistical tests, using the normality assumptions for general interval hypotheses including equivalence testing and testing for nonzero difference (or for non-unit) are presented. These tests are based on the decision theory for Polya Type distributions and are compared with usual confidence tests and with ’two one-sided tests’- procedures. A formal relationship between some optimal tests and the Anderson and Hauck procedure as well as a procedure recommended by Patel and Gupta is given. A new procedure for a generalisation of Student's test as well as for equivalence testing for thet-statistics is shown.     

Sequential parallel comparison design with two coprimary endpoints

A placebo‐controlled randomized clinical trial is required to demonstrate that an experimental treatment is superior to its corresponding placebo on multiple coprimary endpoints. This is particularly true in the field of neurology. In fact, clinical trials for neurological disorders need to show the superiority of an experimental treatment over a placebo in two coprimary endpoints. Unfortunately, these trials often fail to detect a true treatment effect for the experimental treatment versus the placebo owing to an unexpectedly high placebo response rate. Sequential parallel comparison design (SPCD) can be used to address this problem. However, the SPCD has not yet been discussed in relation to clinical trials with coprimary endpoints. In this article, our aim was to develop a hypothesis‐testing method and a method for calculating the corresponding sample size for the SPCD with two coprimary endpoints. In a simulation, we show that the proposed hypothesis‐testing method achieves the nominal type I error rate and power and that the proposed sample size calculation method has adequate power accuracy. In addition, the usefulness of our methods is confirmed by returning to an SPCD trial with a single primary endpoint of Alzheimer disease‐related agitation.     

Confidence intervals for endpoints of a c.d.f. via bootstrap

We propose methods of constructing confidence intervals for endpoints of a distribution. Under a mild condition on the tail of distribution, asymptotically correct confidence intervals are derived by bootstrapping Weissman's (Comm. Statist. Theory Method A 10 (1981) 549–557) statistics. It is also shown that a modification of this method works for type II censored data.     

Evaluation of a weighted multiple comparison procedure

We consider the problem of accounting for multiplicity for two correlated endpoints in the comparison of two treatments using weighted hypothesis tests. Various weighted testing procedures are reviewed, and a more powerful method (a variant of the weighted Simes test) is evaluated for the general bivariate normal case and for a particular clinical trial example. Results from these evaluations are summarized and indicate that the weighted methods perform in a manner similar to unweighted methods. Copyright © 2005 John Wiley & Sons, Ltd.     

A bootstrap method for comparing correlated kappa coefficients

Cohen's kappa coefficient is traditionally used to quantify the degree of agreement between two raters on a nominal scale. Correlated kappas occur in many settings (e.g., repeated agreement by raters on the same individuals, concordance between diagnostic tests and a gold standard) and often need to be compared. While different techniques are now available to model correlated κ coefficients, they are generally not easy to implement in practice. The present paper describes a simple alternative method based on the bootstrap for comparing correlated kappa coefficients. The method is illustrated by examples and its type I error studied using simulations. The method is also compared with the generalized estimating equations of the second order and the weighted least-squares methods.     

A fixed sequence Bonferroni procedure for testing multiple endpoints

A method for controlling the familywise error rate combining the Bonferroni adjustment and fixed testing sequence procedures is proposed. This procedure allots Type I error like the Bonferroni adjustment, but allows the Type I error to accumulate whenever a null hypothesis is rejected. In this manner, power for hypotheses tested later in a prespecified order will be increased. The order of the hypothesis tests needs to be prespecified as in a fixed sequence testing procedure, but unlike the fixed sequence testing procedure all hypotheses can always be tested, allowing for an a priori method of concluding a difference in the various endpoints. An application will be in clinical trials in which mortality is a concern, but it is expected that power to distinguish a difference in mortality will be low. If the effect on mortality is larger than anticipated, this method allows a test with a prespecified method of controlling the Type I error rate. Copyright © 2003 John Wiley & Sons, Ltd.     

Application of Anbar's Approach to Hypothesis Testing to Detect the Difference between Two Proportions

In this paper, Anbar's (1983) approach for estimating a difference between two binomial proportions is discussed with respect to a hypothesis testing problem. Such an approach results in two possible testing strategies. While the results of the tests are expected to agree for a large sample size when two proportions are equal, the tests are shown to perform quite differently in terms of their probabilities of a Type I error for selected sample sizes. Moreover, the tests can lead to different conclusions, which is illustrated via a simple example; and the probability of such cases can be relatively large. In an attempt to improve the tests while preserving their relative simplicity feature, a modified test is proposed. The performance of this test and a conventional test based on normal approximation is assessed. It is shown that the modified Anbar's test better controls the probability of a Type I error for moderate sample sizes.     

Results concerning the generalized partially linear single-index model

The central topic of this article is the estimation of parameters of the generalized partially linear single-index model (GPLSIM). Two numerical optimization procedures are presented and an S-plus program based on these procedures is compared to a program by Wand in a simulation setting. The results from these simulations indicate that the estimates for the new procedures are as good, if not better, than Wand's. Also, this program is much more flexible than Wand's since it can handle more general models. Other simulations are also conducted. The first compares the effects of using linear interpolation versus spline interpolation in an optimization procedure. The results indicate that by using spline interpolation one gets more stable estimates at a cost of increased computational time. A second simulation was conducted to assess the performance of a method for estimating the variance of alpha. A third set of simulations is carried out to determine the best criterion for testing that one of the elements of alpha is equal to zero. The GPLSIM is applied to a water quality data set and the results indicate an interesting relationship between gastrointestinal illness and turbidity (cloudiness) of drinking water.     

A new generalized p-value approach for testing equality of coefficients of variation in k normal populations

A new generalized p-value method is proposed for testing the equality of coefficients of variation in k normal populations. Simulation studies show that the type I error probabilities are close to the nominal level. The proposed test is also compared with likelihood ratio test, modified Bennett's test and score test through Monte Carlo simulation, the results demonstrate that the generalized p-value method has satisfactory performance in terms of sizes and powers.     

A robust test for testing the correlation coefficient

A test based on Tiku's MML (modified maximum likelihood) estimators is developed for testing that the population correlation coefficient is zero. The test is compared with various other tests and shown to have good Type I error robustness and power for numerous symmetric and skew bivariate populations.     

Tables for pairwise multiple comparisons using shaffer's modifiedsequentially-rejective procedure

A sequentially rejective (SR) testing procedure introduced by Holm (1979) and modified (MSR) by Shaffer (1986) is considered for testing all pairwise mean comparisons.For such comparisons, both the SR and MSR methods require that the observed test statistics be ordered and compared, each in turn, to appropriate percentiles on Student's t distribution.For the MSR method these percentiles are based on the maximum number of true null hypotheses remaining at each stage of the sequential procedure, given prior significance at previous stages, A function is developed for determining this number from the number of means being tested and the stage of the test.For a test of all pairwise comparisons, the logical implications which follow the rejection of a null hypothesis renders the MSR procedure uniformly more powerful than the SR procedure.Tables of percentiles for comparing K means, 3 < K < 6, using the MSR method are presented.These tables use Sidak's (1967) multiplicative inequality and simplify the use of t he MSR procedure.Several modifications to the MSR are suggested as a means of further increasing the power for testing the pairwise comparisons.General use of the MSR and the corresponding function for testing other parameters besides the mean is discussed.     

Equating Poisson and Normal Probability Functions to Derive Stirling's Formula

Often texts for introductory courses in probability or mathematical statistics make reference to Stirling's asymptotic formula (for a factorial) without presenting any proof or justification for the formula. A notable exception is Feller. In this article, we present a derivation of Stirling's formula based on the normal approximation of a Poisson probability that is considerably more accessible to the average student than Feller's approach. Besides illustrating a usage of the central limit theorem in conjunction with a continuity correction, the derivation lends itself to a mnemonic device for quickly obtaining Stirling's formula.