Testing multiple primary endpoints in clinical trials with sample size adaptation

In this paper, we propose a design that uses a short‐term endpoint for accelerated approval at interim analysis and a long‐term endpoint for full approval at final analysis with sample size adaptation based on the long‐term endpoint. Two sample size adaptation rules are compared: an adaptation rule to maintain the conditional power at a prespecified level and a step function type adaptation rule to better address the bias issue. Three testing procedures are proposed: alpha splitting between the two endpoints; alpha exhaustive between the endpoints; and alpha exhaustive with improved critical value based on correlation. Family‐wise error rate is proved to be strongly controlled for the two endpoints, sample size adaptation, and two analysis time points with the proposed designs. We show that using alpha exhaustive designs greatly improve the power when both endpoints are effective, and the power difference between the two adaptation rules is minimal. The proposed design can be extended to more general settings. Copyright © 2015 John Wiley & Sons, Ltd.     

Post-hoc analyses in multiple regression based on prediction error

A well-known problem in multiple regression is that it is possible to reject the hypothesis that all slope parameters are equal to zero, yet when applying the usual Student's T-test to the individual parameters, no significant differences are found. An alternative strategy is to estimate prediction error via the 0.632 bootstrap method for all models of interest and declare the parameters associated with the model that yields the smallest prediction error to differ from zero. The main results in this paper are that this latter strategy can have practical value versus Student's T; replacing squared error with absolute error can be beneficial in some situations and replacing least squares with an extension of the Theil-Sen estimator can substantially increase the probability of identifying the correct model under circumstances that are described.     

A generalized step-up-down multiple test procedure

A generalization of step-up and step-down multiple test procedures is proposed. This step-up-down procedure is useful when the objective is to reject a specified minimum number, q, out of a family of k hypotheses. If this basic objective is met at the first step, then it proceeds in a step-down manner to see if more than q hypotheses can be rejected. Otherwise it proceeds in a step-up manner to see if some number less than q hypotheses can be rejected. The usual step-down procedure is the special case where q = 1, and the usual step-up procedure is the special case where q = k. Analytical and numerical comparisons between the powers of the step-up-down procedures with different choices of q are made to see how these powers depend on the actual number of false hypotheses. Examples of application include comparing the efficacy of a treatment to a control for multiple endpoints and testing the sensitivity of a clinical trial for comparing the efficacy of a new treatment with a set of standard treatments.     

Power and stability comparisons of multiple testing procedures with false discovery rate control

High-throughput data analyses are widely used for examining differential gene expression, identifying single nucleotide polymorphisms, and detecting methylation loci. False discovery rate (FDR) has been considered a proper type I error rate to control for discovery-based high-throughput data analysis. Various multiple testing procedures have been proposed to control the FDR. The power and stability properties of some commonly used multiple testing procedures have not been extensively investigated yet, however. Simulation studies were conducted to compare power and stability properties of five widely used multiple testing procedures at different proportions of true discoveries for various sample sizes for both independent and dependent test statistics. Storey's two linear step-up procedures showed the best performance among all tested procedures considering FDR control, power, and variance of true discoveries. Leukaemia and ovarian cancer microarray studies were used to illustrate the power and stability characteristics of these five multiple testing procedures with FDR control.     

Simultaneous inference based on rank statistics in linear models

A class of simultaneous tests based on the aligned rank transform (ART) statistics is proposed for linear functions of parameters in linear models. The asymptotic distributions are derived. The stability of the finite sample behaviour of the sampling distribution of the ART technique is studied by comparing the simulated upper quantiles of its sampling distribution with those of the multivariate t-distribution. Simulation also shows that the tests based on ART have excellent small sample properties and because of their robustness perform better than the methods based on the least-squares estimates.     

Performance of some multiple testing procedures to compare three doses of a test drug and placebo

A study design with two or more doses of a test drug and placebo is frequently used in clinical drug development. Multiplicity issues arise when there are multiple comparisons between doses of test drug and placebo, and also when there are comparisons of doses with one another. An appropriate analysis strategy needs to be specified in advance to avoid spurious results through insufficient control of Type I error, as well as to avoid the loss of power due to excessively conservative adjustments for multiplicity. For evaluation of alternative strategies with possibly complex management of multiplicity, we compare the performance of several testing procedures through the simulated data that represent various patterns of treatment differences. The purpose is to identify which methods perform better or more robustly than the others and under what conditions. Copyright © 2005 John Wiley & Sons, Ltd.     

Evaluations of FWER-controlling methods in multiple hypothesis testing

Simultaneously testing a family of n null hypotheses can arise in many applications. A common problem in multiple hypothesis testing is to control Type-I error. The probability of at least one false rejection referred to as the familywise error rate (FWER) is one of the earliest error rate measures. Many FWER-controlling procedures have been proposed. The ability to control the FWER and achieve higher power is often used to evaluate the performance of a controlling procedure. However, when testing multiple hypotheses, FWER and power are not sufficient for evaluating controlling procedure’s performance. Furthermore, the performance of a controlling procedure is also governed by experimental parameters such as the number of hypotheses, sample size, the number of true null hypotheses and data structure. This paper evaluates, under various experimental settings, the performance of some FWER-controlling procedures in terms of five indices, the FWER, the false discovery rate, the false non-discovery rate, the sensitivity and the specificity. The results can provide guidance on how to select an appropriate FWER-controlling procedure to meet a study’s objective.     

Nested combination tests with a time‐to‐event endpoint using a short‐term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family‐wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short‐term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short‐term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.     

Adaptive and repeated cumulative meta‐analyses of safety data during a new drug development process

During a new drug development process, it is desirable to timely detect potential safety signals. For this purpose, repeated meta‐analyses may be performed sequentially on accumulating safety data. Moreover, if the amount of safety data from the originally planned program is not enough to ensure adequate power to test a specific hypothesis (e.g., the noninferiority hypothesis of an event of interest), the total sample size may be increased by adding new studies to the program. Without appropriate adjustment, it is well known that the type I error rate will be inflated because of repeated analyses and sample size adjustment. In this paper, we discuss potential issues associated with adaptive and repeated cumulative meta‐analyses of safety data conducted during a drug development process. We consider both frequentist and Bayesian approaches. A new drug development example is used to demonstrate the application of the methods. Copyright © 2015 John Wiley & Sons, Ltd.     

Adaptive designs for pivotal trials: discussion points from the PSI Adaptive Design Expert Group

The Committee for Medicinal Products for Human Use (CHMP) is currently preparing a guideline on 'methodological issues in confirmatory clinical trials with flexible design and analysis plan'. PSI (Statisticians in the Pharmaceutical Industry) sponsored a meeting of pharmaceutical statisticians with an interest in the area to share experiences and identify potential opportunities for adaptive designs in late-phase clinical drug development. This article outlines the issues raised, resulting discussions and consensus views reached. Adaptive designs have potential utility in late-phase clinical development. Sample size re-estimation seems to be valuable and widely accepted, but should be made independent of the observed treatment effect where possible. Where unblinding is necessary, careful consideration needs to be given to preserving the integrity of the trial. An area where adaptive designs can be particularly beneficial is to allow dose selection in pivotal trials via adding/dropping treatment arms; for example, combining phase II and III of the drug development program. The more adaptations made during a late-phase clinical trial, the less likely that the clinical trial would be considered as a confirmatory trial. In all cases it would be advisable to consult with regulatory agencies at the protocol design stage. All involved should remain open to scientifically valid opportunities to improve drug development.     

Distribution of the two‐sample t‐test statistic following blinded sample size re‐estimation

We consider the blinded sample size re‐estimation based on the simple one‐sample variance estimator at an interim analysis. We characterize the exact distribution of the standard two‐sample t‐test statistic at the final analysis. We describe a simulation algorithm for the evaluation of the probability of rejecting the null hypothesis at given treatment effect. We compare the blinded sample size re‐estimation method with two unblinded methods with respect to the empirical type I error, the empirical power, and the empirical distribution of the standard deviation estimator and final sample size. We characterize the type I error inflation across the range of standardized non‐inferiority margin for non‐inferiority trials, and derive the adjusted significance level to ensure type I error control for given sample size of the internal pilot study. We show that the adjusted significance level increases as the sample size of the internal pilot study increases. Copyright © 2016 John Wiley & Sons, Ltd.     

Adaptive designs for single‐arm phase II trials in oncology

Clinical phase II trials in oncology are conducted to determine whether the activity of a new anticancer treatment is promising enough to merit further investigation. Two‐stage designs are commonly used for this situation to allow for early termination. Designs proposed in the literature so far have the common drawback that the sample sizes for the two stages have to be specified in the protocol and have to be adhered to strictly during the course of the trial. As a consequence, designs that allow a higher extent of flexibility are desirable. In this article, we propose a new adaptive method that allows an arbitrary modification of the sample size of the second stage using the results of the interim analysis or external information while controlling the type I error rate. If the sample size is not changed during the trial, the proposed design shows very similar characteristics to the optimal two‐stage design proposed by Chang et al. (Biometrics 1987; 43:865–874). However, the new design allows the use of mid‐course information for the planning of the second stage, thus meeting practical requirements when performing clinical phase II trials in oncology. Copyright © 2012 John Wiley & Sons, Ltd.     

Randomized p-values for multiple testing of composite null hypotheses

We are considered with the problem of m simultaneous statistical test problems with composite null hypotheses. Usually, marginal p-values are computed under least favorable parameter configurations (LFCs), thus being over-conservative under non-LFCs. Our proposed randomized p-value leads to a tighter exhaustion of the marginal (local) significance level. In turn, it is stochastically larger than the LFC-based p-value under alternatives. While these distributional properties are typically nonsensical for m  =1, the exhaustion of the local significance level is extremely helpful for cases with m>1$m>1$ in connection with data-adaptive multiple tests as we will demonstrate by considering multiple one-sided tests for Gaussian means.     

Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures

Many multiple testing procedures (MTPs) are available today, and their number is growing. Also available are many type I error rates: the family-wise error rate (FWER), the false discovery rate, the proportion of false positives, and others. Most MTPs are designed to control a specific type I error rate, and it is hard to compare different procedures. We approach the problem by studying the exact level at which threshold step-down (TSD) procedures (an important class of MTPs exemplified by the classic Holm procedure) control the generalized FWER   defined as the probability of k$k$ or more false rejections. We find that level explicitly for any TSD procedure and any k$k$. No assumptions are made about the dependency structure of the p$p$-values of the individual tests. We derive from our formula a criterion for unimprovability   of a procedure in the class of TSD procedures controlling the generalized FWER at a given level. In turn, this criterion implies that for each k$k$ the number of such unimprovable procedures is finite and is greater than one if k>1$k>1$. Consequently, in this case the most rejective procedure in the above class does not exist.     

START: single‐to‐double arm transition design for phase II clinical trials

Phase II clinical trials designed for evaluating a drug's treatment effect can be either single‐arm or double‐arm. A single‐arm design tests the null hypothesis that the response rate of a new drug is lower than a fixed threshold, whereas a double‐arm scheme takes a more objective comparison of the response rate between the new treatment and the standard of care through randomization. Although the randomized design is the gold standard for efficacy assessment, various situations may arise where a single‐arm pilot study prior to a randomized trial is necessary. To combine the single‐ and double‐arm phases and pool the information together for better decision making, we propose a Single‐To‐double ARm Transition design (START) with switching hypotheses tests, where the first stage compares the new drug's response rate with a minimum required level and imposes a continuation criterion, and the second stage utilizes randomization to determine the treatment's superiority. We develop a software package in R to calibrate the frequentist error rates and perform simulation studies to assess the trial characteristics. Finally, a metastatic pancreatic cancer trial is used for illustrating the decision rules under the proposed START design.     

Tolerance‐based error reduction in multi‐agent bioassays

Some degree of error is inevitable in multi‐agent bioassays regardless of design or measurement technology. Estimation error can be reduced post facto by exploiting the matrix partial ordering of the bioassay survival estimates. The standard method for this is order‐restricted regression (ORR). If the joint action of the bioassay agents admits a tolerance‐based interpretation, additional structure beyond matrix partial ordering is available, leading to a new method of error reduction. This tolerance‐based error reduction (TBER) procedure almost always outperforms ORR. Like ORR, TBER applies to complete factorial bioassay designs and, using weighting, to incomplete designs.     

Testing treatment‐by‐period interaction in four‐period crossover trials

Statistical analyses of crossover clinical trials have mainly focused on assessing the treatment effect, carryover effect, and period effect. When a treatment‐by‐period interaction is plausible, it is important to test such interaction first before making inferences on differences among individual treatments. Considerably less attention has been paid to the treatment‐by‐period interaction, which has historically been aliased with the carryover effect in two‐period or three‐period designs. In this article, from the data of a newly developed four‐period crossover design, we propose a statistical method to compare the effects of two active drugs with respect to two response variables. We study estimation and hypothesis testing considering the treatment‐by‐period interaction. Constrained least squares is used to estimate the treatment effect, period effect, and treatment‐by‐period interaction. For hypothesis testing, we extend a general multivariate method for analyzing the crossover design with multiple responses. Results from simulation studies have shown that this method performs very well. We also illustrate how to apply our method to the real data problem.