Nested combination tests with a time‐to‐event endpoint using a short‐term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family‐wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short‐term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short‐term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.     

Stopping rules for long-term clinical trials based on two consecutive rejections of the null hypothesis

A strategy for stopping long-term randomized clinical trials with time-to-event as a primary outcome measure has been considered using the criteria requiring multiple consecutive (or non consecutive) rejections at a specified α-level that controls against elevation of type I error. The procedure using two consecutive rejections is presented in this work along with the corresponding α-levels for the interim tests. The boundary cutoff values for these interim levels were determined based on an overall prespecified test size and were calculated using multidimensional integration and/or simulations. The reduction in the interim α-level values that is required to maintain the experiment-wise error rate is found to be modest. The power of the test is evaluated under various alternative accrual and hazard patterns. This procedure provides a more realistic stopping rule in large multi-center trials where it may be undesirable to terminate a trial unless a sustained effect has been demonstrated.     

Operational challenges in adaptive design implementation

Implementation of adaptive clinical trial designs raises challenges with regard to the processes by which accruing trial data is analyzed, reviewed, and acted upon. In line with current monitoring conventions, it should be viewed that inappropriate knowledge of interim results can raise concerns regarding maintaining trial integrity and interpretability of results. Here we discuss issues related to these processes in adaptive trials, and point out distinctions versus other more familiar monitoring situations. One topic involves the composition of the group of individuals who will have access to interim results in order to recommend adaptations. We discuss operational models for data review by this group; one question addressed is whether in adaptive trials a role in this process for a representative of the study sponsor could at times be warranted, and might be justified if adequate protections are in place. Another issue involves whether adaptations made based upon interim data can convey to observers an amount of information about the results, which could rise to a level of concern. We consider whether different types of adaptations might be more or less problematic with regard to this issue, and recommend steps that might be considered to mitigate this concern.     

Improving interim decisions in randomized trials by exploiting information on short‐term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short‐term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short‐term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long‐term endpoint which enable the incorporation of baseline covariates and multiple short‐term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re‐assessment based on the inverse normal P‐value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.     

An approach to use of an adaptive procedure to clinical trials for molecularly heterogenous subject selection at interim

Abstract

For clinical trials, molecular heterogeneity has played a more important role recently. Many novel clinical trial designs prospectively incorporate molecular information to evaluation of treatment effects. In this paper, an adaptive procedure incorporating a non-pre-specified genomic biomarker is employed in the interim of a conventional trial. A non-pre-specified binary genomic biomarker, which is predictive of treatment effect, is used to classify study patients into two mutually exclusive subgroups at the interim review. According to the observations at the interim stage, adaptations such as adjusting sample size or shifting eligibility of study patients are then made in case of different scenarios.     

Gatekeeping procedures in dose-response clinical trials based on the Dunnett test

This paper discusses multiple testing procedures in dose-response clinical trials with primary and secondary endpoints. A general gatekeeping framework for constructing multiple tests is proposed, which extends the Dunnett test [Journal of the American Statistical Association 1955; 50: 1096-1121] and Bonferroni-based gatekeeping tests developed by Dmitrienko et al. [Statistics in Medicine 2003; 22:2387-2400]. The proposed procedure accounts for the hierarchical structure of the testing problem; for example, it restricts testing of secondary endpoints to the doses for which the primary endpoint is significant. The multiple testing approach is illustrated using a dose-response clinical trial in patients with diabetes. Monte-Carlo simulations demonstrate that the proposed procedure provides a power advantage over the Bonferroni gatekeeping procedure. The power gain generally increases with increasing correlation among the endpoints, especially when all primary dose-control comparisons are significant.     

Exploring changes in treatment effects across design stages in adaptive trials

The recently published Committee for Medicinal Products for Human Use reflection paper on flexible designs highlights a controversial issue regarding the interpretation of adaptive trials. The guideline suggests that a test for heterogeneity should be preplanned and if treatment effect estimates differ significantly between design stages then data collected before and after the interim analysis might not be combined in a formal analysis. In this paper we investigate error rates for such a procedure in the presence of calendar-time effects. Furthermore, we present an alternative testing strategy based on change point methods. In a simulation study we demonstrate that our procedure performs well in comparison to that suggested by the guideline.     

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim

Interest in confirmatory adaptive combined phase II/III studies with treatment selection has increased in the past few years. These studies start comparing several treatments with a control. One (or more) treatment(s) is then selected after the first stage based on the available information at an interim analysis, including interim data from the ongoing trial, external information and expert knowledge. Recruitment continues, but now only for the selected treatment(s) and the control, possibly in combination with a sample size reassessment. The final analysis of the selected treatment(s) includes the patients from both stages and is performed such that the overall Type I error rate is strictly controlled, thus providing confirmatory evidence of efficacy at the final analysis. In this paper we describe two approaches to control the Type I error rate in adaptive designs with sample size reassessment and/or treatment selection. The first method adjusts the critical value using a simulation-based approach, which incorporates the number of patients at an interim analysis, the true response rates, the treatment selection rule, etc. We discuss the underlying assumptions of simulation-based procedures and give several examples where the Type I error rate is not controlled if some of the assumptions are violated. The second method is an adaptive Bonferroni-Holm test procedure based on conditional error rates of the individual treatment-control comparisons. We show that this procedure controls the Type I error rate, even if a deviation from a pre-planned adaptation rule or the time point of such a decision is necessary.     

Bayesian adaptive bandit-based designs using the Gittins index for multi-armed trials with normally distributed endpoints

Adaptive designs for multi-armed clinical trials have become increasingly popular recently because of their potential to shorten development times and to increase patient response. However, developing response-adaptive designs that offer patient-benefit while ensuring the resulting trial provides a statistically rigorous and unbiased comparison of the different treatments included is highly challenging. In this paper, the theory of Multi-Armed Bandit Problems is used to define near optimal adaptive designs in the context of a clinical trial with a normally distributed endpoint with known variance. We report the operating characteristics (type I error, power, bias) and patient-benefit of these approaches and alternative designs using simulation studies based on an ongoing trial. These results are then compared to those recently published in the context of Bernoulli endpoints. Many limitations and advantages are similar in both cases but there are also important differences, specially with respect to type I error control. This paper proposes a simulation-based testing procedure to correct for the observed type I error inflation that bandit-based and adaptive rules can induce.     

MMCTest—A Safe Algorithm for Implementing Multiple Monte Carlo Tests

Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest , a sequential algorithm that gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal p‐values. The method can be used with a class of multiple testing procedures that include the Benjamini and Hochberg false discovery rate procedure and the Bonferroni correction controlling the familywise error rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses that can already be decided as being rejected or non‐rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non‐rejected and the undecided hypotheses. A simulation study motivated by actual biological data shows that MMCTest is usable in practice and that, despite the additional guarantee, it can be computationally more efficient than other methods.     

Bayesian hierarchical models for adaptive basket trial designs

Basket trials evaluate a single drug targeting a single genetic variant in multiple cancer cohorts. Empirical findings suggest that treatment efficacy across baskets may be heterogeneous. Most modern basket trial designs use Bayesian methods. These methods require the prior specification of at least one parameter that permits information sharing across baskets. In this study, we provide recommendations for selecting a prior for scale parameters for adaptive basket trials by using Bayesian hierarchical modeling. Heterogeneity among baskets attracts much attention in basket trial research, and substantial heterogeneity challenges the basic assumption of exchangeability of Bayesian hierarchical approach. Thus, we also allowed each stratum-specific parameter to be exchangeable or nonexchangeable with similar strata by using data observed in an interim analysis. Through a simulation study, we evaluated the overall performance of our design based on statistical power and type I error rates. Our research contributes to the understanding of the properties of Bayesian basket trial designs.     

Testing bioequivalence for multiple formulations with power and sample size calculations

Bioequivalence (BE) trials play an important role in drug development for demonstrating the BE between test and reference formulations. The key statistical analysis for BE trials is the use of two one‐sided tests (TOST), which is equivalent to showing that the 90% confidence interval of the relative bioavailability is within a given range. Power and sample size calculations for the comparison between one test formulation and the reference formulation has been intensively investigated, and tables and software are available for practical use. From a statistical and logistical perspective, it might be more efficient to test more than one formulation in a single trial. However, approaches for controlling the overall type I error may be required. We propose a method called multiplicity‐adjusted TOST (MATOST) combining multiple comparison adjustment approaches, such as Hochberg's or Dunnett's method, with TOST. Because power and sample size calculations become more complex and are difficult to solve analytically, efficient simulation‐based procedures for this purpose have been developed and implemented in an R package. Some numerical results for a range of scenarios are presented in the paper. We show that given the same overall type I error and power, a BE crossover trial designed to test multiple formulations simultaneously only requires a small increase in the total sample size compared with a simple 2 × 2 crossover design evaluating only one test formulation. Hence, we conclude that testing multiple formulations in a single study is generally an efficient approach. The R package MATOST is available at https://sites.google.com/site/matostbe/ . Copyright © 2012 John Wiley & Sons, Ltd.     

Analyzing multiple end points with a two-stage design in clinical trials

In many clinical trials, the assessment of the response to interventions can include a large variety of outcome variables which are generally correlated. The use of multiple significance tests is likely to increase the chance of detecting a difference in at least one of the outcomes between two treatments. Furthermore, univariate tests do not take into account the correlation structure. A new test is proposed that uses information from the interim analysis in a two-stage design to form the rejection region boundaries at the second stage. Initially, the test uses Hotelling’s T² at the end of the first stage allowing only, for early acceptance of the null hypothesis and an O’Brien ‘type’ procedure at the end of the second stage. This test allows one to ‘cheat’ and look at the data at the interim analysis to form rejection regions at the second stage, provided one uses the correct distribution of the final test statistic. This distribution is derived and the power of the new test is compared to the power of three common procedures for testing multiple outcomes: Bonferroni’s inequality, Hotelling’s T²and O’Brien’s test. O’Brien’s test has the best power to detect a difference when the outcomes are thought to be affected in exactly the same direction and the same magnitude or in exactly the same relative effects as those proposed prior to data collection. However, the statistic is not robust to deviations in the alternative parameters proposed a priori, especially for correlated outcomes. The proposed new statistic and the derivation of its distribution allows investigators to consider information from the first stage of a two-stage design and consequently base the final test on the direction observed at the first stage or modify the statistic if the direction differs significantly from what was expected a prior.     

Testing multiple primary endpoints in clinical trials with sample size adaptation

In this paper, we propose a design that uses a short‐term endpoint for accelerated approval at interim analysis and a long‐term endpoint for full approval at final analysis with sample size adaptation based on the long‐term endpoint. Two sample size adaptation rules are compared: an adaptation rule to maintain the conditional power at a prespecified level and a step function type adaptation rule to better address the bias issue. Three testing procedures are proposed: alpha splitting between the two endpoints; alpha exhaustive between the endpoints; and alpha exhaustive with improved critical value based on correlation. Family‐wise error rate is proved to be strongly controlled for the two endpoints, sample size adaptation, and two analysis time points with the proposed designs. We show that using alpha exhaustive designs greatly improve the power when both endpoints are effective, and the power difference between the two adaptation rules is minimal. The proposed design can be extended to more general settings. Copyright © 2015 John Wiley & Sons, Ltd.     

IDENTIFYING TREATMENT EFFECTS IN MULTI-CHANNEL MEASUREMENTS IN ELECTROENCEPHALOGRAPHIC STUDIES: MULTIVARIATE PERMUTATION TESTS AND MULTIPLE COMPARISONS

A versatile procedure is described comprising an application of statistical techniques to the analysis of the large, multi‐dimensional data arrays produced by electroencephalographic (EEG) measurements of human brain function. Previous analytical methods have been unable to identify objectively the precise times at which statistically significant experimental effects occur, owing to the large number of variables (electrodes) and small number of subjects, or have been restricted to two‐treatment experimental designs. Many time‐points are sampled in each experimental trial, making adjustment for multiple comparisons mandatory. Given the typically large number of comparisons and the clear dependence structure among time‐points, simple Bonferroni‐type adjustments are far too conservative. A three‐step approach is proposed: (i) summing univariate statistics across variables; (ii) using permutation tests for treatment effects at each time‐point; and (iii) adjusting for multiple comparisons using permutation distributions to control family‐wise error across the whole set of time‐points. Our approach provides an exact test of the individual hypotheses while asymptotically controlling family‐wise error in the strong sense, and can provide tests of interaction and main effects in factorial designs. An application to two experimental data sets from EEG studies is described, but the approach has application to the analysis of spatio‐temporal multivariate data gathered in many other contexts.     

Characterization of dose‐response for count data using a generalized MCP‐Mod approach in an adaptive dose‐ranging trial

Understanding the dose–response relationship is a key objective in Phase II clinical development. Yet, designing a dose‐ranging trial is a challenging task, as it requires identifying the therapeutic window and the shape of the dose–response curve for a new drug on the basis of a limited number of doses. Adaptive designs have been proposed as a solution to improve both quality and efficiency of Phase II trials as they give the possibility to select the dose to be tested as the trial goes. In this article, we present a ‘shapebased’ two‐stage adaptive trial design where the doses to be tested in the second stage are determined based on the correlation observed between efficacy of the doses tested in the first stage and a set of pre‐specified candidate dose–response profiles. At the end of the trial, the data are analyzed using the generalized MCP‐Mod approach in order to account for model uncertainty. A simulation study shows that this approach gives more precise estimates of a desired target dose (e.g. ED70) than a single‐stage (fixed‐dose) design and performs as well as a two‐stage D‐optimal design. We present the results of an adaptive model‐based dose‐ranging trial in multiple sclerosis that motivated this research and was conducted using the presented methodology. Copyright © 2015 John Wiley & Sons, Ltd.     

Analyzing multiple endpoints in a confirmatory randomized clinical trial—an approach that addresses stratification,missing values,baseline imbalance and multiplicity for strictly ordinal outcomes

Confirmatory randomized clinical trials with a stratified design may have ordinal response outcomes, ie, either ordered categories or continuous determinations that are not compatible with an interval scale. Also, multiple endpoints are often collected when 1 single endpoint does not represent the overall efficacy of the treatment. In addition, random baseline imbalances and missing values can add another layer of difficulty in the analysis plan. Therefore, the development of an approach that provides a consolidated strategy to all issues collectively is essential. For a real case example that is from a clinical trial comparing a test treatment and a control for the pain management for patients with osteoarthritis, which has all aforementioned issues, multivariate Mann‐Whitney estimators with stratification adjustment are applicable to the strictly ordinal responses with stratified design. Randomization based nonparametric analysis of covariance is applied to account for the possible baseline imbalances. Several approaches that handle missing values are provided. A global test followed by a closed testing procedure controls the family wise error rate in the strong sense for the analysis of multiple endpoints. Four outcomes indicating joint pain, stiffness, and functional status were analyzed collectively and also individually through the procedures. Treatment efficacy was observed in the combined endpoint as well as in the individual endpoints. The proposed approach is effective in addressing the aforementioned problems simultaneously and straightforward to implement.     

Blinded continuous monitoring in clinical trials with recurrent event endpoints

In studies with recurrent event endpoints, misspecified assumptions of event rates or dispersion can lead to underpowered trials or overexposure of patients. Specification of overdispersion is often a particular problem as it is usually not reported in clinical trial publications. Changing event rates over the years have been described for some diseases, adding to the uncertainty in planning. To mitigate the risks of inadequate sample sizes, internal pilot study designs have been proposed with a preference for blinded sample size reestimation procedures, as they generally do not affect the type I error rate and maintain trial integrity. Blinded sample size reestimation procedures are available for trials with recurrent events as endpoints. However, the variance in the reestimated sample size can be considerable in particular with early sample size reviews. Motivated by a randomized controlled trial in paediatric multiple sclerosis, a rare neurological condition in children, we apply the concept of blinded continuous monitoring of information, which is known to reduce the variance in the resulting sample size. Assuming negative binomial distributions for the counts of recurrent relapses, we derive information criteria and propose blinded continuous monitoring procedures. The operating characteristics of these are assessed in Monte Carlo trial simulations demonstrating favourable properties with regard to type I error rate, power, and stopping time, ie, sample size.     

Robust inference from multiple test statistics via permutations: a better alternative to the single test statistic approach for randomized trials

Formal inference in randomized clinical trials is based on controlling the type I error rate associated with a single pre‐specified statistic. The deficiency of using just one method of analysis is that it depends on assumptions that may not be met. For robust inference, we propose pre‐specifying multiple test statistics and relying on the minimum p‐value for testing the null hypothesis of no treatment effect. The null hypothesis associated with the various test statistics is that the treatment groups are indistinguishable. The critical value for hypothesis testing comes from permutation distributions. Rejection of the null hypothesis when the smallest p‐value is less than the critical value controls the type I error rate at its designated value. Even if one of the candidate test statistics has low power, the adverse effect on the power of the minimum p‐value statistic is not much. Its use is illustrated with examples. We conclude that it is better to rely on the minimum p‐value rather than a single statistic particularly when that single statistic is the logrank test, because of the cost and complexity of many survival trials. Copyright © 2013 John Wiley & Sons, Ltd.