Assessment of futility in clinical trials

The term 'futility' is used to refer to the inability of a clinical trial to achieve its objectives. In particular, stopping a clinical trial when the interim results suggest that it is unlikely to achieve statistical significance can save resources that could be used on more promising research. There are various approaches that have been proposed to assess futility, including stochastic curtailment, predictive power, predictive probability, and group sequential methods. In this paper, we describe and contrast these approaches, and discuss several issues associated with futility analyses, such as ethical considerations, whether or not type I error can or should be reclaimed, one-sided vs two-sided futility rules, and the impact of futility analyses on power.     

Random stopping sets in a sequential analysis of random measures and fields

A random stopping set is defined and some of its properties are proved. Namely, we prove the theorem on the absolute continuity of measures on a σ-algebra F_τ connected with a random stopping set τ, which can be applied to the sequential analysis of random measures and fields.     

Stopping guidelines vs stopping rules: a practitioner's point of view

Monitoring interim accumulating data in a clinical trial for evidence of therapeutic benefit or toxicity is a frequent policy, usually carried out by an independent scientific committee. While statistical methodology has been developed to assess the significance of these interim analyses, such methods should not be viewed as absolute rules but only serve as useful guides. The decision process to terminate a trial early is very complex and many factors must be taken into account. The complexity of this decision process is illustrated by reviewing the experience of several recent clinical trials.     

Increasing the sample size at interim for a two‐sample experiment without Type I error inflation

For the case of a one‐sample experiment with known variance σ²=1, it has been shown that at interim analysis the sample size (SS) may be increased by any arbitrary amount provided: (1) The conditional power (CP) at interim is ?50% and (2) there can be no decision to decrease the SS (stop the trial early). In this paper we verify this result for the case of a two‐sample experiment with proportional SS in the treatment groups and an arbitrary common variance. Numerous authors have presented the formula for the CP at interim for a two‐sample test with equal SS in the treatment groups and an arbitrary common variance, for both the one‐ and two‐sided hypothesis tests. In this paper we derive the corresponding formula for the case of unequal, but proportional SS in the treatment groups for both one‐sided superiority and two‐sided hypothesis tests. Finally, we present an SAS macro for doing this calculation and provide a worked out hypothetical example. In discussion we note that this type of trial design trades the ability to stop early (for lack of efficacy) for the elimination of the Type I error penalty. The loss of early stopping requires that such a design employs a data monitoring committee, blinding of the sponsor to the interim calculations, and pre‐planning of how much and under what conditions to increase the SS and that this all be formally written into an interim analysis plan before the start of the study. Copyright © 2009 John Wiley & Sons, Ltd.     

Decision‐making in a phase II clinical trial: a new approach combining Bayesian and frequentist concepts

The aim of a phase II clinical trial is to decide whether or not to develop an experimental therapy further through phase III clinical evaluation. In this paper, we present a Bayesian approach to the phase II trial, although we assume that subsequent phase III clinical trials will have standard frequentist analyses. The decision whether to conduct the phase III trial is based on the posterior predictive probability of a significant result being obtained. This fusion of Bayesian and frequentist techniques accepts the current paradigm for expressing objective evidence of therapeutic value, while optimizing the form of the phase II investigation that leads to it. By using prior information, we can assess whether a phase II study is needed at all, and how much or what sort of evidence is required. The proposed approach is illustrated by the design of a phase II clinical trial of a multi‐drug resistance modulator used in combination with standard chemotherapy in the treatment of metastatic breast cancer. Copyright © 2005 John Wiley & Sons, Ltd     

Choice of alpha spending function and time points in clinical trials with one or two interim analyses

One characterization of group sequential methods uses alpha spending functions to allocate the false positive rate throughout a study. We consider and evaluate several such spending functions as well as the time points of the interim analyses at which they apply. In addition, we evaluate the double triangular test as an alternative procedure that allows for early termination of the trial not only due to efficacy differences between treatments, but also due to lack of such differences. We motivate and illustrate our work by reference to the analysis of survival data from a proposed oncology study. Such group sequential procedures with one or two interim analyses are only slightly less powerful than fixed sample trials, but provide for the strong possibility of early stopping. Therefore, in all situations where they can practically be applied, we recommend their routine use in clinical trials. The double triangular test provides a suitable alternative to the group sequential procedures in that they do not provide for early stopping with acceptance of the null hypothesis. Again, there is only a modest loss in power relative to fixed sample tests. Copyright © 2004 John Wiley & Sons, Ltd.     

Applications of Bayesian statistical methodology to clinical trial design: A case study of a phase 2 trial with an interim futility assessment in patients with knee osteoarthritis

Development of new pharmacological treatments for osteoarthritis that address unmet medical needs in a competitive market place is challenging. Bayesian approaches to trial design offer advantages in defining treatment benefits by addressing clinically relevant magnitude of effects relative to comparators and in optimizing efficiency in analysis. Such advantages are illustrated by a motivating case study, a proof of concept, and dose finding study in patients with osteoarthritis. Patients with osteoarthritis were randomized to receive placebo, celecoxib, or 1 of 4 doses of galcanezumab. Primary outcome measure was change from baseline WOMAC pain after 8 weeks of treatment. Literature review of clinical trials with targeted comparator therapies quantified treatment effects versus placebo. Two success criteria were defined: one to address superiority to placebo with adequate precision and another to ensure a clinically relevant treatment effect. Trial simulations used a Bayesian dose response and longitudinal model. An interim analysis for futility was incorporated. Simulations indicated the study had ≥85% power to detect a 14‐mm improvement and ≤1% risk for a placebo‐like drug to pass. The addition of the second success criterion substantially reduced the risk of an inadequate, weakly efficacious drug proceeding to future development. The study was terminated at the interim analysis due to inadequate analgesic efficacy. A Bayesian approach using probabilistic statements enables clear understanding of success criteria, leading to informed decisions for study conduct. Incorporating an interim analysis can effectively reduce sample size, save resources, and minimize exposure of patients to an inadequate treatment.     

Information fraction estimation: Strategies for a phase 3 non-inferiority maximum duration design with time to event outcome

There is considerable debate surrounding the choice of methods to estimate information fraction for futility monitoring in a randomized non-inferiority maximum duration trial. This question was motivated by a pediatric oncology study that aimed to establish non-inferiority for two primary outcomes. While non-inferiority was determined for one outcome, the futility monitoring of the other outcome failed to stop the trial early, despite accumulating evidence of inferiority. For a one-sided trial design for which the intervention is inferior to the standard therapy, futility monitoring should provide the opportunity to terminate the trial early. Our research focuses on the Total Control Only (TCO) method, which is defined as a ratio of observed events to total events exclusively within the standard treatment regimen. We investigate its properties in stopping a trial early in favor of inferiority. Simulation results comparing the TCO method with alternative methods, one based on the assumption of an inferior treatment effect (TH0), and the other based on a specified hypothesis of a non-inferior treatment effect (THA), were provided under various pediatric oncology trial design settings. The TCO method is the only method that provides unbiased information fraction estimates regardless of the hypothesis assumptions and exhibits a good power and a comparable type I error rate at each interim analysis compared to other methods. Although none of the methods is uniformly superior on all criteria, the TCO method possesses favorable characteristics, making it a compelling choice for estimating the information fraction when the aim is to reduce cancer treatment-related adverse outcomes.     

Incorporating scientific,ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach

A flexible sequential approach to the design of clinical trials is discussed herein. This approach is based on a “confidence sequence” viewpoint instead of the rigid stopping and terminal decision rules in conventional sequential testing theory. By using an appropriate confidence sequence, one can always ensure a prescribed degree of scientific rigor (confidence) in establishing the drug to be effective. Moreover, one also has the option of terminating the trial early when there is already enough statistical evidence for concluding that the drug is effective, or when the drug shows uniorseen harmful effects, or when the data predict that there is little chance of arriving at a definitive conclusion in favor of the drug by the scheduled end of the trial. We discuss how these and other ethical and economic considerations can be readily incorporated into the stopping criteria of the trial.     

Strategies for a sequential selection of the better of two trinomial populations

A sequential procedure for a selection of the better of two trinomial populations has been proposed by ?idók (1988). The present paper shows some Monte Carlo results for 4 different strategies of sequential experimentation in this procedure, on this basis compares the strategies, and gives some practical recommendations for choosing the strategy.     

Hybrid methods for calculating optimal few-stage sequential strategies: Data monitoring for a clinical trial

Optimal batch-sequential designs are difficult to compute, even when sufficient statistics and relatively uncomplicated loss functions simplify the calculations required. While backward induction applies, its difficulty grows exponentially in the number of stages, while a recently developed forward algorithm grows only linearly, but involves a maximization over a rather flat surface. This paper explores a hybrid algorithm, partially backward induction, partially forward, that has some of the advantages of each.     

Use of a historical control group in a noninferiority trial assessing a new antibacterial treatment: A case study and discussion of practical implementation aspects

When recruitment into a clinical trial is limited due to rarity of the disease of interest, or when recruitment to the control arm is limited due to ethical reasons (eg, pediatric studies or important unmet medical need), exploiting historical controls to augment the prospectively collected database can be an attractive option. Statistical methods for combining historical data with randomized data, while accounting for the incompatibility between the two, have been recently proposed and remain an active field of research. The current literature is lacking a rigorous comparison between methods but also guidelines about their use in practice. In this paper, we compare the existing methods based on a confirmatory phase III study design exercise done for a new antibacterial therapy with a binary endpoint and a single historical dataset. A procedure to assess the relative performance of the different methods for borrowing information from historical control data is proposed, and practical questions related to the selection and implementation of methods are discussed. Based on our examination, we found that the methods have a comparable performance, but we recommend the robust mixture prior for its ease of implementation.     

Integrating phase 2 into phase 3 based on an intermediate endpoint while accounting for a cure proportion—With an application to the design of a clinical trial in acute myeloid leukemia

For a trial with primary endpoint overall survival for a molecule with curative potential, statistical methods that rely on the proportional hazards assumption may underestimate the power and the time to final analysis. We show how a cure proportion model can be used to get the necessary number of events and appropriate timing via simulation. If phase 1 results for the new drug are exceptional and/or the medical need in the target population is high, a phase 3 trial might be initiated after phase 1. Building in a futility interim analysis into such a pivotal trial may mitigate the uncertainty of moving directly to phase 3. However, if cure is possible, overall survival might not be mature enough at the interim to support a futility decision. We propose to base this decision on an intermediate endpoint that is sufficiently associated with survival. Planning for such an interim can be interpreted as making a randomized phase 2 trial a part of the pivotal trial: If stopped at the interim, the trial data would be analyzed, and a decision on a subsequent phase 3 trial would be made. If the trial continues at the interim, then the phase 3 trial is already underway. To select a futility boundary, a mechanistic simulation model that connects the intermediate endpoint and survival is proposed. We illustrate how this approach was used to design a pivotal randomized trial in acute myeloid leukemia and discuss historical data that informed the simulation model and operational challenges when implementing it.     

The Rheumatoid Arthritis Drug Development Model: a case study in Bayesian clinical trial simulation

The development of a new drug is a major undertaking and it is important to consider carefully the key decisions in the development process. Decisions are made in the presence of uncertainty and outcomes such as the probability of successful drug registration depend on the clinical development programmme. The Rheumatoid Arthritis Drug Development Model was developed to support key decisions for drugs in development for the treatment of rheumatoid arthritis. It is configured to simulate Phase 2b and 3 trials based on the efficacy of new drugs at the end of Phase 2a, evidence about the efficacy of existing treatments, and expert opinion regarding key safety criteria. The model evaluates the performance of different development programmes with respect to the duration of disease of the target population, Phase 2b and 3 sample sizes, the dose(s) of the experimental treatment, the choice of comparator, the duration of the Phase 2b clinical trial, the primary efficacy outcome and decision criteria for successfully passing Phases 2b and 3. It uses Bayesian clinical trial simulation to calculate the probability of successful drug registration based on the uncertainty about parameters of interest, thereby providing a more realistic assessment of the likely outcomes of individual trials and sequences of trials for the purpose of decision making. In this case study, the results show that, depending on the trial design, the new treatment has assurances of successful drug registration in the range 0.044–0.142 for an ACR20 outcome and 0.057–0.213 for an ACR50 outcome. Copyright © 2009 John Wiley & Sons, Ltd.     

Moving statistics beyond the individual clinical trial: applying decision science to optimize a clinical development plan

This paper describes how simple decision science techniques can be used to optimize the clinical development plan for a given compound. Using a case study from the stroke therapeutic area it is shown how methods such as decision trees can be utilized to describe, and adjudicate on, individual development plans. Terminology pertinent to decision sciences is described and areas where it is recommended statisticians should focus are highlighted. Copyright © 2005 John Wiley & Sons, Ltd.     

Evaluation of the statistical power for multiple tests: a case study

It is challenging to estimate the statistical power when a complicated testing strategy is used to adjust for the type-I error for multiple comparisons in a clinical trial. In this paper, we use the Bonferroni Inequality to estimate the lower bound of the statistical power assuming that test statistics are approximately normally distributed and the correlation structure among test statistics is unknown or only partially known. The method was applied to the design of a clinical study for sample size and statistical power estimation.     

Handling of missing data in long‐term clinical trials: a case study

Missing data in clinical trials is a well‐known problem, and the classical statistical methods used can be overly simple. This case study shows how well‐established missing data theory can be applied to efficacy data collected in a long‐term open‐label trial with a discontinuation rate of almost 50%. Satisfaction with treatment in chronically constipated patients was the efficacy measure assessed at baseline and every 3 months postbaseline. The improvement in treatment satisfaction from baseline was originally analyzed with a paired t‐test ignoring missing data and discarding the correlation structure of the longitudinal data. As the original analysis started from missing completely at random assumptions regarding the missing data process, the satisfaction data were re‐examined, and several missing at random (MAR) and missing not at random (MNAR) techniques resulted in adjusted estimate for the improvement in satisfaction over 12 months. Throughout the different sensitivity analyses, the effect sizes remained significant and clinically relevant. Thus, even for an open‐label trial design, sensitivity analysis, with different assumptions for the nature of dropouts (MAR or MNAR) and with different classes of models (selection, pattern‐mixture, or multiple imputation models), has been found useful and provides evidence towards the robustness of the original analyses; additional sensitivity analyses could be undertaken to further qualify robustness. Copyright © 2012 John Wiley & Sons, Ltd.     

Analysing data from a cluster randomized trial (cRCT) in primary care: a case study

Health technology assessment often requires the evaluation of interventions which are implemented at the level of the health service organization unit (e.g. GP practice) for clusters of individuals. In a cluster randomized controlled trial (cRCT), clusters of patients are randomized; not each patient individually.

The majority of statistical analyses, in individually RCT, assume that the outcomes on different patients are independent. In cRCTs there is doubt about the validity of this assumption as the outcomes of patients, in the same cluster, may be correlated. Hence, the analysis of data from cRCTs presents a number of difficulties. The aim of this paper is to describe the statistical methods of adjusting for clustering, in the context of cRCTs.

There are essentially four approaches to analysing cRCTs: 1. Cluster-level analysis using aggregate summary data.

2. Regression analysis with robust standard errors.

3. Random-effects/cluster-specific approach.

4. Marginal/population-averaged approach.

This paper will compare and contrast the four approaches, using example data, with binary and continuous outcomes, from a cRCT designed to evaluate the effectiveness of training Health Visitors in psychological approaches to identify post-natal depressive symptoms and support post-natal women compared with usual care. The PoNDER Trial randomized 101 clusters (GP practices) and collected data on 2659 new mothers with an 18-month follow-up.     

Sequential estimation for a family of counting processes in the nuisance parameter case

The paper deals with the problem of sequential estimation for stochastic processes in the presence of a nuisance parameter. Using the approach to estimation through estimating equations, optimum estimating functions based on a random observation time are investigated in some models for processes appearing in reliability systems theory.     

The evaluation study on tourism websites: from the perspective of triangular intuitionistic fuzzy multiple attribute group decision making

The research on the evaluation of tourism websites can measure the websites' development in the same industry, and it is helpful to improve the quality of the websites. In view of the complexity of the tourism website evaluation, this study firstly constructs the evaluation index system of tourism websites, including subjective indicators and objective indicators. Six websites are taken for example from non-profit tourism websites, direct tourism websites and intermediary tourism websites, two from each type. And then the evaluation index data is obtained; triangular fuzzy number and subjective preference value of evaluation index are calculated; fuzzy evaluation matrix, normalization matrix and the expectation matrix are constructed; the index weight is determined; tourism website comprehensive expectation is calculated and the website ranking according to comprehensive expectation is given from high to low.