Inference and Decision Making for 21st-Century Drug Development and Approval

ABSTRACT

The cost and time of pharmaceutical drug development continue to grow at rates that many say are unsustainable. These trends have enormous impact on what treatments get to patients, when they get them and how they are used. The statistical framework for supporting decisions in regulated clinical development of new medicines has followed a traditional path of frequentist methodology. Trials using hypothesis tests of “no treatment effect” are done routinely, and the p-value < 0.05 is often the determinant of what constitutes a “successful” trial. Many drugs fail in clinical development, adding to the cost of new medicines, and some evidence points blame at the deficiencies of the frequentist paradigm. An unknown number effective medicines may have been abandoned because trials were declared “unsuccessful” due to a p-value exceeding 0.05. Recently, the Bayesian paradigm has shown utility in the clinical drug development process for its probability-based inference. We argue for a Bayesian approach that employs data from other trials as a “prior” for Phase 3 trials so that synthesized evidence across trials can be utilized to compute probability statements that are valuable for understanding the magnitude of treatment effect. Such a Bayesian paradigm provides a promising framework for improving statistical inference and regulatory decision making.     

Decision‐making in drug development using a composite definition of success

Evidence‐based quantitative methodologies have been proposed to inform decision‐making in drug development, such as metrics to make go/no‐go decisions or predictions of success, identified with statistical significance of future clinical trials. While these methodologies appropriately address some critical questions on the potential of a drug, they either consider the past evidence without predicting the outcome of the future trials or focus only on efficacy, failing to account for the multifaceted aspects of a successful drug development. As quantitative benefit‐risk assessments could enhance decision‐making, we propose a more comprehensive approach using a composite definition of success based not only on the statistical significance of the treatment effect on the primary endpoint but also on its clinical relevance and on a favorable benefit‐risk balance in the next pivotal studies. For one drug, we can thus study several development strategies before starting the pivotal trials by comparing their predictive probability of success. The predictions are based on the available evidence from the previous trials, to which new hypotheses on the future development could be added. The resulting predictive probability of composite success provides a useful summary to support the discussions of the decision‐makers. We present a fictive, but realistic, example in major depressive disorder inspired by a real decision‐making case.     

A flexible multi-stage design for phase II oncology trials

Phase II trials evaluate whether a new drug or a new therapy is worth further pursuing or certain treatments are feasible or not. A typical phase II is a single arm (open label) trial with a binary clinical endpoint (response to therapy). Although many oncology Phase II clinical trials are designed with a two-stage procedure, multi-stage design for phase II cancer clinical trials are now feasible due to increased capability of data capture. Such design adjusts for multiple analyses and variations in analysis time, and provides greater flexibility such as minimizing the number of patients treated on an ineffective therapy and identifying the minimum number of patients needed to evaluate whether the trial would warrant further development. In most of the NIH sponsored studies, the early stopping rule is determined so that the number of patients treated on an ineffective therapy is minimized. In pharmaceutical trials, it is also of importance to know as early as possible if the trial is highly promising and what is the likelihood the early conclusion can sustain. Although various methods are available to address these issues, practitioners often use disparate methods for addressing different issues and do not realize a single unified method exists. This article shows how to utilize a unified approach via a fully sequential procedure, the sequential conditional probability ratio test, to address the multiple needs of a phase II trial. We show the fully sequential program can be used to derive an optimized efficient multi-stage design for either a low activity or a high activity, to identify the minimum number of patients required to assess whether a new drug warrants further study and to adjust for unplanned interim analyses. In addition, we calculate a probability of discordance that the statistical test will conclude otherwise should the trial continue to the planned end that is usually at the sample size of a fixed sample design. This probability can be used to aid in decision making in a drug development program. All computations are based on exact binomial distribution.     

Oncology phase II proof‐of‐concept studies with multiple targets: Randomized controlled trial or single arm?

For oncology drug development, phase II proof‐of‐concept studies have played a key role in determining whether or not to advance to a confirmatory phase III trial. With the increasing number of immunotherapies, efficient design strategies are crucial in moving successful drugs quickly to market. Our research examines drug development decision making under the framework of maximizing resource investment, characterized by benefit cost ratios (BCRs). In general, benefit represents the likelihood that a drug is successful, and cost is characterized by the risk adjusted total sample size of the phases II and III studies. Phase III studies often include a futility interim analysis; this sequential component can also be incorporated into BCRs. Under this framework, multiple scenarios can be considered. For example, for a given drug and cancer indication, BCRs can yield insights into whether to use a randomized control trial or a single‐arm study. Importantly, any uncertainty in historical control estimates that are used to benchmark single‐arm studies can be explicitly incorporated into BCRs. More complex scenarios, such as restricted resources or multiple potential cancer indications, can also be examined. Overall, BCR analyses indicate that single‐arm trials are favored for proof‐of‐concept trials when there is low uncertainty in historical control data and smaller phase III sample sizes. Otherwise, especially if the most likely to succeed tumor indication can be identified, randomized controlled trials may be a better option. While the findings are consistent with intuition, we provide a more objective approach.     

Bayesian optimal phase II designs with dual-criterion decision making

The conventional phase II trial design paradigm is to make the go/no-go decision based on the hypothesis testing framework. Statistical significance itself alone, however, may not be sufficient to establish that the drug is clinically effective enough to warrant confirmatory phase III trials. We propose the Bayesian optimal phase II trial design with dual-criterion decision making (BOP2-DC), which incorporates both statistical significance and clinical relevance into decision making. Based on the posterior probability that the treatment effect reaches the lower reference value (statistical significance) and the clinically meaningful value (clinical significance), BOP2-DC allows for go/consider/no-go decisions, rather than a binary go/no-go decision. BOP2-DC is highly flexible and accommodates various types of endpoints, including binary, continuous, time-to-event, multiple, and coprimary endpoints, in single-arm and randomized trials. The decision rule of BOP2-DC is optimized to maximize the probability of a go decision when the treatment is effective or minimize the expected sample size when the treatment is futile. Simulation studies show that the BOP2-DC design yields desirable operating characteristics. The software to implement BOP2-DC is freely available at www.trialdesign.org .     

Selection bias,investment decisions and treatment effect distributions

When making decisions regarding the investment and design for a Phase 3 programme in the development of a new drug, the results from preceding Phase 2 trials are an important source of information. However, only projects in which the Phase 2 results show promising treatment effects will typically be considered for a Phase 3 investment decision. This implies that, for those projects where Phase 3 is pursued, the underlying Phase 2 estimates are subject to selection bias. We will in this article investigate the nature of this selection bias based on a selection of distributions for the treatment effect. We illustrate some properties of Bayesian estimates, providing shrinkage of the Phase 2 estimate to counteract the selection bias. We further give some empirical guidance regarding the choice of prior distribution and comment on the consequences for decision-making in investment and planning for Phase 3 programmes.     

Using short-term evidence to predict six-month outcomes in clinical trials of signs and symptoms in rheumatoid arthritis

A model is presented to generate a distribution for the probability of an ACR response at six months for a new treatment for rheumatoid arthritis given evidence from a one- or three-month clinical trial. The model is based on published evidence from 11 randomized controlled trials on existing treatments. A hierarchical logistic regression model is used to find the relationship between the proportion of patients achieving ACR20 and ACR50 at one and three months and the proportion at six months. The model is assessed by Bayesian predictive P-values that demonstrate that the model fits the data well. The model can be used to predict the number of patients with an ACR response for proposed six-month clinical trials given data from clinical trials of one or three months duration.     

Adapting the sample size planning of a phase III trial based on phase II data

Traditionally, in clinical development plan, phase II trials are relatively small and can be expected to result in a large degree of uncertainty in the estimates based on which Phase III trials are planned. Phase II trials are also to explore appropriate primary efficacy endpoint(s) or patient populations. When the biology of the disease and pathogenesis of disease progression are well understood, the phase II and phase III studies may be performed in the same patient population with the same primary endpoint, e.g. efficacy measured by HbA1c in non-insulin dependent diabetes mellitus trials with treatment duration of at least three months. In the disease areas that molecular pathways are not well established or the clinical outcome endpoint may not be observed in a short-term study, e.g. mortality in cancer or AIDS trials, the treatment effect may be postulated through use of intermediate surrogate endpoint in phase II trials. However, in many cases, we generally explore the appropriate clinical endpoint in the phase II trials. An important question is how much of the effect observed in the surrogate endpoint in the phase II study can be translated into the clinical effect in the phase III trial. Another question is how much of the uncertainty remains in phase III trials. In this work, we study the utility of adaptation by design (not by statistical test) in the sense of adapting the phase II information for planning the phase III trials. That is, we investigate the impact of using various phase II effect size estimates on the sample size planning for phase III trials. In general, if the point estimate of the phase II trial is used for planning, it is advisable to size the phase III trial by choosing a smaller alpha level or a higher power level. The adaptation via using the lower limit of the one standard deviation confidence interval from the phase II trial appears to be a reasonable choice since it balances well between the empirical power of the launched trials and the proportion of trials not launched if a threshold lower than the true effect size of phase III trial can be chosen for determining whether the phase III trial is to be launched.     

An empirical Bayes approach to evaluation of results for a specific region in multiregional clinical trials

To accelerate the drug development process and shorten approval time, the design of multiregional clinical trials (MRCTs) incorporates subjects from many countries/regions around the world under the same protocol. After showing the overall efficacy of a drug in all global regions, one can also simultaneously evaluate the possibility of applying the overall trial results to all regions and subsequently support drug registration in each of them. In this paper, we focus on a specific region and establish a statistical criterion to assess the consistency between the specific region and overall results in an MRCT. More specifically, we treat each region in an MRCT as an independent clinical trial, and each perhaps has different treatment effect. We then construct the empirical prior information for the treatment effect for the specific region on the basis of all of the observed data from other regions. We will conclude similarity between the specific region and all regions if the posterior probability of deriving a positive treatment effect in the specific region is large, say 80%. Numerical examples illustrate applications of the proposed approach in different scenarios. Copyright © 2013 John Wiley & Sons, Ltd.     

Characterization of dose‐response for count data using a generalized MCP‐Mod approach in an adaptive dose‐ranging trial

Understanding the dose–response relationship is a key objective in Phase II clinical development. Yet, designing a dose‐ranging trial is a challenging task, as it requires identifying the therapeutic window and the shape of the dose–response curve for a new drug on the basis of a limited number of doses. Adaptive designs have been proposed as a solution to improve both quality and efficiency of Phase II trials as they give the possibility to select the dose to be tested as the trial goes. In this article, we present a ‘shapebased’ two‐stage adaptive trial design where the doses to be tested in the second stage are determined based on the correlation observed between efficacy of the doses tested in the first stage and a set of pre‐specified candidate dose–response profiles. At the end of the trial, the data are analyzed using the generalized MCP‐Mod approach in order to account for model uncertainty. A simulation study shows that this approach gives more precise estimates of a desired target dose (e.g. ED70) than a single‐stage (fixed‐dose) design and performs as well as a two‐stage D‐optimal design. We present the results of an adaptive model‐based dose‐ranging trial in multiple sclerosis that motivated this research and was conducted using the presented methodology. Copyright © 2015 John Wiley & Sons, Ltd.     

START: single‐to‐double arm transition design for phase II clinical trials

Phase II clinical trials designed for evaluating a drug's treatment effect can be either single‐arm or double‐arm. A single‐arm design tests the null hypothesis that the response rate of a new drug is lower than a fixed threshold, whereas a double‐arm scheme takes a more objective comparison of the response rate between the new treatment and the standard of care through randomization. Although the randomized design is the gold standard for efficacy assessment, various situations may arise where a single‐arm pilot study prior to a randomized trial is necessary. To combine the single‐ and double‐arm phases and pool the information together for better decision making, we propose a Single‐To‐double ARm Transition design (START) with switching hypotheses tests, where the first stage compares the new drug's response rate with a minimum required level and imposes a continuation criterion, and the second stage utilizes randomization to determine the treatment's superiority. We develop a software package in R to calibrate the frequentist error rates and perform simulation studies to assess the trial characteristics. Finally, a metastatic pancreatic cancer trial is used for illustrating the decision rules under the proposed START design.     

Joint probability of statistical success of multiple phase III trials

In drug development, after completion of phase II proof‐of‐concept trials, the sponsor needs to make a go/no‐go decision to start expensive phase III trials. The probability of statistical success (PoSS) of the phase III trials based on data from earlier studies is an important factor in that decision‐making process. Instead of statistical power, the predictive power of a phase III trial, which takes into account the uncertainty in the estimation of treatment effect from earlier studies, has been proposed to evaluate the PoSS of a single trial. However, regulatory authorities generally require statistical significance in two (or more) trials for marketing licensure. We show that the predictive statistics of two future trials are statistically correlated through use of the common observed data from earlier studies. Thus, the joint predictive power should not be evaluated as a simplistic product of the predictive powers of the individual trials. We develop the relevant formulae for the appropriate evaluation of the joint predictive power and provide numerical examples. Our methodology is further extended to the more complex phase III development scenario comprising more than two (K > 2) trials, that is, the evaluation of the PoSS of at least k₀ () trials from a program of K total trials. Copyright © 2013 John Wiley & Sons, Ltd.     

Selection bias for treatments with positive Phase 2 results

In drug development, treatments are most often selected at Phase 2 for further development when an initial trial of a new treatment produces a result that is considered positive. This selection due to a positive result means, however, that an estimator of the treatment effect, which does not take account of the selection is likely to over‐estimate the true treatment effect (ie, will be biased). This bias can be large and researchers may face a disappointingly lower estimated treatment effect in further trials. In this paper, we review a number of methods that have been proposed to correct for this bias and introduce three new methods. We present results from applying the various methods to two examples and consider extensions of the examples. We assess the methods with respect to bias of estimation of the treatment effect and compare the probabilities that a bias‐corrected treatment effect estimate will exceed a decision threshold. Following previous work, we also compare average power for the situation where a Phase 3 trial is launched given that the bias‐corrected observed Phase 2 treatment effect exceeds a launch threshold. Finally, we discuss our findings and potential application of the bias correction methods.     

Defining estimands for efficacy assessment in single arm phase 1b or phase 2 clinical trials in oncology early development

The addendum of the ICH E9 guideline on the statistical principles for clinical trials introduced the estimand framework. The framework is designed to strengthen the dialog between different stakeholders, to introduce greater clarity in the clinical trial objectives and to provide alignment between the estimand and statistical analysis. Estimand framework related publications thus far have mainly focused on randomized clinical trials. The intention of the Early Development Estimand Nexus (EDEN), a task force of the cross-industry Oncology Estimand Working Group ( www.oncoestimand.org ), is to apply it to single arms Phase 1b or Phase 2 trials designed to detect a treatment-related efficacy signal, typically measured by objective response rate. Key recommendations regarding the estimand attributes include that in a single arm early clinical trial, the treatment attribute should start when the first dose is received by the participant. Focusing on the estimation of an absolute effect, the population-level summary measure should reflect only the property used for the estimation. Another major component introduced in the ICH E9 addendum is the definition of intercurrent events and the associated possible ways to handle them. Different strategies reflect different clinical questions of interest that can be answered based on the journeys an individual subject can take during a trial. We provide detailed strategy recommendations for intercurrent events typically seen in early-stage oncology. We highlight where implicit assumptions should be made transparent as whenever follow-up is suspended, a while-on-treatment strategy is implied.     

Modelling PK/QT relationships from Phase I dose‐escalation trials for drug combinations and developing quantitative risk assessments of clinically relevant QT prolongations

In current industry practice, it is difficult to assess QT effects at potential therapeutic doses based on Phase I dose‐escalation trials in oncology due to data scarcity, particularly in combinations trials. In this paper, we propose to use dose‐concentration and concentration‐QT models jointly to model the exposures and effects of multiple drugs in combination. The fitted models then can be used to make early predictions for QT prolongation to aid choosing recommended dose combinations for further investigation. The models consider potential correlation between concentrations of test drugs and potential drug–drug interactions at PK and QT levels. In addition, this approach allows for the assessment of the probability of QT prolongation exceeding given thresholds of clinical significance. The performance of this approach was examined via simulation under practical scenarios for dose‐escalation trials for a combination of two drugs. The simulation results show that invaluable information of QT effects at therapeutic dose combinations can be gained by the proposed approaches. Early detection of dose combinations with substantial QT prolongation is evaluated effectively through the CIs of the predicted peak QT prolongation at each dose combination. Furthermore, the probability of QT prolongation exceeding a certain threshold is also computed to support early detection of safety signals while accounting for uncertainty associated with data from Phase I studies. While the prediction of QT effects is sensitive to the dose escalation process, the sensitivity and limited sample size should be considered when providing support to the decision‐making process for further developing certain dose combinations. Copyright © 2016 John Wiley & Sons, Ltd.     

A case study in identifying targeted patients population in major depressive disorder by enhanced enrichment design

Despite advances in clinical trial design, failure rates near 80% in phase 2 and 50% in phase 3 have recently been reported. The challenges to successful drug development are particularly acute in central nervous system trials such as for pain, schizophrenia, mania, and depression because high‐placebo response rates lessen assay sensitivity, diminish estimated treatment effect sizes, and thereby decrease statistical power. This paper addresses the importance of rigorous patient selection in major depressive disorder trials through an enhanced enrichment paradigm. This approach led to a redefinition of an ongoing, blinded phase 3 trial algorithm for patient inclusion (1) to eliminate further randomization of transient placebo responders and (2) to exclude previously randomized transient responders from the primary analysis of the double blind phase of the trial. It is illustrated for a case study for the comparison between brexpiprazole + antidepressant therapy and placebo + antidepressant therapy. Analysis of the primary endpoint showed that efficacy of brexpiprazole versus placebo could not be established statistically if the original algorithm for identification of placebo responders was used, but the enhanced enrichment approach did statistically demonstrate efficacy. Additionally, the enhanced enrichment approach identified a target population with a clinically meaningful treatment effect. Through its successful identification of a target population, the innovative enhanced enrichment approach enabled the demonstration of a positive treatment effect in a very challenging area of depression research.     

抗肿瘤药物贝叶斯Ⅰ期临床试验设计方法比较研究

Ⅰ期临床试验的主要目的是探索药物毒性最大耐受剂量MTD,而MTD估计的准确与否将影响之后的Ⅱ期和Ⅲ期临床试验研究的结果.抗肿瘤药物Ⅰ期试验的特点是直接对病人进行试验,且样本量较小,这对构造估计精确度高并具有安全性保障要求的统计设计方法提出了挑战.回顾三种常用的Ⅰ期试验设计方法有:3+3设计、CRM设计和mTPI设计.3+3设计是应用较为广泛的传统方法,后两者是当前常用的贝叶斯自适应试验设计方法.通过大量模拟研究对三种方法从最优分配、安全性和估计MTD精确性三方面给以全面考察,并结合中国实际得出mTPI设计是比较适合推荐的Ⅰ期临床试验设计方法的结论.     

Structured approach to the elicitation of expert beliefs for a Bayesian‐designed clinical trial: a case study

To quantify uncertainty in a formal manner, statisticians play a vital role in identifying a prior distribution for a Bayesian‐designed clinical trial. However, when expert beliefs are to be used to form the prior, the literature is sparse on how feasible and how reliable it is to elicit beliefs from experts. For late‐stage clinical trials, high importance is placed on reliability; however, feasibility may be equally important in early‐stage trials. This article describes a case study to assess how feasible it is to conduct an elicitation session in a structured manner and to form a probability distribution that would be used in a hypothetical early‐stage trial. The case study revealed that by using a structured approach to planning, training and conduct, it is feasible to elicit expert beliefs and form a probability distribution in a timely manner. We argue that by further increasing the published accounts of elicitation of expert beliefs in drug development, there will be increased confidence in the feasibility of conducting elicitation sessions. Furthermore, this will lead to wider dissemination of the pertinent issues on how to quantify uncertainty to both practicing statisticians and others involved with designing trials in a Bayesian manner. Copyright © 2013 John Wiley & Sons, Ltd.     

Integrating dose estimation into a decision‐making framework for model‐based drug development

Model‐informed drug discovery and development offers the promise of more efficient clinical development, with increased productivity and reduced cost through scientific decision making and risk management. Go/no‐go development decisions in the pharmaceutical industry are often driven by effect size estimates, with the goal of meeting commercially generated target profiles. Sufficient efficacy is critical for eventual success, but the decision to advance development phase is also dependent on adequate knowledge of appropriate dose and dose‐response. Doses which are too high or low pose risk of clinical or commercial failure. This paper addresses this issue and continues the evolution of formal decision frameworks in drug development. Here, we consider the integration of both efficacy and dose‐response estimation accuracy into the go/no‐go decision process, using a model‐based approach. Using prespecified target and lower reference values associated with both efficacy and dose accuracy, we build a decision framework to more completely characterize development risk. Given the limited knowledge of dose response in early development, our approach incorporates a set of dose‐response models and uses model averaging. The approach and its operating characteristics are illustrated through simulation. Finally, we demonstrate the decision approach on a post hoc analysis of the phase 2 data for naloxegol (a drug approved for opioid‐induced constipation).