Decision‐making in early clinical drug development

This paper illustrates an approach to setting the decision framework for a study in early clinical drug development. It shows how the criteria for a go and a stop decision are calculated based on pre‐specified target and lower reference values. The framework can lead to a three‐outcome approach by including a consider zone; this could enable smaller studies to be performed in early development, with other information either external to or within the study used to reach a go or stop decision. In this way, Phase I/II trials can be geared towards providing actionable decision‐making rather than the traditional focus on statistical significance. The example provided illustrates how the decision criteria were calculated for a Phase II study, including an interim analysis, and how the operating characteristics were assessed to ensure the decision criteria were robust. Copyright © 2016 John Wiley & Sons, Ltd.     

Modelling and simulation to improve decision‐making in clinical development

Modelling and simulation are buzz words in clinical drug development. But is clinical trial simulation (CTS) really a revolutionary technique? There is not much more to CTS than applying standard methods of modelling, statistics and decision theory. However, doing this in a systematic way can mean a significant improvement in pharmaceutical research. This paper describes in simple examples how modelling could be used in clinical development. Four steps are identified: gathering relevant information about a drug and the disease; building a mathematical model; predicting the results of potential future trials; and optimizing clinical trials and the entire clinical programme. We discuss these steps and give a number of examples of model components, demonstrating that relatively unsophisticated models may also prove useful. We stress that modelling and simulation are decision tools and point out the benefits of integrating them with decision analysis. Copyright © 2005 John Wiley & Sons, Ltd.     

Conditional assurance: the answer to the questions that should be asked within drug development

In this paper, we extend the use of assurance for a single study to explore how meeting a study's pre-defined success criteria could update our beliefs about the true treatment effect and impact the assurance of subsequent studies. This concept of conditional assurance, the assurance of a subsequent study conditional on success in an initial study, can be used assess the de-risking potential of the study requiring immediate investment, to ensure it provides value within the overall development plan. If the planned study does not discharge sufficient later phase risk, alternative designs and/or success criteria should be explored. By transparently laying out the different design options and the risks associated, this allows for decision makers to make quantitative investment choices based on their risk tolerance levels and potential return on investment. This paper lays out the derivation of conditional assurance, discusses how changing the design of a planned study will impact the conditional assurance of a future study, as well as presenting a simple illustrative example of how this methodology could be used to transparently compare development plans to aid decision making within an organisation.     

Simulation-based designs for accelerated life tests

In this paper we present a Bayesian decision theoretic approach to the design of accelerated life tests (ALT). We discuss computational issues regarding the evaluation of expectation and optimization steps in the solution of the decision problem. We illustrate how Monte Carlo methods can be used in preposterior analysis to find optimal designs and how the required computational effort can be avoided by using curve-fitting techniques. In so doing, we adopt the recent Monte-Carlo-based approaches of Muller and Parmigiani (1995. J. Amer. Statist. Assoc. 90, 503–510) and Muller (2000. Bayesian Statistics 6, forthcoming) to develop optimal Bayesian designs. These approaches facilitate the preposterior analysis by replacing it with a sequence of scatter plot smoothing/regression techniques and optimization of the corresponding fitted surfaces. We present our development by considering single and multiple-point fixed, as well as, sequential design problems when the underlying life model is exponential, and illustrate the implementation of our approach with some examples.     

Integrating phase 2 into phase 3 based on an intermediate endpoint while accounting for a cure proportion—With an application to the design of a clinical trial in acute myeloid leukemia

For a trial with primary endpoint overall survival for a molecule with curative potential, statistical methods that rely on the proportional hazards assumption may underestimate the power and the time to final analysis. We show how a cure proportion model can be used to get the necessary number of events and appropriate timing via simulation. If phase 1 results for the new drug are exceptional and/or the medical need in the target population is high, a phase 3 trial might be initiated after phase 1. Building in a futility interim analysis into such a pivotal trial may mitigate the uncertainty of moving directly to phase 3. However, if cure is possible, overall survival might not be mature enough at the interim to support a futility decision. We propose to base this decision on an intermediate endpoint that is sufficiently associated with survival. Planning for such an interim can be interpreted as making a randomized phase 2 trial a part of the pivotal trial: If stopped at the interim, the trial data would be analyzed, and a decision on a subsequent phase 3 trial would be made. If the trial continues at the interim, then the phase 3 trial is already underway. To select a futility boundary, a mechanistic simulation model that connects the intermediate endpoint and survival is proposed. We illustrate how this approach was used to design a pivotal randomized trial in acute myeloid leukemia and discuss historical data that informed the simulation model and operational challenges when implementing it.     

Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study

This paper illustrates how the design and statistical analysis of the primary endpoint of a proof‐of‐concept study can be formulated within a Bayesian framework and is motivated by and illustrated with a Pfizer case study in chronic kidney disease. It is shown how decision criteria for success can be formulated, and how the study design can be assessed in relation to these, both using the traditional approach of probability of success conditional on the true treatment difference and also using Bayesian assurance and pre‐posterior probabilities. The case study illustrates how an informative prior on placebo response can have a dramatic effect in reducing sample size, saving time and resource, and we argue that in some cases, it can be considered unethical not to include relevant literature data in this way. Copyright © 2015 John Wiley & Sons, Ltd.     

Modelling and simulation in the pharmaceutical industry--some reflections

Modelling and simulation (M&S) is increasingly being applied in (clinical) drug development. It provides an opportune area for the community of pharmaceutical statisticians to pursue. In this article, we highlight useful principles behind the application of M&S. We claim that M&S should be focussed on decisions, tailored to its purpose and based in applied sciences, not relying entirely on data-driven statistical analysis. Further, M&S should be a continuous process making use of diverse information sources and applying Bayesian and frequentist methodology, as appropriate. In addition to forming a basis for analysing decision options, M&S provides a framework that can facilitate communication between stakeholders. Besides the discussion on modelling philosophy, we also describe how standard simulation practice can be ineffective and how simulation efficiency can often be greatly improved.     

Adaptive paediatric investigation plans,a small step to improve regulatory decision making in drug development for children?

Different arguments have been put forward why drug developers should commit themselves early for what they are planning to do for children. By EU regulation, paediatric investigation plans should be agreed on in early phases of drug development in adults. Here, extrapolation from adults to children is widely applied to reduce the burden and avoids unnecessary clinical trials in children, but early regulatory decisions on how far extrapolation can be used may be highly uncertain. Under special circumstances, the regulatory process should allow for adaptive paediatric investigation plans explicitly foreseeing a re‐evaluation of the early decision based on the information accumulated later from adults or elsewhere. A small step towards adaptivity and learning from experience may improve the quality of regulatory decisions in particular with regard to how much information can be borrowed from adults. Copyright © 2016 John Wiley & Sons, Ltd.     

Optimal Selection of the Most Reliable Design Based on Gamma Degradation Processes

Manufacturers are often faced with the problem of how to select the most reliable design among several competing designs in the stage of development. It becomes complicated if products are highly reliable. Under the circumstances, recent work has focused on the study with degradation data by assuming that degradation paths follow Wiener processes or random-effect models. However, it is more appropriate to use gamma processes to model degradation data with monotone-increasing pattern. This article deals with the selection problem for such processes. With a minimum probability of correct decision, optimal test plans can be obtained by minimizing the total cost.     

The Value of Information to the Acidic Deposition Debates

This article is addressed to those interested in how Bayesian approaches can be brought to bear on research and development planning and management issues. It provides a conceptual framework for estimating the value of information to environmental policy decisions. The methodology is applied to assess the expected value of research concerning the effects of acidic deposition on forests. To calculate the expected value of research requires modeling the possible actions of policymakers under conditions of uncertainty. Information is potentially valuable only if it leads to actions that differ from the actions that would be taken without the information. The relevant issue is how research on forest effects would change choices of emissions controls from those that would be made in the absence of such research. The approach taken is to model information with a likelihood function embedded in a decision tree describing possible policy options. The value of information is then calculated as a function of information accuracy. The results illustrate how accurate the information must be to have an impact on the choice of policy options. The results also illustrate situations in which additional research can have a negative value.     

Optimal planning of phase II/III programs for clinical trials with multiple endpoints

Owing to increased costs and competition pressure, drug development becomes more and more challenging. Therefore, there is a strong need for improving efficiency of clinical research by developing and applying methods for quantitative decision making. In this context, the integrated planning for phase II/III programs plays an important role as numerous quantities can be varied that are crucial for cost, benefit, and program success. Recently, a utility‐based framework has been proposed for an optimal planning of phase II/III programs that puts the choice of decision boundaries and phase II sample sizes on a quantitative basis. However, this method is restricted to studies with a single time‐to‐event endpoint. We generalize this procedure to the setting of clinical trials with multiple endpoints and (asymptotically) normally distributed test statistics. Optimal phase II sample sizes and go/no‐go decision rules are provided for both the “all‐or‐none” and “at‐least‐one” win criteria. Application of the proposed method is illustrated by drug development programs in the fields of Alzheimer disease and oncology.     

The number of decision sets involving n means

A decision set is the result of the multiple comparison of n means. The means are placed in order in a diagram and a line is drawn under those that are not significantly different. In this paper, we show how the number of decision sets involving n means can be calculated.     

Decision‐making in a phase II clinical trial: a new approach combining Bayesian and frequentist concepts

The aim of a phase II clinical trial is to decide whether or not to develop an experimental therapy further through phase III clinical evaluation. In this paper, we present a Bayesian approach to the phase II trial, although we assume that subsequent phase III clinical trials will have standard frequentist analyses. The decision whether to conduct the phase III trial is based on the posterior predictive probability of a significant result being obtained. This fusion of Bayesian and frequentist techniques accepts the current paradigm for expressing objective evidence of therapeutic value, while optimizing the form of the phase II investigation that leads to it. By using prior information, we can assess whether a phase II study is needed at all, and how much or what sort of evidence is required. The proposed approach is illustrated by the design of a phase II clinical trial of a multi‐drug resistance modulator used in combination with standard chemotherapy in the treatment of metastatic breast cancer. Copyright © 2005 John Wiley & Sons, Ltd     

The Splitting Method for Decision Making

We show how a simple modification of the splitting method based on Gibbs sampler can be efficiently used for decision making in the sense that one can efficiently decide whether or not a given set of integer program constraints has at least one feasible solution. We also show how to incorporate the classic capture-recapture method into the splitting algorithm in order to obtain a low variance estimator for the counting quantity representing, say the number of feasible solutions on the set of the constraints of an integer program. We finally present numerical with with both, the decision making and the capture-recapture estimators and show their superiority as compared to the conventional one, while solving quite general decision making and counting ones, like the satisfiability problems.     

Teaching Decision Theory Proof Strategies Using a Crowdsourcing Problem

Teaching how to derive minimax decision rules can be challenging because of the lack of examples that are simple enough to be used in the classroom. Motivated by this challenge, we provide a new example that illustrates the use of standard techniques in the derivation of optimal decision rules under the Bayes and minimax approaches. We discuss how to predict the value of an unknown quantity, θ ∈ {0, 1}, given the opinions of n experts. An important example of such crowdsourcing problem occurs in modern cosmology, where θ indicates whether a given galaxy is merging or not, and Y₁, …, Y_n are the opinions from n astronomers regarding θ. We use the obtained prediction rules to discuss advantages and disadvantages of the Bayes and minimax approaches to decision theory. The material presented here is intended to be taught to first-year graduate students.     

The Rheumatoid Arthritis Drug Development Model: a case study in Bayesian clinical trial simulation

The development of a new drug is a major undertaking and it is important to consider carefully the key decisions in the development process. Decisions are made in the presence of uncertainty and outcomes such as the probability of successful drug registration depend on the clinical development programmme. The Rheumatoid Arthritis Drug Development Model was developed to support key decisions for drugs in development for the treatment of rheumatoid arthritis. It is configured to simulate Phase 2b and 3 trials based on the efficacy of new drugs at the end of Phase 2a, evidence about the efficacy of existing treatments, and expert opinion regarding key safety criteria. The model evaluates the performance of different development programmes with respect to the duration of disease of the target population, Phase 2b and 3 sample sizes, the dose(s) of the experimental treatment, the choice of comparator, the duration of the Phase 2b clinical trial, the primary efficacy outcome and decision criteria for successfully passing Phases 2b and 3. It uses Bayesian clinical trial simulation to calculate the probability of successful drug registration based on the uncertainty about parameters of interest, thereby providing a more realistic assessment of the likely outcomes of individual trials and sequences of trials for the purpose of decision making. In this case study, the results show that, depending on the trial design, the new treatment has assurances of successful drug registration in the range 0.044–0.142 for an ACR20 outcome and 0.057–0.213 for an ACR50 outcome. Copyright © 2009 John Wiley & Sons, Ltd.     

Incorporating scientific,ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach

A flexible sequential approach to the design of clinical trials is discussed herein. This approach is based on a “confidence sequence” viewpoint instead of the rigid stopping and terminal decision rules in conventional sequential testing theory. By using an appropriate confidence sequence, one can always ensure a prescribed degree of scientific rigor (confidence) in establishing the drug to be effective. Moreover, one also has the option of terminating the trial early when there is already enough statistical evidence for concluding that the drug is effective, or when the drug shows uniorseen harmful effects, or when the data predict that there is little chance of arriving at a definitive conclusion in favor of the drug by the scheduled end of the trial. We discuss how these and other ethical and economic considerations can be readily incorporated into the stopping criteria of the trial.     

How to improve a team's position in the FIFA ranking? A simulation study

In this paper, we study the efficacy of the official ranking for international football teams compiled by FIFA, the body governing football competition around the globe. We present strategies for improving a team's position in the ranking. By combining several statistical techniques, we derive an objective function in a decision problem of optimal scheduling of future matches. The presented results display how a team's position can be improved. Along the way, we compare the official procedure to the famous Elo rating system. Although it originates from chess, it has been successfully tailored to ranking football teams as well.     

Leading beyond regulatory approval: Opportunities for statisticians to optimize evidence generation and impact clinical practice

The role and value of statistical contributions in drug development up to the point of health authority approval are well understood. But health authority approval is only a true ‘win’ if the evidence enables access and adoption into clinical practice. In today's complex and evolving healthcare environment, there is additional strategic evidence generation, communication, and decision support that can benefit from statistical contributions. In this article, we describe the history of medical affairs in the context of drug development, the factors driving post-approval evidence generation needs, and the opportunities for statisticians to optimize evidence generation for stakeholders beyond health authorities in order to ensure that new medicines reach appropriate patients.