Bayesian predictive power: choice of prior and some recommendations for its use as probability of success in drug development

Bayesian predictive power, the expectation of the power function with respect to a prior distribution for the true underlying effect size, is routinely used in drug development to quantify the probability of success of a clinical trial. Choosing the prior is crucial for the properties and interpretability of Bayesian predictive power. We review recommendations on the choice of prior for Bayesian predictive power and explore its features as a function of the prior. The density of power values induced by a given prior is derived analytically and its shape characterized. We find that for a typical clinical trial scenario, this density has a u‐shape very similar, but not equal, to a β‐distribution. Alternative priors are discussed, and practical recommendations to assess the sensitivity of Bayesian predictive power to its input parameters are provided. Copyright © 2016 John Wiley & Sons, Ltd.     

Weibull prediction of event times in clinical trials

In clinical trials with interim analyses planned at pre-specified event counts, one may wish to predict the times of these landmark events as a tool for logistical planning. Currently available methods use either a parametric approach based on an exponential model for survival (Bagiella and Heitjan, Statistics in Medicine 2001; 20:2055) or a non-parametric approach based on the Kaplan-Meier estimate (Ying et al., Clinical Trials 2004; 1:352). Ying et al. (2004) demonstrated the trade-off between bias and variance in these models; the exponential method is highly efficient when its assumptions hold but potentially biased when they do not, whereas the non-parametric method has minimal bias and is well calibrated under a range of survival models but typically gives wider prediction intervals and may fail to produce useful predictions early in the trial. As a potential compromise, we propose here to make predictions under a Weibull survival model. Computations are somewhat more difficult than with the simpler exponential model, but Monte Carlo studies show that predictions are robust under a broader range of assumptions. We demonstrate the method using data from a trial of immunotherapy for chronic granulomatous disease.     

Utilizing Bayesian predictive power in clinical trial design

The Bayesian paradigm provides an ideal platform to update uncertainties and carry them over into the future in the presence of data. Bayesian predictive power (BPP) reflects our belief in the eventual success of a clinical trial to meet its goals. In this paper we derive mathematical expressions for the most common types of outcomes, to make the BPP accessible to practitioners, facilitate fast computations in adaptive trial design simulations that use interim futility monitoring, and propose an organized BPP-based phase II-to-phase III design framework.     

Assessment of futility in clinical trials

The term 'futility' is used to refer to the inability of a clinical trial to achieve its objectives. In particular, stopping a clinical trial when the interim results suggest that it is unlikely to achieve statistical significance can save resources that could be used on more promising research. There are various approaches that have been proposed to assess futility, including stochastic curtailment, predictive power, predictive probability, and group sequential methods. In this paper, we describe and contrast these approaches, and discuss several issues associated with futility analyses, such as ethical considerations, whether or not type I error can or should be reclaimed, one-sided vs two-sided futility rules, and the impact of futility analyses on power.     

Type I error and power in trials with one interim futility analysis

Futility analysis reduces the opportunity to commit Type I error. For a superiority study testing a two‐sided hypothesis, an interim futility analysis can substantially reduce the overall Type I error while keeping the overall power relatively intact. In this paper, we quantify the extent of the reduction for both one‐sided and two‐sided futility analysis. We argue that, because of the reduction, we should be allowed to set the significance level for the final analysis at a level higher than the allowable Type I error rate for the study. We propose a method to find the significance level for the final analysis. We illustrate the proposed methodology and show that a design employing a futility analysis can reduce the sample size, and therefore reduce the exposure of patients to unnecessary risk and lower the cost of a clinical trial. Copyright © 2004 John Wiley & Sons, Ltd.     

Bayesian interim analysis of lifetime data

In hypothesis testing involving censored lifetime data that are independently distributed according to an accelerated-failure-time model, it is often of interest to predict whether continuation of the experiment will significantly alter the inferences drawn at an interim point. Approaching the problem from a Bayesian viewpoint, we suggest a possible solution based on Laplace approximations to the posterior distribution of the parameters of interest and on Markov-chain Monte Carlo. We apply our results to Weibull data from a carcinogenesis experiment on mice.     

Increasing the sample size at interim for a two‐sample experiment without Type I error inflation

For the case of a one‐sample experiment with known variance σ²=1, it has been shown that at interim analysis the sample size (SS) may be increased by any arbitrary amount provided: (1) The conditional power (CP) at interim is ?50% and (2) there can be no decision to decrease the SS (stop the trial early). In this paper we verify this result for the case of a two‐sample experiment with proportional SS in the treatment groups and an arbitrary common variance. Numerous authors have presented the formula for the CP at interim for a two‐sample test with equal SS in the treatment groups and an arbitrary common variance, for both the one‐ and two‐sided hypothesis tests. In this paper we derive the corresponding formula for the case of unequal, but proportional SS in the treatment groups for both one‐sided superiority and two‐sided hypothesis tests. Finally, we present an SAS macro for doing this calculation and provide a worked out hypothetical example. In discussion we note that this type of trial design trades the ability to stop early (for lack of efficacy) for the elimination of the Type I error penalty. The loss of early stopping requires that such a design employs a data monitoring committee, blinding of the sponsor to the interim calculations, and pre‐planning of how much and under what conditions to increase the SS and that this all be formally written into an interim analysis plan before the start of the study. Copyright © 2009 John Wiley & Sons, Ltd.     

Improving interim decisions in randomized trials by exploiting information on short‐term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short‐term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short‐term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long‐term endpoint which enable the incorporation of baseline covariates and multiple short‐term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re‐assessment based on the inverse normal P‐value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.     

Methods for clarifying criteria for study continuation at interim analysis

In monitoring clinical trials, the question of futility, or whether the data thus far suggest that the results at the final analysis are unlikely to be statistically successful, is regularly of interest over the course of a study. However, the opposite viewpoint of whether the study is sufficiently demonstrating proof of concept (POC) and should continue is a valuable consideration and ultimately should be addressed with high POC power so that a promising study is not prematurely terminated. Conditional power is often used to assess futility, and this article interconnects the ideas of assessing POC for the purpose of study continuation with conditional power, while highlighting the importance of the POC type I error and the POC type II error for study continuation or not at the interim analysis. Methods for analyzing subgroups motivate the interim analyses to maintain high POC power via an adjusted interim POC significance level criterion for study continuation or testing against an inferiority margin. Furthermore, two versions of conditional power based on the assumed effect size or the observed interim effect size are considered. Graphical displays illustrate the relationship of the POC type II error for premature study termination to the POC type I error for study continuation and the associated conditional power criteria.     

Common threads between sample size recalculation and group sequential procedures

Sample size recalculation (SSR) methods ensure that a study closely achieves the required power by allowing for the original sample size to be modified through updating misspecified nuisance parameters. Some SSR procedures extend this concept by incorporating information on the estimated treatment difference as well. While group sequential methodology allows for early stopping of a study in the context of repeated hypothesis tests, SSR usually involves the extension of a study to accommodate misspecified design parameters. This work describes these methods and places them in a common framework. Copyright © 2003 John Wiley & Sons, Ltd.     

Bayesian forecasting for AR(1) models with normal coefficients

This paper presents a Bayesian solution to the problem of time series forecasting, for the case in which the generating process is an autoregressive of order one, with a normal random coefficient. The proposed procedure is based on the predictive density of the future observation. Conjugate priors are used for some parameters, while improper vague priors are used for others.     

Bayesian estimates of predictive value and related parameters of a diagnostic test

Bayesian analysis of predictive values and related parameters of a diagnostic test are derived. In one case, the estimates are conditional on values of the prevalence of the disease; in the second case, the corresponding unconditional estimates are presented. Small-sample point estimates, posterior moments, and credibility intervals for all related parameters are obtained. Numerical methods of solution are also discussed.     

On the accuracy of a normal approximation to the power of the mantel-haenszel procedure

The accuracy of a normal approximation to the exact conditional power function of the Manic!-Haenszcl one degree ni freedom chi-squared procedure is nnesugaied The maximum approximation error over the parameter space is studied for designs with equal numbers of cases and controls within and across strata. An empirical rule is provided for the estimation of the maximum error, the rule performs well over a range of design constants of importance in stratified case-control studies.     

A Bayesian predictive approach to determining the number of components in a mixture distribution

This paper describes a Bayesian approach to mixture modelling and a method based on predictive distribution to determine the number of components in the mixtures. The implementation is done through the use of the Gibbs sampler. The method is described through the mixtures of normal and gamma distributions. Analysis is presented in one simulated and one real data example. The Bayesian results are then compared with the likelihood approach for the two examples.     

Exact power computation for stratified dose-response studies

Toxicological and pharmaceutical studies often use the stratified dose-response design. This paper deals with the problem of exact power computation for such a design. Recourse to exact methods is advised in study with sparse data. We give three algorithms for exact power calculation and apply them to three data sets. The algorithms include an exact power computation approach and two alternatives to further reduce the computation time: a -precision approach and Monte Carlo approach. Four exact and two asymptotic tests are used for illustration.     

Combining evidence from clinical trials in conditional or accelerated approval

Conditional (European Medicines Agency) or accelerated (U.S. Food and Drug Administration) approval of drugs allows earlier access to promising new treatments that address unmet medical needs. Certain post-marketing requirements must typically be met in order to obtain full approval, such as conducting a new post-market clinical trial. We study the applicability of the recently developed harmonic mean ${\chi }^2$-test to this conditional or accelerated approval framework. The proposed approach can be used both to support the design of the post-market trial and the analysis of the combined evidence provided by both trials. Other methods considered are the two-trials rule, Fisher's criterion and Stouffer's method. In contrast to some of the traditional methods, the harmonic mean ${\chi }^2$-test always requires a post-market clinical trial. If the $p$-value from the pre-market clinical trial is $\ll 0.025$, a smaller sample size for the post-market clinical trial is needed than with the two-trials rule. For illustration, we apply the harmonic mean ${\chi }^2$-test to a drug which received conditional (and later full) market licensing by the EMA. A simulation study is conducted to study the operating characteristics of the harmonic mean ${\chi }^2$-test and two-trials rule in more detail. We finally investigate the applicability of these two methods to compute the power at interim of an ongoing post-market trial. These results are expected to aid in the design and assessment of the required post-market studies in terms of the level of evidence required for full approval.     

A predictive approach for the selection of a fixed number of good treatments

This paper offers a predictive approach for the selection of a fixed number (= t) of treatments from k treatments with the goal of controlling for predictive losses. For the ith treatment, independent observations X _ij (j = 1,2,…,n) can be observed where X _ij ’s are normally distributed N(θ _i ; σ ²). The ranked values of θ _i ’s and X _i ’s are θ ₍₁₎ ≤ … ≤ θ _(k) and X _[1] ≤ … ≤ X _[k] and the selected subset S = {[k], [k? 1], … , [k ? t+1]} will be considered. This paper distinguishes between two types of loss functions. A type I loss function associated with a selected subset S is the loss in utility from the selector’s view point and is a function of θ _i with i ? S. A type II loss function associated with S measures the unfairness in the selection from candidates’ viewpoint and is a function of θ _i with i ? S. This paper shows that under mild assumptions on the loss functions S is optimal and provides the necessary formulae for choosing n so that the two types of loss can be controlled individually or simultaneously with a high probability. Predictive bounds for the losses are provided, Numerical examples support the usefulness of the predictive approach over the design of experiment approach.     

Bayesian influence diagnostics in radiocarbon dating

Linear models constitute the primary statistical technique for any experimental science. A major topic in this area is the detection of influential subsets of data, that is, of observations that are influential in terms of their effect on the estimation of parameters in linear regression or of the total population parameters. Numerous studies exist on radiocarbon dating which propose a value consensus and remove possible outliers after the corresponding testing. An influence analysis for the value consensus from a Bayesian perspective is developed in this article.     

The limiting power of the durbin-watson test

We compute the limiting povwer of the Durbin-Watson test in a general linear regression model Our treatment includes previous results due to W. Kramer and H, Zeisel as well as new results. In particular, we provide new insight under which conditions the limiting power is zero or one. Stochastic-simulations correspond to our investigations.     

A comparison of the estimative and predictive methods of estimating posterior probabilities

An asymptotic account Is presented on the relative performance of the so-called estimative and predictive methods of estimating the posterior probability that an object belongs to one of two possible multivariate normal populations. For equal prior probabil-ities it is concluded that the predictive method generally gives a less extreme estimate than the estimative. This is supported by previously available results based essentially on simulation studies. Conditions under which the predictive method provides less extreme estimates for arbitrary prior probabilities are considered. Also, the asymptotic biases associated with the two methods are compared.