Randomization as a basis for inference in noninferiority trials

Noninferiority testing in clinical trials is commonly understood in a Neyman-Pearson framework, and has been discussed in a Bayesian framework as well. In this paper, we discuss noninferiority testing in a Fisherian framework, in which the only assumption necessary for inference is the assumption of randomization of treatments to study subjects. Randomization plays an important role in not only the design but also the analysis of clinical trials, no matter the underlying inferential field. The ability to utilize permutation tests depends on assumptions around exchangeability, and we discuss the possible uses of permutation tests in active control noninferiority analyses. The other practical implications of this paper are admittedly minor but lead to better understanding of the historical and philosophical development of active control noninferiority testing. The conclusion may also frame discussion of other complicated issues in noninferiority testing, such as the role of an intention to treat analysis.     

An evidential approach to non-inferiority clinical trials

We present likelihood methods for defining the non-inferiority margin and measuring the strength of evidence in non-inferiority trials using the 'fixed-margin' framework. Likelihood methods are used to (1) evaluate and combine the evidence from historical trials to define the non-inferiority margin, (2) assess and report the smallest non-inferiority margin supported by the data, and (3) assess potential violations of the constancy assumption. Data from six aspirin-controlled trials for acute coronary syndrome and data from an active-controlled trial for acute coronary syndrome, Organisation to Assess Strategies for Ischemic Syndromes (OASIS-2) trial, are used for illustration. The likelihood framework offers important theoretical and practical advantages when measuring the strength of evidence in non-inferiority trials. Besides eliminating the influence of sample spaces and prior probabilities on the 'strength of evidence in the data', the likelihood approach maintains good frequentist properties. Violations of the constancy assumption can be assessed in the likelihood framework when it is appropriate to assume a unifying regression model for trial data and a constant control effect including a control rate parameter and a placebo rate parameter across historical placebo controlled trials and the non-inferiority trial. In situations where the statistical non-inferiority margin is data driven, lower likelihood support interval limits provide plausibly conservative candidate margins.     

Design and analysis of a 3‐arm noninferiority trial with a prespecified margin for the hazard ratio

A 3‐arm trial design that includes an experimental treatment, an active reference treatment, and a placebo is useful for assessing the noninferiority of an experimental treatment. The inclusion of a placebo arm enables the assessment of assay sensitivity and internal validation, in addition to the testing of the noninferiority of the experimental treatment compared with the reference treatment. In 3‐arm noninferiority trials, various statistical test procedures have been considered to evaluate the following 3 hypotheses: (i) superiority of the experimental treatment over the placebo, (ii) superiority of the reference treatment over the placebo, and (iii) noninferiority of the experimental treatment compared with the reference treatment. However, hypothesis (ii) can be insufficient and may not accurately assess the assay sensitivity for the noninferiority of the experimental treatment compared with the reference treatment. Thus, demonstrating that the superiority of the reference treatment over the placebo is greater than the noninferiority margin (the nonsuperiority of the reference treatment compared with the placebo) can be necessary. Here, we propose log‐rank statistical procedures for evaluating data obtained from 3‐arm noninferiority trials to assess assay sensitivity with a prespecified margin Δ. In addition, we derive the approximate sample size and optimal allocation required to minimize the total sample size and that of the placebo treatment sample size, hierarchically.     

Sample size calculations for noninferiority trials for time-to-event data using the concept of proportional time

Noninferiority trials intend to show that a new treatment is ‘not worse'' than a standard-of-care active control and can be used as an alternative when it is likely to cause fewer side effects compared to the active control. In the case of time-to-event endpoints, existing methods of sample size calculation are done either assuming proportional hazards between the two study arms, or assuming exponentially distributed lifetimes. In scenarios where these assumptions are not true, there are few reliable methods for calculating the sample sizes for a time-to-event noninferiority trial. Additionally, the choice of the non-inferiority margin is obtained either from a meta-analysis of prior studies, or strongly justifiable ‘expert opinion'', or from a ‘well conducted'' definitive large-sample study. Thus, when historical data do not support the traditional assumptions, it would not be appropriate to use these methods to design a noninferiority trial. For such scenarios, an alternate method of sample size calculation based on the assumption of Proportional Time is proposed. This method utilizes the generalized gamma ratio distribution to perform the sample size calculations. A practical example is discussed, followed by insights on choice of the non-inferiority margin, and the indirect testing of superiority of treatment compared to placebo.KEYWORDS: Generalized gamma, noninferiority, non-proportional hazards, proportional time, relative time, sample size     

A dynamic power prior for borrowing historical data in noninferiority trials with binary endpoint

Traditionally, noninferiority hypotheses have been tested using a frequentist method with a fixed margin. Given that information for the control group is often available from previous studies, it is interesting to consider a Bayesian approach in which information is “borrowed” for the control group to improve efficiency. However, construction of an appropriate informative prior can be challenging. In this paper, we consider a hybrid Bayesian approach for testing noninferiority hypotheses in studies with a binary endpoint. To account for heterogeneity between the historical information and the current trial for the control group, a dynamic P value–based power prior parameter is proposed to adjust the amount of information borrowed from the historical data. This approach extends the simple test‐then‐pool method to allow a continuous discounting power parameter. An adjusted α level is also proposed to better control the type I error. Simulations are conducted to investigate the performance of the proposed method and to make comparisons with other methods including test‐then‐pool and hierarchical modeling. The methods are illustrated with data from vaccine clinical trials.     

Count data and treatment heterogeneity in 2×2 crossover trials

Count data are routinely assumed to have a Poisson distribution, especially when there are no straightforward diagnostic procedures for checking this assumption. We reanalyse two data sets from crossover trials of treatments for angina pectoris , in which the outcomes are counts of anginal attacks. Standard analyses focus on treatment effects, averaged over subjects; we are also interested in the dispersion of these effects (treatment heterogeneity). We set up a log-Poisson model with random coefficients to estimate the distribution of the treatment effects and show that the analysis is very sensitive to the distributional assumption; the population variance of the treatment effects is confounded with the (variance) function that relates the conditional variance of the outcomes, given the subject's rate of attacks, to the conditional mean. Diagnostic model checks based on resampling from the fitted distribution indicate that the default choice of the Poisson distribution for the analysed data sets is poorly supported. We propose to augment the data sets with observations of the counts, made possibly outside the clinical setting, so that the conditional distribution of the counts could be established.     

Non‐inferiority and networks: inferring efficacy from a web of data

In the absence of placebo‐controlled trials, the efficacy of a test treatment can be alternatively examined by showing its non‐inferiority to an active control; that is, the test treatment is not worse than the active control by a pre‐specified margin. The margin is based on the effect of the active control over placebo in historical studies. In other words, the non‐inferiority setup involves a network of direct and indirect comparisons between test treatment, active controls, and placebo. Given this framework, we consider a Bayesian network meta‐analysis that models the uncertainty and heterogeneity of the historical trials into the non‐inferiority trial in a data‐driven manner through the use of the Dirichlet process and power priors. Depending on whether placebo was present in the historical trials, two cases of non‐inferiority testing are discussed that are analogs of the synthesis and fixed‐margin approach. In each of these cases, the model provides a more reliable estimate of the control given its effect in other trials in the network, and, in the case where placebo was only present in the historical trials, the model can predict the effect of the test treatment over placebo as if placebo had been present in the non‐inferiority trial. It can further answer other questions of interest, such as comparative effectiveness of the test treatment among its comparators. More importantly, the model provides an opportunity for disproportionate randomization or the use of small sample sizes by allowing borrowing of information from a network of trials to draw explicit conclusions on non‐inferiority. Copyright © 2015 John Wiley & Sons, Ltd.     

Covariate‐adjusted borrowing of historical control data in randomized clinical trials

The borrowing of historical control data can be an efficient way to improve the treatment effect estimate of the current control group in a randomized clinical trial. When the historical and current control data are consistent, the borrowing of historical data can increase power and reduce Type I error rate. However, when these 2 sources of data are inconsistent, it may result in a combination of biased estimates, reduced power, and inflation of Type I error rate. In some situations, inconsistency between historical and current control data may be caused by a systematic variation in the measured baseline prognostic factors, which can be appropriately addressed through statistical modeling. In this paper, we propose a Bayesian hierarchical model that can incorporate patient‐level baseline covariates to enhance the appropriateness of the exchangeability assumption between current and historical control data. The performance of the proposed method is shown through simulation studies, and its application to a clinical trial design for amyotrophic lateral sclerosis is described. The proposed method is developed for scenarios involving multiple imbalanced prognostic factors and thus has meaningful implications for clinical trials evaluating new treatments for heterogeneous diseases such as amyotrophic lateral sclerosis.     

Designing historical control studies with survival endpoints using exact statistical inference

Historical control trials compare an experimental treatment with a previously conducted control treatment. By assigning all recruited samples to the experimental arm, historical control trials can better identify promising treatments in early phase trials compared with randomized control trials. Existing designs of historical control trials with survival endpoints are based on asymptotic normal distribution. However, it remains unclear whether the asymptotic distribution of the test statistic is close enough to the true distribution given relatively small sample sizes in early phase trials. In this article, we address this question by introducing an exact design approach for exponentially distributed survival endpoints, and compare it with an asymptotic design in both real examples and simulation examples. Simulation results show that the asymptotic test could lead to bias in the sample size estimation. We conclude the proposed exact design should be used in the design of historical control trials.     

A meta-analytic framework to adjust for bias in external control studies

While randomized controlled trials (RCTs) are the gold standard for estimating treatment effects in medical research, there is increasing use of and interest in using real-world data for drug development. One such use case is the construction of external control arms for evaluation of efficacy in single-arm trials, particularly in cases where randomization is either infeasible or unethical. However, it is well known that treated patients in non-randomized studies may not be comparable to control patients—on either measured or unmeasured variables—and that the underlying population differences between the two groups may result in biased treatment effect estimates as well as increased variability in estimation. To address these challenges for analyses of time-to-event outcomes, we developed a meta-analytic framework that uses historical reference studies to adjust a log hazard ratio estimate in a new external control study for its additional bias and variability. The set of historical studies is formed by constructing external control arms for historical RCTs, and a meta-analysis compares the trial controls to the external control arms. Importantly, a prospective external control study can be performed independently of the meta-analysis using standard causal inference techniques for observational data. We illustrate our approach with a simulation study and an empirical example based on reference studies for advanced non-small cell lung cancer. In our empirical analysis, external control patients had lower survival than trial controls (hazard ratio: 0.907), but our methodology is able to correct for this bias. An implementation of our approach is available in the R package ecmeta .     

A novel equivalence probability weighted power prior for using historical control data in an adaptive clinical trial design: A comparison to standard methods

A standard two-arm randomised controlled trial usually compares an intervention to a control treatment with equal numbers of patients randomised to each treatment arm and only data from within the current trial are used to assess the treatment effect. Historical data are used when designing new trials and have recently been considered for use in the analysis when the required number of patients under a standard trial design cannot be achieved. Incorporating historical control data could lead to more efficient trials, reducing the number of controls required in the current study when the historical and current control data agree. However, when the data are inconsistent, there is potential for biased treatment effect estimates, inflated type I error and reduced power. We introduce two novel approaches for binary data which discount historical data based on the agreement with the current trial controls, an equivalence approach and an approach based on tail area probabilities. An adaptive design is used where the allocation ratio is adapted at the interim analysis, randomising fewer patients to control when there is agreement. The historical data are down-weighted in the analysis using the power prior approach with a fixed power. We compare operating characteristics of the proposed design to historical data methods in the literature: the modified power prior; commensurate prior; and robust mixture prior. The equivalence probability weight approach is intuitive and the operating characteristics can be calculated exactly. Furthermore, the equivalence bounds can be chosen to control the maximum possible inflation in type I error.     

Statistical testing strategies for assessing treatment efficacy and marker accuracy in phase III trials

When a candidate predictive marker is available, but evidence on its predictive ability is not sufficiently reliable, all‐comers trials with marker stratification are frequently conducted. We propose a framework for planning and evaluating prospective testing strategies in confirmatory, phase III marker‐stratified clinical trials based on a natural assumption on heterogeneity of treatment effects across marker‐defined subpopulations, where weak rather than strong control is permitted for multiple population tests. For phase III marker‐stratified trials, it is expected that treatment efficacy is established in a particular patient population, possibly in a marker‐defined subpopulation, and that the marker accuracy is assessed when the marker is used to restrict the indication or labelling of the treatment to a marker‐based subpopulation, ie, assessment of the clinical validity of the marker. In this paper, we develop statistical testing strategies based on criteria that are explicitly designated to the marker assessment, including those examining treatment effects in marker‐negative patients. As existing and developed statistical testing strategies can assert treatment efficacy for either the overall patient population or the marker‐positive subpopulation, we also develop criteria for evaluating the operating characteristics of the statistical testing strategies based on the probabilities of asserting treatment efficacy across marker subpopulations. Numerical evaluations to compare the statistical testing strategies based on the developed criteria are provided.     

Bayesian Experimental Design for Long-Term Longitudinal HIV Dynamic Studies

The study of HIV dynamics is one of the most important developments in recent AIDS research for understanding the pathogenesis of HIV-1 infection and antiviral treatment strategies. Currently a large number of AIDS clinical trials on HIV dynamics are in development worldwide. However, many design issues that arise from AIDS clinical trials have not been addressed. In this paper, we use a simulation-based approach to deal with design problems in Bayesian hierarchical nonlinear (mixed-effects) models. The underlying model characterizes the long-term viral dynamics with antiretroviral treatment where we directly incorporate drug susceptibility and exposure into a function of treatment efficacy. The Bayesian design method is investigated under the framework of hierarchical Bayesian (mixed-effects) models. We compare a finite number of feasible candidate designs numerically, which are currently used in AIDS clinical trials from different perspectives, and provide guidance on how a design might be chosen in practice.     

Indirect comparisons in the comparative efficacy and non-inferiority settings

International Conference on Harmonization E10 concerns non-inferiority trials and the assessment of comparative efficacy, both of which often involve indirect comparisons. In the non-inferiority setting, there are clinical trial results directly comparing an experimental treatment with an active control, and clinical trial results directly comparing the active control with placebo, and there is an interest in the indirect comparison of the experimental treatment with placebo. In the comparative efficacy setting, there may be separate clinical trial results comparing each of two treatments with placebo, and there is interest in an indirect comparison of the treatments. First, we show that the sample size required for a trial intended to demonstrate superiority through an indirect comparison is always greater than the sample size required for a direct comparison. In addition, by introducing the concept of preservation of effect, we show that the hypothesis addressed in the two settings is identical. Our main result concerns the logical inconsistency between a reasonable criterion for preference of an experimental treatment to a standard treatment and existing regulatory guidance for approval of the experimental treatment on the basis of an indirect comparison. Specifically, the preferred treatment will not always meet the criterion for regulatory approval. This is due to the fact that the experimental treatment bears the burden of overcoming the uncertainty in the effect of the standard treatment. We consider an alternative approval criterion that avoids this logical inconsistency.     

Leveraging historical data into oncology development programs: Two case studies of phase 2 Bayesian augmented control trial designs

Leveraging historical data into the design and analysis of phase 2 randomized controlled trials can improve efficiency of drug development programs. Such approaches can reduce sample size without loss of power. Potential issues arise when the current control arm is inconsistent with historical data, which may lead to biased estimates of treatment efficacy, loss of power, or inflated type 1 error. Consideration as to how to borrow historical information is important, and in particular, adjustment for prognostic factors should be considered. This paper will illustrate two motivating case studies of oncology Bayesian augmented control (BAC) trials. In the first example, a glioblastoma study, an informative prior was used for the control arm hazard rate. Sample size savings were 15% to 20% by using a BAC design. In the second example, a pancreatic cancer study, a hierarchical model borrowing method was used, which enabled the extent of borrowing to be determined by consistency of observed study data with historical studies. Supporting Bayesian analyses also adjusted for prognostic factors. Incorporating historical data via Bayesian trial design can provide sample size savings, reduce study duration, and enable a more scientific approach to development of novel therapies by avoiding excess recruitment to a control arm. Various sensitivity analyses are necessary to interpret results. Current industry efforts for data transparency have meaningful implications for access to patient‐level historical data, which, while not critical, is helpful to adjust for potential imbalances in prognostic factors.     

Subgroup analysis and interpretation for phase 3 confirmatory trials: White paper of the EFSPI/PSI working group on subgroup analysis

Subgroup by treatment interaction assessments are routinely performed when analysing clinical trials and are particularly important for phase 3 trials where the results may affect regulatory labelling. Interpretation of such interactions is particularly difficult, as on one hand the subgroup finding can be due to chance, but equally such analyses are known to have a low chance of detecting differential treatment effects across subgroup levels, so may overlook important differences in therapeutic efficacy. EMA have therefore issued draft guidance on the use of subgroup analyses in this setting. Although this guidance provided clear proposals on the importance of pre‐specification of likely subgroup effects and how to use this when interpreting trial results, it is less clear which analysis methods would be reasonable, and how to interpret apparent subgroup effects in terms of whether further evaluation or action is necessary. A PSI/EFSPI Working Group has therefore been investigating a focused set of analysis approaches to assess treatment effect heterogeneity across subgroups in confirmatory clinical trials that take account of the number of subgroups explored and also investigating the ability of each method to detect such subgroup heterogeneity. This evaluation has shown that the plotting of standardised effects, bias‐adjusted bootstrapping method and SIDES method all perform more favourably than traditional approaches such as investigating all subgroup‐by‐treatment interactions individually or applying a global test of interaction. Therefore, these approaches should be considered to aid interpretation and provide context for observed results from subgroup analyses conducted for phase 3 clinical trials.     

Tests for noninferiority trials with binomial endpoints: A guide to modern and quasi‐exact methods for biomedical researchers

Applied statisticians and pharmaceutical researchers are frequently involved in the design and analysis of clinical trials where at least one of the outcomes is binary. Treatments are judged by the probability of a positive binary response. A typical example is the noninferiority trial, where it is tested whether a new experimental treatment is practically not inferior to an active comparator with a prespecified margin δ. Except for the special case of δ = 0, no exact conditional test is available although approximate conditional methods (also called second‐order methods) can be applied. However, in some situations, the approximation can be poor and the logical argument for approximate conditioning is not compelling. The alternative is to consider an unconditional approach. Standard methods like the pooled z‐test are already unconditional although approximate. In this article, we review and illustrate unconditional methods with a heavy emphasis on modern methods that can deliver exact, or near exact, results. For noninferiority trials based on either rate difference or rate ratio, our recommendation is to use the so‐called E‐procedure, based on either the score or likelihood ratio statistic. This test is effectively exact, computationally efficient, and respects monotonicity constraints in practice. We support our assertions with a numerical study, and we illustrate the concepts developed in theory with a clinical example in pulmonary oncology; R code to conduct all these analyses is available from the authors.     

Testing noninferiority in three‐armed clinical trials based on likelihood ratio statistics

Clinical noninferiority trials with at least three groups have received much attention recently, perhaps due to the fact that regulatory agencies often require that a placebo group be evaluated along with a new experimental drug and an active control. The authors discuss likelihood ratio tests for binary endpoints and various noninferiority hypotheses. They find that, depending on the particular hypothesis, the test reduces asymptotically either to the intersection‐union test or to a test which follows asymptotically a mixture of generalized chi‐squared distributions. They investigate the performance of this asymptotic test and provide an exact modification. They show that this test considerably outperforms multiple testing methods such as the Bonferroni adjustment with respect to power. They illustrate their methods with a cancer study to compare antiemetic agents. Finally, they discuss the extension of the results to other settings, such as Gaussian endpoints.     

A computationally simple bivariate survival estimator for efficacy and safety

Both treatment efficacy and safety are typically the primary endpoints in Phase II, and even in some Phase III, clinical trials. Efficacy is frequently measured by time to response, death, or some other milestone event and thus is a continuous, possibly censored, outcome. Safety, however, is frequently measured on a discrete scale; in Eastern Cooperative Oncology Group clinical trial E2290, it was measured as the number of weekly rounds of chemotherapy that were tolerable to colorectal cancer patients. For the joint analysis of efficacy and safety, we propose a non-parametric, computationally simple estimator for the bivariate survival function when one time-to-event is continuous, one is discrete, and both are subject to right-censoring. The bivariate censoring times may depend on each other, but they are assumed to be independent of both event times. We derive a closed-form covariance estimator for the survivor function which allows for inference to be based on any of several possible statistics of interest. In addition, we derive its covariance with respect to calendar time of analysis, allowing for its use in sequential studies.     

Adjusting overall survival for treatment switches: commonly used methods and practical application

In parallel group trials, long‐term efficacy endpoints may be affected if some patients switch or cross over to the alternative treatment arm prior to the event. In oncology trials, switch to the experimental treatment can occur in the control arm following disease progression and potentially impact overall survival. It may be a clinically relevant question to estimate the efficacy that would have been observed if no patients had switched, for example, to estimate ‘real‐life’ clinical effectiveness for a health technology assessment. Several commonly used statistical methods are available that try to adjust time‐to‐event data to account for treatment switching, ranging from naive exclusion and censoring approaches to more complex inverse probability of censoring weighting and rank‐preserving structural failure time models. These are described, along with their key assumptions, strengths, and limitations. Best practice guidance is provided for both trial design and analysis when switching is anticipated. Available statistical software is summarized, and examples are provided of the application of these methods in health technology assessments of oncology trials. Key considerations include having a clearly articulated rationale and research question and a well‐designed trial with sufficient good quality data collection to enable robust statistical analysis. No analysis method is universally suitable in all situations, and each makes strong untestable assumptions. There is a need for further research into new or improved techniques. This information should aid statisticians and their colleagues to improve the design and analysis of clinical trials where treatment switch is anticipated. Copyright © 2013 John Wiley & Sons, Ltd.