An evidential approach to non-inferiority clinical trials

We present likelihood methods for defining the non-inferiority margin and measuring the strength of evidence in non-inferiority trials using the 'fixed-margin' framework. Likelihood methods are used to (1) evaluate and combine the evidence from historical trials to define the non-inferiority margin, (2) assess and report the smallest non-inferiority margin supported by the data, and (3) assess potential violations of the constancy assumption. Data from six aspirin-controlled trials for acute coronary syndrome and data from an active-controlled trial for acute coronary syndrome, Organisation to Assess Strategies for Ischemic Syndromes (OASIS-2) trial, are used for illustration. The likelihood framework offers important theoretical and practical advantages when measuring the strength of evidence in non-inferiority trials. Besides eliminating the influence of sample spaces and prior probabilities on the 'strength of evidence in the data', the likelihood approach maintains good frequentist properties. Violations of the constancy assumption can be assessed in the likelihood framework when it is appropriate to assume a unifying regression model for trial data and a constant control effect including a control rate parameter and a placebo rate parameter across historical placebo controlled trials and the non-inferiority trial. In situations where the statistical non-inferiority margin is data driven, lower likelihood support interval limits provide plausibly conservative candidate margins.     

Coping with missing data in phase III pivotal registration trials: Tolvaptan in subjects with kidney disease,a case study

Missing data cause challenging issues, particularly in phase III registration trials, as highlighted by the European Medicines Agency (EMA) and the US National Research Council. We explore, as a case study, how the issues from missing data were tackled in a double‐blind phase III trial in subjects with autosomal dominant polycystic kidney disease. A total of 1445 subjects were randomized in a 2:1 ratio to receive active treatment (tolvaptan), or placebo. The primary outcome, the rate of change in total kidney volume, favored tolvaptan (P < .0001). The key secondary efficacy endpoints of clinical progression of disease and rate of decline in kidney function also favored tolvaptan. However, as highlighted by Food and Drug Administration and EMA, the interpretation of results was hampered by a high number of unevenly distributed dropouts, particularly early dropouts. In this paper, we outline the analyses undertaken to address the issue of missing data thoroughly. “Tipping point analyses” were performed to explore how extreme and detrimental outcomes among subjects with missing data must be to overturn the positive treatment effect attained in those subjects who had complete data. Nonparametric rank‐based analyses were also performed accounting for missing data. In conclusion, straightforward and transparent analyses directly taking into account missing data convincingly support the robustness of the preplanned analyses on the primary and secondary endpoints. Tolvaptan was confirmed to be effective in slowing total kidney volume growth, which is considered an efficacy endpoint by EMA, and in lessening the decline in renal function in patients with autosomal dominant polycystic kidney disease.     

Active‐controlled,non‐inferiority trials in oncology: arbitrary limits,infeasible sample sizes and uninformative data analysis. Is there another way?

In oncology, it may not always be possible to evaluate the efficacy of new medicines in placebo-controlled trials. Furthermore, while some newer, biologically targeted anti-cancer treatments may be expected to deliver therapeutic benefit in terms of better tolerability or improved symptom control, they may not always be expected to provide increased efficacy relative to existing therapies. This naturally leads to the use of active-control, non-inferiority trials to evaluate such treatments. In recent evaluations of anti-cancer treatments, the non-inferiority margin has often been defined in terms of demonstrating that at least 50% of the active control effect has been retained by the new drug using methods such as those described by Rothmann et al., Statistics in Medicine 2003; 22:239-264 and Wang and Hung Controlled Clinical Trials 2003; 24:147-155. However, this approach can lead to prohibitively large clinical trials and results in a tendency to dichotomize trial outcome as either 'success' or 'failure' and thus oversimplifies interpretation. With relatively modest modification, these methods can be used to define a stepwise approach to design and analysis. In the first design step, the trial is sized to show indirectly that the new drug would have beaten placebo; in the second analysis step, the probability that the new drug is superior to placebo is assessed and, if sufficiently high in the third and final step, the relative efficacy of the new drug to control is assessed on a continuum of effect retention via an 'effect retention likelihood plot'. This stepwise approach is likely to provide a more complete assessment of relative efficacy so that the value of new treatments can be better judged.     

Non‐inferiority and networks: inferring efficacy from a web of data

In the absence of placebo‐controlled trials, the efficacy of a test treatment can be alternatively examined by showing its non‐inferiority to an active control; that is, the test treatment is not worse than the active control by a pre‐specified margin. The margin is based on the effect of the active control over placebo in historical studies. In other words, the non‐inferiority setup involves a network of direct and indirect comparisons between test treatment, active controls, and placebo. Given this framework, we consider a Bayesian network meta‐analysis that models the uncertainty and heterogeneity of the historical trials into the non‐inferiority trial in a data‐driven manner through the use of the Dirichlet process and power priors. Depending on whether placebo was present in the historical trials, two cases of non‐inferiority testing are discussed that are analogs of the synthesis and fixed‐margin approach. In each of these cases, the model provides a more reliable estimate of the control given its effect in other trials in the network, and, in the case where placebo was only present in the historical trials, the model can predict the effect of the test treatment over placebo as if placebo had been present in the non‐inferiority trial. It can further answer other questions of interest, such as comparative effectiveness of the test treatment among its comparators. More importantly, the model provides an opportunity for disproportionate randomization or the use of small sample sizes by allowing borrowing of information from a network of trials to draw explicit conclusions on non‐inferiority. Copyright © 2015 John Wiley & Sons, Ltd.     

The ABC of non-inferiority margin setting from indirect comparisons

In a non-inferiority trial to assess a new investigative treatment, there may need to be consideration of an indirect comparison with placebo using the active control in the current trial. We can, therefore, use the fact that there is a common active control in the comparisons of the investigative treatment and placebo. In analysing a non-inferiority trial, the ABC of: Assay sensitivity, Bias minimisation and Constancy assumption needs to be considered. It is highlighted how the ABC assumptions can potentially fail when there is placebo creep or a patient population shift. In this situation, the belief about the placebo response expressed in terms of a prior probability in Bayesian formulation could be used with the observed treatment effects to set the non-inferiority limit.     

A unifying approach to non-inferiority, equivalence and superiority tests via multiple decision processes

Two approaches of multiple decision processes are proposed for unifying the non-inferiority, equivalence and superiority tests in a comparative clinical trial for a new drug against an active control. One is a method of confidence set with confidence coefficient 0.95 improving the conventional 0.95 confidence interval in the producer's risk and also the consumer's risk in some cases. It requires to include 0 within the region as well as to clear the non-inferiority margin so that a trial with somewhat large number of subjects and inappropriately large non-inferiority margin for proving non-inferiority of a drug that is actually inferior should be unsuccessful. The other is the closed testing procedure which combines the one- and two-sided tests by applying the partitioning principle and justifies the switching procedure by unifying the non-inferiority, equivalence and superiority tests. In particular regarding the non-inferiority, the proposed method justifies simultaneously the old Japanese Statistical Guideline (one-sided 0.05 test) and the International Guideline ICH E9 (one-sided 0.025 test). The method is particularly attractive, changing the strength of the evidence of relative efficacy of the test drug against a control at five levels according to the achievement of the clinical trial. The meaning of the non-inferiority test and also the rationale of switching from it to superiority test will be discussed.     

Indirect comparisons in the comparative efficacy and non-inferiority settings

International Conference on Harmonization E10 concerns non-inferiority trials and the assessment of comparative efficacy, both of which often involve indirect comparisons. In the non-inferiority setting, there are clinical trial results directly comparing an experimental treatment with an active control, and clinical trial results directly comparing the active control with placebo, and there is an interest in the indirect comparison of the experimental treatment with placebo. In the comparative efficacy setting, there may be separate clinical trial results comparing each of two treatments with placebo, and there is interest in an indirect comparison of the treatments. First, we show that the sample size required for a trial intended to demonstrate superiority through an indirect comparison is always greater than the sample size required for a direct comparison. In addition, by introducing the concept of preservation of effect, we show that the hypothesis addressed in the two settings is identical. Our main result concerns the logical inconsistency between a reasonable criterion for preference of an experimental treatment to a standard treatment and existing regulatory guidance for approval of the experimental treatment on the basis of an indirect comparison. Specifically, the preferred treatment will not always meet the criterion for regulatory approval. This is due to the fact that the experimental treatment bears the burden of overcoming the uncertainty in the effect of the standard treatment. We consider an alternative approval criterion that avoids this logical inconsistency.     

Sample size calculations for noninferiority trials for time-to-event data using the concept of proportional time

Noninferiority trials intend to show that a new treatment is ‘not worse'' than a standard-of-care active control and can be used as an alternative when it is likely to cause fewer side effects compared to the active control. In the case of time-to-event endpoints, existing methods of sample size calculation are done either assuming proportional hazards between the two study arms, or assuming exponentially distributed lifetimes. In scenarios where these assumptions are not true, there are few reliable methods for calculating the sample sizes for a time-to-event noninferiority trial. Additionally, the choice of the non-inferiority margin is obtained either from a meta-analysis of prior studies, or strongly justifiable ‘expert opinion'', or from a ‘well conducted'' definitive large-sample study. Thus, when historical data do not support the traditional assumptions, it would not be appropriate to use these methods to design a noninferiority trial. For such scenarios, an alternate method of sample size calculation based on the assumption of Proportional Time is proposed. This method utilizes the generalized gamma ratio distribution to perform the sample size calculations. A practical example is discussed, followed by insights on choice of the non-inferiority margin, and the indirect testing of superiority of treatment compared to placebo.KEYWORDS: Generalized gamma, noninferiority, non-proportional hazards, proportional time, relative time, sample size     

Design and analysis of a 3‐arm noninferiority trial with a prespecified margin for the hazard ratio

A 3‐arm trial design that includes an experimental treatment, an active reference treatment, and a placebo is useful for assessing the noninferiority of an experimental treatment. The inclusion of a placebo arm enables the assessment of assay sensitivity and internal validation, in addition to the testing of the noninferiority of the experimental treatment compared with the reference treatment. In 3‐arm noninferiority trials, various statistical test procedures have been considered to evaluate the following 3 hypotheses: (i) superiority of the experimental treatment over the placebo, (ii) superiority of the reference treatment over the placebo, and (iii) noninferiority of the experimental treatment compared with the reference treatment. However, hypothesis (ii) can be insufficient and may not accurately assess the assay sensitivity for the noninferiority of the experimental treatment compared with the reference treatment. Thus, demonstrating that the superiority of the reference treatment over the placebo is greater than the noninferiority margin (the nonsuperiority of the reference treatment compared with the placebo) can be necessary. Here, we propose log‐rank statistical procedures for evaluating data obtained from 3‐arm noninferiority trials to assess assay sensitivity with a prespecified margin Δ. In addition, we derive the approximate sample size and optimal allocation required to minimize the total sample size and that of the placebo treatment sample size, hierarchically.     

A Bayesian approach to the statistical analysis of device preference studies

Drug delivery devices are required to have excellent technical specifications to deliver drugs accurately, and in addition, the devices should provide a satisfactory experience to patients because this can have a direct effect on drug compliance. To compare patients' experience with two devices, cross-over studies with patient-reported outcomes (PRO) as response variables are often used. Because of the strength of cross-over designs, each subject can directly compare the two devices by using the PRO variables, and variables indicating preference (preferring A, preferring B, or no preference) can be easily derived. Traditionally, methods based on frequentist statistics can be used to analyze such preference data, but there are some limitations for the frequentist methods. Recently, Bayesian methods are considered an acceptable method by the US Food and Drug Administration to design and analyze device studies. In this paper, we propose a Bayesian statistical method to analyze the data from preference trials. We demonstrate that the new Bayesian estimator enjoys some optimal properties versus the frequentist estimator.     

Information fraction estimation: Strategies for a phase 3 non-inferiority maximum duration design with time to event outcome

There is considerable debate surrounding the choice of methods to estimate information fraction for futility monitoring in a randomized non-inferiority maximum duration trial. This question was motivated by a pediatric oncology study that aimed to establish non-inferiority for two primary outcomes. While non-inferiority was determined for one outcome, the futility monitoring of the other outcome failed to stop the trial early, despite accumulating evidence of inferiority. For a one-sided trial design for which the intervention is inferior to the standard therapy, futility monitoring should provide the opportunity to terminate the trial early. Our research focuses on the Total Control Only (TCO) method, which is defined as a ratio of observed events to total events exclusively within the standard treatment regimen. We investigate its properties in stopping a trial early in favor of inferiority. Simulation results comparing the TCO method with alternative methods, one based on the assumption of an inferior treatment effect (TH0), and the other based on a specified hypothesis of a non-inferior treatment effect (THA), were provided under various pediatric oncology trial design settings. The TCO method is the only method that provides unbiased information fraction estimates regardless of the hypothesis assumptions and exhibits a good power and a comparable type I error rate at each interim analysis compared to other methods. Although none of the methods is uniformly superior on all criteria, the TCO method possesses favorable characteristics, making it a compelling choice for estimating the information fraction when the aim is to reduce cancer treatment-related adverse outcomes.     

Determining the non-inferiority margin for patient reported outcomes

One of the cornerstones of any non-inferiority trial is the choice of the non-inferiority margin delta. This threshold of clinical relevance is very difficult to determine, and in practice, delta is often "negotiated" between the sponsor of the trial and the regulatory agencies. However, for patient reported, or more precisely patient observed outcomes, the patients' minimal clinically important difference (MCID) can be determined empirically by relating the treatment effect, for example, a change on a 100-mm visual analogue scale, to the patient's satisfaction with the change. This MCID can then be used to define delta. We used an anchor-based approach with non-parametric discriminant analysis and ROC analysis and a distribution-based approach with Norman's half standard deviation rule to determine delta in three examples endometriosis-related pelvic pain measured on a 100-mm visual analogue scale, facial acne measured by lesion counts, and hot flush counts. For each of these examples, all three methods yielded quite similar results. In two of the cases, the empirically derived MCIDs were smaller or similar of deltas used before in non-inferiority trials, and in the third case, the empirically derived MCID was used to derive a responder definition that was accepted by the FDA. In conclusion, for patient-observed endpoints, the delta can be derived empirically. In our view, this is a better approach than that of asking the clinician for a "nice round number" for delta, such as 10, 50%, π, e, or i.     

The Rheumatoid Arthritis Drug Development Model: a case study in Bayesian clinical trial simulation

The development of a new drug is a major undertaking and it is important to consider carefully the key decisions in the development process. Decisions are made in the presence of uncertainty and outcomes such as the probability of successful drug registration depend on the clinical development programmme. The Rheumatoid Arthritis Drug Development Model was developed to support key decisions for drugs in development for the treatment of rheumatoid arthritis. It is configured to simulate Phase 2b and 3 trials based on the efficacy of new drugs at the end of Phase 2a, evidence about the efficacy of existing treatments, and expert opinion regarding key safety criteria. The model evaluates the performance of different development programmes with respect to the duration of disease of the target population, Phase 2b and 3 sample sizes, the dose(s) of the experimental treatment, the choice of comparator, the duration of the Phase 2b clinical trial, the primary efficacy outcome and decision criteria for successfully passing Phases 2b and 3. It uses Bayesian clinical trial simulation to calculate the probability of successful drug registration based on the uncertainty about parameters of interest, thereby providing a more realistic assessment of the likely outcomes of individual trials and sequences of trials for the purpose of decision making. In this case study, the results show that, depending on the trial design, the new treatment has assurances of successful drug registration in the range 0.044–0.142 for an ACR20 outcome and 0.057–0.213 for an ACR50 outcome. Copyright © 2009 John Wiley & Sons, Ltd.     

Sample size estimation for non-inferiority trials of time-to-event data

We consider the problem of sample size calculation for non-inferiority based on the hazard ratio in time-to-event trials where overall study duration is fixed and subject enrollment is staggered with variable follow-up. An adaptation of previously developed formulae for the superiority framework is presented that specifically allows for effect reversal under the non-inferiority setting, and its consequent effect on variance. Empirical performance is assessed through a small simulation study, and an example based on an ongoing trial is presented. The formulae are straightforward to program and may prove a useful tool in planning trials of this type.     

A comparative study of adaptive trial designs for dose optimization

Recently, the US Food and Drug Administration Oncology Center of Excellence initiated Project Optimus to reform the dose optimization and dose selection paradigm in oncology drug development. The agency pointed out that the current paradigm for dose selection—based on the maximum tolerated dose (MTD)—is not sufficient for molecularly targeted therapies and immunotherapies, for which efficacy may not increase after the dose reaches a certain level. In these cases, it is more appropriate to identify the optimal biological dose (OBD) that optimizes the risk–benefit tradeoff of the drug. Project Optimus has spurred tremendous interest and urgent need for guidance on designing dose optimization trials. In this article, we review several representative dose optimization designs, including model-based and model-assisted designs, and compare their operating characteristics based on 10,000 randomly generated scenarios with various dose-toxicity and dose-efficacy curves and some fixed representative scenarios. The results show that, compared with model-based designs, model-assisted methods have advantages of easy-to-implement, robustness, and high accuracy to identify OBD. Some guidance is provided to help biostatisticians and clinicians to choose appropriate dose optimization methods in practice.     

Letter to the editor by the authors of Exact Calculation of Power and Sample Size in Bioequivalence Studies Using Two One‐sided Tests,Pharmaceutical Statistics,DOI: 10.1002/pst.1666

This article reflects the views of the authors and should not be construed to be those of the US Food and Drug Administration. Copyright © 2015 John Wiley & Sons, Ltd.     

Multistate modeling of clinical hold in randomized clinical trials

A clinical hold order by the Food and Drug Administration (FDA) to the sponsor of a clinical trial is a measure to delay a proposed or to suspend an ongoing clinical investigation. The phase III clinical trial START serves as motivating data example to explore implications and potential statistical approaches for a trial continuing after a clinical hold is lifted. In spite of a modified intention‐to‐treat (ITT) analysis introduced to account for the clinical hold by excluding patients potentially affected most by the clinical hold, results of the trial did not show a significant improvement of overall survival duration, and the question remains whether the negative result was an effect of the clinical hold. In this paper, we propose a multistate model incorporating the clinical hold as well as disease progression as intermediate events to investigate the impact of the clinical hold on the treatment effect. Moreover, we consider a simple counterfactual censoring approach as alternative strategy to the modified ITT analysis to deal with a clinical hold. Using a realistic simulation study informed by the START data and with a design based on our multistate model, we show that the modified ITT analysis used in the START trial was reasonable. However, the censoring approach will be shown to have some benefits in terms of power and flexibility.     

Assessing the treatment effect in a randomized controlled trial with extensive non‐adherence: the EVOLVE trial

Intention‐to‐treat (ITT) analysis is widely used to establish efficacy in randomized clinical trials. However, in a long‐term outcomes study where non‐adherence to study drug is substantial, the on‐treatment effect of the study drug may be underestimated using the ITT analysis. The analyses presented herein are from the EVOLVE trial, a double‐blind, placebo‐controlled, event‐driven cardiovascular outcomes study conducted to assess whether a treatment regimen including cinacalcet compared with placebo in addition to other conventional therapies reduces the risk of mortality and major cardiovascular events in patients receiving hemodialysis with secondary hyperparathyroidism. Pre‐specified sensitivity analyses were performed to assess the impact of non‐adherence on the estimated effect of cinacalcet. These analyses included lag‐censoring, inverse probability of censoring weights (IPCW), rank preserving structural failure time model (RPSFTM) and iterative parameter estimation (IPE). The relative hazard (cinacalcet versus placebo) of mortality and major cardiovascular events was 0.93 (95% confidence interval 0.85, 1.02) using the ITT analysis; 0.85 (0.76, 0.95) using lag‐censoring analysis; 0.81 (0.70, 0.92) using IPCW; 0.85 (0.66, 1.04) using RPSFTM and 0.85 (0.75, 0.96) using IPE. These analyses, while not providing definitive evidence, suggest that the intervention may have an effect while subjects are receiving treatment. The ITT method remains the established method to evaluate efficacy of a new treatment; however, additional analyses should be considered to assess the on‐treatment effect when substantial non‐adherence to study drug is expected or observed. Copyright © 2015 John Wiley & Sons, Ltd.     

A case study in identifying targeted patients population in major depressive disorder by enhanced enrichment design

Despite advances in clinical trial design, failure rates near 80% in phase 2 and 50% in phase 3 have recently been reported. The challenges to successful drug development are particularly acute in central nervous system trials such as for pain, schizophrenia, mania, and depression because high‐placebo response rates lessen assay sensitivity, diminish estimated treatment effect sizes, and thereby decrease statistical power. This paper addresses the importance of rigorous patient selection in major depressive disorder trials through an enhanced enrichment paradigm. This approach led to a redefinition of an ongoing, blinded phase 3 trial algorithm for patient inclusion (1) to eliminate further randomization of transient placebo responders and (2) to exclude previously randomized transient responders from the primary analysis of the double blind phase of the trial. It is illustrated for a case study for the comparison between brexpiprazole + antidepressant therapy and placebo + antidepressant therapy. Analysis of the primary endpoint showed that efficacy of brexpiprazole versus placebo could not be established statistically if the original algorithm for identification of placebo responders was used, but the enhanced enrichment approach did statistically demonstrate efficacy. Additionally, the enhanced enrichment approach identified a target population with a clinically meaningful treatment effect. Through its successful identification of a target population, the innovative enhanced enrichment approach enabled the demonstration of a positive treatment effect in a very challenging area of depression research.