Step-up and step-down methods for testing multiple hypotheses in sequential experiments

Sequential methods are developed for testing multiple hypotheses, resulting in a statistical decision for each individual test and controlling the familywise error rate and the familywise power in the strong sense. Extending the ideas of step-up and step-down methods for multiple comparisons to sequential designs, the new techniques improve over the Bonferroni and closed testing methods proposed earlier by a substantial reduction of the expected sample size.     

Testing non-inferiority and superiority for two endpoints for several treatments with a control

Some multiple comparison procedures are described for multiple armed studies. The procedures are appropriate for testing all hypotheses for comparing two endpoints and multiple test arms to a single control group, for example three different fixed doses compared to a placebo. The procedure assumes that among the two endpoints, one is designated as a primary endpoint such that for a given treatment arm, no hypothesis for the secondary endpoint can be rejected unless the hypothesis for the primary endpoint was rejected. The procedures described control the family-wise error rate in the strong sense at a specified level α.     

An empirical Bayes approach to evaluation of results for a specific region in multiregional clinical trials

To accelerate the drug development process and shorten approval time, the design of multiregional clinical trials (MRCTs) incorporates subjects from many countries/regions around the world under the same protocol. After showing the overall efficacy of a drug in all global regions, one can also simultaneously evaluate the possibility of applying the overall trial results to all regions and subsequently support drug registration in each of them. In this paper, we focus on a specific region and establish a statistical criterion to assess the consistency between the specific region and overall results in an MRCT. More specifically, we treat each region in an MRCT as an independent clinical trial, and each perhaps has different treatment effect. We then construct the empirical prior information for the treatment effect for the specific region on the basis of all of the observed data from other regions. We will conclude similarity between the specific region and all regions if the posterior probability of deriving a positive treatment effect in the specific region is large, say 80%. Numerical examples illustrate applications of the proposed approach in different scenarios. Copyright © 2013 John Wiley & Sons, Ltd.     

Competing risk analysis in a large cardiovascular clinical trial: An APEX substudy

Competing risk methods are time‐to‐event analyses that account for fatal and/or nonfatal events that may potentially alter or prevent a subject from experiencing the primary endpoint. Competing risk methods may provide a more accurate and less biased estimate of the incidence of an outcome but are rarely applied in cardiology trials. APEX investigated the efficacy of extended‐duration betrixaban versus standard‐duration enoxaparin to prevent a composite of symptomatic deep‐vein thrombosis (proximal or distal), nonfatal pulmonary embolism, or venous thromboembolism (VTE)–related death in acute medically ill patients (n = 7513). The aim of the current analysis was to determine the efficacy of betrixaban vs standard‐duration enoxaparin accounting for non‐VTE–related deaths using the Fine and Gray method for competing risks. The proportion of non‐VTE–related death was similar in both the betrixaban (133, 3.6%) and enoxaparin (136, 3.7%) arms, P = .85. Both the traditional Kaplan‐Meier method and the Fine and Gray method accounting for non‐VTE–related death as a competing risk showed equal reduction of VTE events when comparing betrixaban to enoxaparin (HR/SHR = 0.65, 95% 0.42‐0.99, P = 0.046). Due to the similar proportion of non‐VTE–related deaths in both treatment arms and the use of a univariate model, the Fine and Gray method provided identical results to the traditional Cox model. Using the Fine and Gray method in addition to the traditional Cox proportional hazards method can indicate whether the presence of a competing risk, which is dependent of the outcome, altered the risk estimate.     

An evaluation of the trimmed mean approach in clinical trials with dropout

The trimmed mean is a method of dealing with patient dropout in clinical trials that considers early discontinuation of treatment a bad outcome rather than leading to missing data. The present investigation is the first comprehensive assessment of the approach across a broad set of simulated clinical trial scenarios. In the trimmed mean approach, all patients who discontinue treatment prior to the primary endpoint are excluded from analysis by trimming an equal percentage of bad outcomes from each treatment arm. The untrimmed values are used to calculated means or mean changes. An explicit intent of trimming is to favor the group with lower dropout because having more completers is a beneficial effect of the drug, or conversely, higher dropout is a bad effect. In the simulation study, difference between treatments estimated from trimmed means was greater than the corresponding effects estimated from untrimmed means when dropout favored the experimental group, and vice versa. The trimmed mean estimates a unique estimand. Therefore, comparisons with other methods are difficult to interpret and the utility of the trimmed mean hinges on the reasonableness of its assumptions: dropout is an equally bad outcome in all patients, and adherence decisions in the trial are sufficiently similar to clinical practice in order to generalize the results. Trimming might be applicable to other inter‐current events such as switching to or adding rescue medicine. Given the well‐known biases in some methods that estimate effectiveness, such as baseline observation carried forward and non‐responder imputation, the trimmed mean may be a useful alternative when its assumptions are justifiable.     

Practical usage of O'Brien's OLS and GLS statistics in clinical trials

Multivariate techniques of O'Brien's OLS and GLS statistics are discussed in the context of their application in clinical trials. We introduce the concept of an operational effect size and illustrate its use to evaluate power. An extension describing how to handle covariates and missing data is developed in the context of Mixed models. This extension allowing adjustment for covariates is easily programmed in any statistical package including SAS. Monte Carlo simulation is used for a number of different sample sizes to compare the actual size and power of the tests based on O'Brien's OLS and GLS statistics.     

How to avoid concerns with the interpretation of two primary endpoints if significant superiority in one is sufficient for formal proof of efficacy

Formal proof of efficacy of a drug requires that in a prospective experiment, superiority over placebo, or either superiority or at least non-inferiority to an established standard, is demonstrated. Traditionally one primary endpoint is specified, but various diseases exist where treatment success needs to be based on the assessment of two primary endpoints. With co-primary endpoints, both need to be “significant” as a prerequisite to claim study success. Here, no adjustment of the study-wise type-1-error is needed, but sample size is often increased to maintain the pre-defined power. Studies that use an at-least-one concept have been proposed where study success is claimed if superiority for at least one of the endpoints is demonstrated. This is sometimes also called the dual primary endpoint concept, and an appropriate adjustment of the study-wise type-1-error is required. This concept is not covered in the European Guideline on multiplicity because study success can be claimed if one endpoint shows significant superiority, despite a possible deterioration in the other. In line with Röhmel's strategy, we discuss an alternative approach including non-inferiority hypotheses testing that avoids obvious contradictions to proper decision-making. This approach leads back to the co-primary endpoint assessment, and has the advantage that minimum requirements for endpoints can be modeled flexibly for several practical needs. Our simulations show that, if planning assumptions are correct, the proposed additional requirements improve interpretation with only a limited impact on power, that is, on sample size.     

A modified combination test for the analysis of a clinical trial when a protocol amendment changed the inclusion criteria

Protocol amendments are often necessary in clinical trials. They can change the entry criteria and, therefore, the population. Simply analysing the pooled data is not acceptable. Instead, each phase should be analysed separately and a combination test such as Fisher's test should be applied to the resulting p-values. In this situation, an asymmetric decision rule is not appropriate. Therefore, we propose a modification of Bauer and Köhne's test. We compare this new test with the tests of Liptak, Fisher, Bauer/Köhne and Edgington. In case of differences in variance only or only small differences in mean, Liptak's Z-score approach is the best, and the new test keeps up with the rest and is in most cases slightly superior. In other situations, the new test and the Z-score approach are not preferable. But no big differences in populations are usually to be expected due to amendments. Then, the new method is a recommendable alternative.     

An exact test for comparing two predictive values in small‐size clinical trials

Positive and negative predictive values describe the performance of a diagnostic test. There are several methods to test the equality of predictive values in paired designs. However, these methods were premised on large sample theory, and they may not be suitable for small‐size clinical trials because of inflation of the type 1 error rate. In this study, we propose an exact test to control the type 1 error rate strictly for conducting a small‐size clinical trial that investigates the equality of predictive values in paired designs. In addition, we execute simulation studies to evaluate the performance of the proposed exact test and existing methods in small‐size clinical trials. The proposed test can calculate the exact P value, and as a result of simulations, the empirical type 1 error rate for the proposed test did not exceed the significance level regardless of the setting, and the empirical power for the proposed test is not much different from the other methods based on large‐sample theory. Therefore, it is considered that the proposed exact test is useful when the type 1 error rate needs to be controlled strictly.     

Penalized likelihood ratio test for a biomarker threshold effect in clinical trials based on generalized linear models

In a clinical trial, the responses to the new treatment may vary among patient subsets with different characteristics in a biomarker. It is often necessary to examine whether there is a cutpoint for the biomarker that divides the patients into two subsets of those with more favourable and less favourable responses. More generally, we approach this problem as a test of homogeneity in the effects of a set of covariates in generalized linear regression models. The unknown cutpoint results in a model with nonidentifiability and a nonsmooth likelihood function to which the ordinary likelihood methods do not apply. We first use a smooth continuous function to approximate the indicator function defining the patient subsets. We then propose a penalized likelihood ratio test to overcome the model irregularities. Under the null hypothesis, we prove that the asymptotic distribution of the proposed test statistic is a mixture of chi-squared distributions. Our method is based on established asymptotic theory, is simple to use, and works in a general framework that includes logistic, Poisson, and linear regression models. In extensive simulation studies, we find that the proposed test works well in terms of size and power. We further demonstrate the use of the proposed method by applying it to clinical trial data from the Digitalis Investigation Group (DIG) on heart failure.     

Non-parametric testing for the number of change points in a sequence of independent random variables

A non-parametric procedure is derived for testing for the number of change points in a sequence of independent continuously distributed variables when there is no prior information available. The procedure is based on the Kruskal–Wallis test, which is maximized as a function of all possible places of the change points. The procedure consists of a sequence of non-parametric tests of nested hypotheses corresponding to a decreasing number of change points. The properties of this procedure are analyzed by Monte Carlo methods and compared to a parametric procedure for the case that the variables are exponentially distributed. The critical values are given for sample sizes up to 200.     

An evaluation of change-point estimators for a sequence of normal observations with unknown parameters

Performance of maximum likelihood estimators (MLE) of the change-point in normal series is evaluated considering three scenarios where process parameters are assumed to be unknown. Different shifts, sample sizes, and locations of a change-point were tested. A comparison is made with estimators based on cumulative sums and Bartlett's test. Performance analysis done with extensive simulations for normally distributed series showed that the MLEs perform better (or equal) in almost every scenario, with smaller bias and standard error. In addition, robustness of MLE to non-normality is also studied.     

An evaluation of common methods for dichotomization of continuous variables to discriminate disease status

Dichotomization of continuous variables to discriminate a dichotomous outcome is often useful in statistical applications. If a true threshold for a continuous variable exists, the challenge is identifying it. This paper examines common methods for dichotomization to identify which ones recover a true threshold. We provide mathematical and numeric proofs demonstrating that maximizing the odds ratio, Youden’s statistic, Gini Index, chi-square statistic, relative risk and kappa statistic all theoretically recover a true threshold. A simulation study evaluating the ability of these statistics to recover a threshold when sampling from a population indicates that maximizing the chi-square statistic and Gini Index have the smallest bias and variability when the probability of being larger than the threshold is small while maximizing Kappa or Youden’s statistics is best when this probability is larger. Maximizing odds ratio is the most variable and biased of the methods.     

An MSE‐reduced estimator for the response proportion in a two‐stage clinical trial

Two‐stage design is very useful in clinical trials for evaluating the validity of a specific treatment regimen. When the second stage is allowed to continue, the method used to estimate the response rate based on the results of both stages is critical for the subsequent design. The often‐used sample proportion has an evident upward bias. However, the maximum likelihood estimator or the moment estimator tends to underestimate the response rate. A mean‐square error weighted estimator is considered here; its performance is thoroughly investigated via Simon's optimal and minimax designs and Shuster's design. Compared with the sample proportion, the proposed method has a smaller bias, and compared with the maximum likelihood estimator, the proposed method has a smaller mean‐square error. Copyright © 2010 John Wiley & Sons, Ltd.     

Enhancement of the adaptive signature design for learning and confirming in a single pivotal trial

Because of the complexity of cancer biology, often the target pathway is not well understood at the time that phase III trials are initiated. A 2‐stage trial design was previously proposed for identifying a subgroup of interest in a learn stage, on the basis of 1 or more baseline biomarkers, and then subsequently confirming it in a confirmation stage. In this article, we discuss some practical aspects of this type of design and describe an enhancement to this approach that can be built into the study randomization to increase the robustness of the evaluation. Furthermore, we show via simulation studies how the proportion of patients allocated to the learn stage versus the confirm stage impacts the power and provide recommendations.     

Group sequential x2 statistic for testing the difference of two mean vectors in clinical trials

In this study we discuss the group sequential procedures for comparing two treatments based on multivariate observations in clinical trials. Also we suppose that a response vector on each of two treatments has a multivariate normal distribution with unknown covariance matrix. Then we propose a group sequential x² statistic in order to carry out repeated significance test for hypothesis of no difference between two population mean vectors. In order to realize the group sequential test where average sample number is reduced, we propose another modified group sequential x² statistic by extension of Jennison and Turnbull ( 1991 ). After construction of repeated confidence boundaries for making the repeated significance test, we compare two group sequential procedures based on two statistics regarding the average sample number and the power of the test in the simulations.     

An extensive power evaluation of a novel two-sample density-based empirical likelihood ratio test for paired data with an application to a treatment study of attention-deficit/hyperactivity disorder and severe mood dysregulation

In many case-control studies, it is common to utilize paired data when treatments are being evaluated. In this article, we propose and examine an efficient distribution-free test to compare two independent samples, where each is based on paired observations. We extend and modify the density-based empirical likelihood ratio test presented by Gurevich and Vexler [7] to formulate an appropriate parametric likelihood ratio test statistic corresponding to the hypothesis of our interest and then to approximate the test statistic nonparametrically. We conduct an extensive Monte Carlo study to evaluate the proposed test. The results of the performed simulation study demonstrate the robustness of the proposed test with respect to values of test parameters. Furthermore, an extensive power analysis via Monte Carlo simulations confirms that the proposed method outperforms the classical and general procedures in most cases related to a wide class of alternatives. An application to a real paired data study illustrates that the proposed test can be efficiently implemented in practice.     

Integrating phase 2 into phase 3 based on an intermediate endpoint while accounting for a cure proportion—With an application to the design of a clinical trial in acute myeloid leukemia

For a trial with primary endpoint overall survival for a molecule with curative potential, statistical methods that rely on the proportional hazards assumption may underestimate the power and the time to final analysis. We show how a cure proportion model can be used to get the necessary number of events and appropriate timing via simulation. If phase 1 results for the new drug are exceptional and/or the medical need in the target population is high, a phase 3 trial might be initiated after phase 1. Building in a futility interim analysis into such a pivotal trial may mitigate the uncertainty of moving directly to phase 3. However, if cure is possible, overall survival might not be mature enough at the interim to support a futility decision. We propose to base this decision on an intermediate endpoint that is sufficiently associated with survival. Planning for such an interim can be interpreted as making a randomized phase 2 trial a part of the pivotal trial: If stopped at the interim, the trial data would be analyzed, and a decision on a subsequent phase 3 trial would be made. If the trial continues at the interim, then the phase 3 trial is already underway. To select a futility boundary, a mechanistic simulation model that connects the intermediate endpoint and survival is proposed. We illustrate how this approach was used to design a pivotal randomized trial in acute myeloid leukemia and discuss historical data that informed the simulation model and operational challenges when implementing it.     

Review of guidelines and literature for handling missing data in longitudinal clinical trials with a case study

Missing data in clinical trials are inevitable. We highlight the ICH guidelines and CPMP points to consider on missing data. Specifically, we outline how we should consider missing data issues when designing, planning and conducting studies to minimize missing data impact. We also go beyond the coverage of the above two documents, provide a more detailed review of the basic concepts of missing data and frequently used terminologies, and examples of the typical missing data mechanism, and discuss technical details and literature for several frequently used statistical methods and associated software. Finally, we provide a case study where the principles outlined in this paper are applied to one clinical program at protocol design, data analysis plan and other stages of a clinical trial.