Pivotal Quantities Based on Sequential Data: A Bootstrap Approach

A bootstrap algorithm is provided for obtaining a confidence interval for the mean of a probability distribution when sequential data are considered. For this kind of data the empirical distribution can be biased but its bias is bounded by the coefficient of variation of the stopping rule associated with the sequential procedure. When using this distribution for resampling the validity of the bootstrap approach is established by means of a series expansion of the corresponding pivotal quantity. A simulation study is carried out using Wang and Tsiatis type tests and considering the normal and exponential distributions to generate the data. This study confirms that for moderate coefficients of variation of the stopping rule, the bootstrap method allows adequate confidence intervals for the parameters to be obtained, whichever is the distribution of data.     

A global stopping rule for recursive density estimators

In this paper asymptotic sequential fixed-width confidence bounds for an unknown density on the real line, based on integrated squared error, are studied. Using a sequence of Wolverton-Wagner kernel estimators, two classes of stopping rules are established.By the same approach, analogous results can be provided for other types of recursive density estimators.     

A flexible multi-stage design for phase II oncology trials

Phase II trials evaluate whether a new drug or a new therapy is worth further pursuing or certain treatments are feasible or not. A typical phase II is a single arm (open label) trial with a binary clinical endpoint (response to therapy). Although many oncology Phase II clinical trials are designed with a two-stage procedure, multi-stage design for phase II cancer clinical trials are now feasible due to increased capability of data capture. Such design adjusts for multiple analyses and variations in analysis time, and provides greater flexibility such as minimizing the number of patients treated on an ineffective therapy and identifying the minimum number of patients needed to evaluate whether the trial would warrant further development. In most of the NIH sponsored studies, the early stopping rule is determined so that the number of patients treated on an ineffective therapy is minimized. In pharmaceutical trials, it is also of importance to know as early as possible if the trial is highly promising and what is the likelihood the early conclusion can sustain. Although various methods are available to address these issues, practitioners often use disparate methods for addressing different issues and do not realize a single unified method exists. This article shows how to utilize a unified approach via a fully sequential procedure, the sequential conditional probability ratio test, to address the multiple needs of a phase II trial. We show the fully sequential program can be used to derive an optimized efficient multi-stage design for either a low activity or a high activity, to identify the minimum number of patients required to assess whether a new drug warrants further study and to adjust for unplanned interim analyses. In addition, we calculate a probability of discordance that the statistical test will conclude otherwise should the trial continue to the planned end that is usually at the sample size of a fixed sample design. This probability can be used to aid in decision making in a drug development program. All computations are based on exact binomial distribution.     

An Introduction to Practical Sequential Inferences via Single-Arm Binary Response Studies Using the binseqtest R Package

We review sequential designs, including group sequential and two-stage designs, for testing or estimating a single binary parameter. We use this simple case to introduce ideas common to many sequential designs, which in this case can be explained without explicitly using stochastic processes. We focus on methods provided by our newly developed R package, binseqtest, which exactly bound the Type I error rate of tests and exactly maintain proper coverage of confidence intervals. Within this framework, we review some allowable practical adaptations of the sequential design. We explore issues such as the following: How should the design be modified if no assessment was made at one of the planned sequential stopping times? How should the parameter be estimated if the study needs to be stopped early? What reasons for stopping early are allowed? How should inferences be made when the study is stopped for crossing the boundary, but later information is collected about responses of subjects that had enrolled before the decision to stop but had not responded by that time? Answers to these questions are demonstrated using basic methods that are available in our binseqtest R package. Supplementary materials for this article are available online.     

Optimal group sequential designs constrained on both overall and stage one error rates

Abstract

Optimized group sequential designs proposed in the literature have designs minimizing average sample size with respect to a prior distribution of treatment effect with overall type I and type II error rates well-controlled (i.e., at final stage). The optimized asymmetric group sequential designs that we present here additionally consider constrains on stopping probabilities at stage one: probability of stopping for futility at stage one when no drug effect exists as well as the probability of rejection when the maximum effect size is true at stage one so that accountability of group sequential design is ensured from the first stage throughout.     

The perils with the misuse of predictive power

In early drug development, especially when studying new mechanisms of action or in new disease areas, little is known about the targeted or anticipated treatment effect or variability estimates. Adaptive designs that allow for early stopping but also use interim data to adapt the sample size have been proposed as a practical way of dealing with these uncertainties. Predictive power and conditional power are two commonly mentioned techniques that allow predictions of what will happen at the end of the trial based on the interim data. Decisions about stopping or continuing the trial can then be based on these predictions. However, unless the user of these statistics has a deep understanding of their characteristics important pitfalls may be encountered, especially with the use of predictive power. The aim of this paper is to highlight these potential pitfalls. It is critical that statisticians understand the fundamental differences between predictive power and conditional power as they can have dramatic effects on decision making at the interim stage, especially if used to re-evaluate the sample size. The use of predictive power can lead to much larger sample sizes than either conditional power or standard sample size calculations. One crucial difference is that predictive power takes account of all uncertainty, parts of which are ignored by standard sample size calculations and conditional power. By comparing the characteristics of each of these statistics we highlight important characteristics of predictive power that experimenters need to be aware of when using this approach.     

Choice of alpha spending function and time points in clinical trials with one or two interim analyses

One characterization of group sequential methods uses alpha spending functions to allocate the false positive rate throughout a study. We consider and evaluate several such spending functions as well as the time points of the interim analyses at which they apply. In addition, we evaluate the double triangular test as an alternative procedure that allows for early termination of the trial not only due to efficacy differences between treatments, but also due to lack of such differences. We motivate and illustrate our work by reference to the analysis of survival data from a proposed oncology study. Such group sequential procedures with one or two interim analyses are only slightly less powerful than fixed sample trials, but provide for the strong possibility of early stopping. Therefore, in all situations where they can practically be applied, we recommend their routine use in clinical trials. The double triangular test provides a suitable alternative to the group sequential procedures in that they do not provide for early stopping with acceptance of the null hypothesis. Again, there is only a modest loss in power relative to fixed sample tests. Copyright © 2004 John Wiley & Sons, Ltd.     

Early stopping in seamless phase I/II clinical trials

In recent years, seamless phase I/II clinical trials have drawn much attention, as they consider both toxicity and efficacy endpoints in finding an optimal dose (OD). Engaging an appropriate number of patients in a trial is a challenging task. This paper attempts a dynamic stopping rule to save resources in phase I/II trials. That is, the stopping rule aims to save patients from unnecessary toxic or subtherapeutic doses. We allow a trial to stop early when widths of the confidence intervals for the dose-response parameters become narrower or when the sample size is equal to a predefined size, whichever comes first. The simulation study of dose-response scenarios in various settings demonstrates that the proposed stopping rule can engage an appropriate number of patients. Therefore, we suggest its use in clinical trials.     

A stochastically curtailed two-arm randomised phase II trial design for binary outcomes

Randomised controlled trials are considered the gold standard in trial design. However, phase II oncology trials with a binary outcome are often single-arm. Although a number of reasons exist for choosing a single-arm trial, the primary reason is that single-arm designs require fewer participants than their randomised equivalents. Therefore, the development of novel methodology that makes randomised designs more efficient is of value to the trials community. This article introduces a randomised two-arm binary outcome trial design that includes stochastic curtailment (SC), allowing for the possibility of stopping a trial before the final conclusions are known with certainty. In addition to SC, the proposed design involves the use of a randomised block design, which allows investigators to control the number of interim analyses. This approach is compared with existing designs that also use early stopping, through the use of a loss function comprised of a weighted sum of design characteristics. Comparisons are also made using an example from a real trial. The comparisons show that for many possible loss functions, the proposed design is superior to existing designs. Further, the proposed design may be more practical, by allowing a flexible number of interim analyses. One existing design produces superior design realisations when the anticipated response rate is low. However, when using this design, the probability of rejecting the null hypothesis is sensitive to misspecification of the null response rate. Therefore, when considering randomised designs in phase II, we recommend the proposed approach be preferred over other sequential designs.     

Assessment of futility in clinical trials

The term 'futility' is used to refer to the inability of a clinical trial to achieve its objectives. In particular, stopping a clinical trial when the interim results suggest that it is unlikely to achieve statistical significance can save resources that could be used on more promising research. There are various approaches that have been proposed to assess futility, including stochastic curtailment, predictive power, predictive probability, and group sequential methods. In this paper, we describe and contrast these approaches, and discuss several issues associated with futility analyses, such as ethical considerations, whether or not type I error can or should be reclaimed, one-sided vs two-sided futility rules, and the impact of futility analyses on power.     

Oncology phase II proof‐of‐concept studies with multiple targets: Randomized controlled trial or single arm?

For oncology drug development, phase II proof‐of‐concept studies have played a key role in determining whether or not to advance to a confirmatory phase III trial. With the increasing number of immunotherapies, efficient design strategies are crucial in moving successful drugs quickly to market. Our research examines drug development decision making under the framework of maximizing resource investment, characterized by benefit cost ratios (BCRs). In general, benefit represents the likelihood that a drug is successful, and cost is characterized by the risk adjusted total sample size of the phases II and III studies. Phase III studies often include a futility interim analysis; this sequential component can also be incorporated into BCRs. Under this framework, multiple scenarios can be considered. For example, for a given drug and cancer indication, BCRs can yield insights into whether to use a randomized control trial or a single‐arm study. Importantly, any uncertainty in historical control estimates that are used to benchmark single‐arm studies can be explicitly incorporated into BCRs. More complex scenarios, such as restricted resources or multiple potential cancer indications, can also be examined. Overall, BCR analyses indicate that single‐arm trials are favored for proof‐of‐concept trials when there is low uncertainty in historical control data and smaller phase III sample sizes. Otherwise, especially if the most likely to succeed tumor indication can be identified, randomized controlled trials may be a better option. While the findings are consistent with intuition, we provide a more objective approach.     

A sequential conditional probability ratio test procedure for comparing diagnostic tests

In this paper, we derive sequential conditional probability ratio tests to compare diagnostic tests without distributional assumptions on test results. The test statistics in our method are nonparametric weighted areas under the receiver-operating characteristic curves. By using the new method, the decision of stopping the diagnostic trial early is unlikely to be reversed should the trials continue to the planned end. The conservatism reflected in this approach to have more conservative stopping boundaries during the course of the trial is especially appealing for diagnostic trials since the end point is not death. In addition, the maximum sample size of our method is not greater than a fixed sample test with similar power functions. Simulation studies are performed to evaluate the properties of the proposed sequential procedure. We illustrate the method using data from a thoracic aorta imaging study.     

Sequential estimation for semiparametric models with application to the proportional hazards model

In this paper, we show that if the Euclidean parameter of a semiparametric model can be estimated through an estimating function, we can extend straightforwardly conditions by Dmitrienko and Govindarajulu [2000. Ann. Statist. 28 (5), 1472–1501] in order to prove that the estimator indexed by any regular sequence (sequential estimator), has the same asymptotic behavior as the non-sequential estimator. These conditions also allow us to obtain the asymptotic normality of the stopping rule, for the special case of sequential confidence sets. These results are applied to the proportional hazards model, for which we show that after slight modifications, the classical assumptions given by Andersen and Gill [1982. Ann. Statist. 10(4), 1100–1120] are sufficient to obtain the asymptotic behavior of the sequential version of the well-known [Cox, 1972. J. Roy. Statist. Soc. Ser. B (34), 187–220] partial maximum likelihood estimator. To prove this result we need to establish a strong convergence result for the regression parameter estimator, involving mainly exponential inequalities for both continuous martingales and some basic empirical processes. A typical example of a fixed-width confidence interval is given and illustrated by a Monte Carlo study.     

On Comparison of Stopping Times in Sequential Procedures for Exponential Families of Stochastic Processes

For curved ( k + 1), k -exponential families of stochastic processes a natural and often studied sequential procedure is to stop observation when a linear combination of the coordinates of the canonical process crosses a prescribed level. For such procedures the model is, approximately or exactly, a non-curved exponential family. Subfamilies of these stopping rules defined by having the same Fisher (expected) information are considered. Within a subfamily the Bartlett correction for a point hypothesis is also constant. Methods for comparing the durations of the sampling periods for the stopping rules in such a subfamily are discussed. It turns out that some stopping times tend to be smaller than others. For exponential families of diffusions and of counting processes the probability that one such stopping time is smaller than another can be given explicity. More generally, an Edgeworth expansion of this probability is given     

Point estimates and confidence regions for sequential trials involving selection

A number of authors have proposed clinical trial designs involving the comparison of several experimental treatments with a control treatment in two or more stages. At the end of the first stage, the most promising experimental treatment is selected, and all other experimental treatments are dropped from the trial. Provided it is good enough, the selected experimental treatment is then compared with the control treatment in one or more subsequent stages. The analysis of data from such a trial is problematic because of the treatment selection and the possibility of stopping at interim analyses. These aspects lead to bias in the maximum-likelihood estimate of the advantage of the selected experimental treatment over the control and to inaccurate coverage for the associated confidence interval. In this paper, we evaluate the bias of the maximum-likelihood estimate and propose a bias-adjusted estimate. We also propose an approach to the construction of a confidence region for the vector of advantages of the experimental treatments over the control based on an ordering of the sample space. These regions are shown to have accurate coverage, although they are also shown to be necessarily unbounded. Confidence intervals for the advantage of the selected treatment are obtained from the confidence regions and are shown to have more accurate coverage than the standard confidence interval based upon the maximum-likelihood estimate and its asymptotic standard error.     

Hybrid regions for post-change mean in a sequence of normal variables

The hybrid bootstrap uses resampling ideas to extend the duality approach to the interval estimation for a parameter of interest when there are nuisance parameters. The confidence region constructed by the hybrid bootstrap may perform much better than the ordinary bootstrap region in a situation where the data provide substantial information about the nuisance parameter, but limited information about the parameter of interest. We apply this method to estimate the post-change mean after a change is detected by a stopping procedure in a sequence of independent normal variables. Since distribution theory in change point problems is generally a challenge, we use bootstrap simulation to find empirical distributions of test statistics and calculate critical thresholds. Both likelihood ratio and Bayesian test statistics are considered to set confidence regions for post-change means in the normal model. In the simulation studies, the performance of hybrid regions are compared with that of ordinary bootstrap regions in terms of the widths and coverage probabilities of confidence intervals.     

Fixed size confidence regions for parameters of threshold AR(1) models

For parameters of single and multiple threshold autoregressive models of order one, sequential procedures are proposed for constructing fixed size confidence ellipsoids. Sequential procedures are also proposed for constructing fixed proportional accuracy confidence ellipsoids and fixed width confidence intervals for linear combination of parameters. The confidence ellipsoids and intervals are shown to be asymptotically consistent and the associated stopping rules are shown to be asymptotically efficient as the size/width of the region becomes small.     

Repeated Confidence Intervals Under Fractional Brownian Motion in Long-Term Clinical Trials

Repeated confidence interval (RCI) is an important tool for design and monitoring of group sequential trials according to which we do not need to stop the trial with planned statistical stopping rules. In this article, we derive RCIs when data from each stage of the trial are not independent thus it is no longer a Brownian motion (BM) process. Under this assumption, a larger class of stochastic processes fractional Brownian motion (FBM) is considered. Comparisons of RCI width and sample size requirement are made to those under Brownian motion for different analysis times, Type I error rates and number of interim analysis. Power family spending functions including Pocock, O'Brien-Fleming design types are considered for these simulations. Interim data from BHAT and oncology trials is used to illustrate how to derive RCIs under FBM for efficacy and futility monitoring.     

A hybrid approach to achieving both marginal and conditional balances for stratification variables in sequential clinical trials

Various methods exist in the literature for achieving marginal balance for baseline stratification variables in sequential clinical trials. One major limitation with balancing on the margins of the stratification variables is that there is an efficiency loss when the primary analysis is stratified. To preserve the efficiency of a stratified analysis one recently proposed approach balances on the crossing of the stratification variables included in the analysis, which achieves conditional balance for the variables. A hybrid approach to achieving both marginal and conditional balances in sequential clinical trials is proposed, which is applicable to both continuous and categorical stratification variables. Numerical results based on extensive simulation studies and a real dataset show that the proposed approach outperforms the existing ones and is particularly useful when both additive and stratified models are planned for a trial. Copyright © 2013 John Wiley & Sons, Ltd.     

Sequential sampling in the search for new shared species

In microbial sciences, as well as other disciplines, it is often valuable to sample communities in a sequential or group sequential manner, in order to determine their structure or their similarity. We develop sequential sampling procedures to accomplish this by first assuming that one observation is drawn with replacement from each population at a time. Suppose that the sampling is terminated after n pairs of observations and k shared species were discovered, and assume that we receive payoff h(k)−cn, where h(k) is non-decreasing and the sampling cost c is non-negative. Similar to Rasmussen and Starr (1979), we show that an optimal stopping rule exists if h(k+1)−h(k) is non-increasing. An analogous result holds for group sequential sampling. This leads to using an estimate of the probability of discovering new shared species as a stopping indicator for comparing two populations with respect to the similarity index. We show by simulation and real examples that this is a feasible approach which can help to reduce the sample size.