Type I error and power in trials with one interim futility analysis

Futility analysis reduces the opportunity to commit Type I error. For a superiority study testing a two‐sided hypothesis, an interim futility analysis can substantially reduce the overall Type I error while keeping the overall power relatively intact. In this paper, we quantify the extent of the reduction for both one‐sided and two‐sided futility analysis. We argue that, because of the reduction, we should be allowed to set the significance level for the final analysis at a level higher than the allowable Type I error rate for the study. We propose a method to find the significance level for the final analysis. We illustrate the proposed methodology and show that a design employing a futility analysis can reduce the sample size, and therefore reduce the exposure of patients to unnecessary risk and lower the cost of a clinical trial. Copyright © 2004 John Wiley & Sons, Ltd.     

Informing the selection of futility stopping thresholds: case study from a late-phase clinical trial

In an environment where (i) potential risks to subjects participating in clinical studies need to be managed carefully, (ii) trial costs are increasing, and (iii) there are limited research resources available, it is necessary to prioritize research projects and sometimes re-prioritize if early indications suggest that a trial has low probability of success. Futility designs allow this re-prioritization to take place. This paper reviews a number of possible futility methods available and presents a case study from a late-phase study of an HIV therapeutic, which utilized conditional power-based stopping thresholds. The two most challenging aspects of incorporating a futility interim analysis into a trial design are the selection of optimal stopping thresholds and the timing of the analysis, both of which require the balancing of various risks. The paper outlines a number of graphical aids that proved useful in explaining the statistical risks involved to the study team. Further, the paper outlines a decision analysis undertaken which combined expectations of drug performance with conditional power calculations in order to produce probabilities of different interim and final outcomes, and which ultimately led to the selection of the final stopping thresholds.     

Assessment of futility in clinical trials

The term 'futility' is used to refer to the inability of a clinical trial to achieve its objectives. In particular, stopping a clinical trial when the interim results suggest that it is unlikely to achieve statistical significance can save resources that could be used on more promising research. There are various approaches that have been proposed to assess futility, including stochastic curtailment, predictive power, predictive probability, and group sequential methods. In this paper, we describe and contrast these approaches, and discuss several issues associated with futility analyses, such as ethical considerations, whether or not type I error can or should be reclaimed, one-sided vs two-sided futility rules, and the impact of futility analyses on power.     

A two- sample partially sequential probability ratio test

A two-sample partially sequential probability ratio test (PSPRT) is considered for the two-sample location problem with one sample fixed and the other sequential. Observations are assumed to come from two normal poptilatlons with equal and known variances. Asymptotically in the fixed-sample size the PSPRT is a truncated Wald one sample sequential probability test. Brownian motion approximations for boundary-crossing probabilities and expected sequential sample size are obtained. These calculations are compared to values obtained by Monte Carlo simulation.     

A two-sample parametric group sequential procedure for monitoring clinical trials

A group sequential procedure is presented which allows for staggered entry of patients, random loss to followup, and utilizes the flexible boundary approach of Lan and DeMets. The proposed procedure assumes that response times are nearly exponentially distributed. An example as well as simulation studies comparing the performance of the proposed procedure with the group sequential logrank are also presented.     

A new test for decreasing mean residual lifetimes

The mean residual life of a non negative random variable X with a finite mean is defined by M(t) = E[X ? t|X > t] for t ? 0. One model of aging is the decreasing mean residual life (DMRL): M is decreasing (non increasing) in time. It vastly generalizes the more stringent model of increasing failure rate (IFR). The exponential distribution lies at the boundary of both of these classes. There is a large literature on testing exponentiality against DMRL alternatives which are all of the integral type. Because most parametric families of DMRL distributions are IFR, their relative merits have been compared only at some IFR alternatives. We introduce a new Kolmogorov–Smirnov type sup-test and derive its asymptotic properties. We compare the powers of this test with some integral tests by simulations using a class of DMRL, but not IFR alternatives, as well as some popular IFR alternatives. The results show that the sup-test is much more powerful than the integral tests in all cases.     

Distribution-free test in Tobit mean regression model

This paper discusses the problem of fitting a parametric model in Tobit mean regression models. The proposed test is based on the supremum of the Khamaladze-type transformation of a partial sum process of calibrated residuals. The asymptotic null distribution of this transformed process is shown to be the same as that of a time-transformed standard Brownian motion. Consistency of this sequence of tests against some fixed alternatives and asymptotic power under some local nonparametric alternatives are also discussed. Simulation studies are conducted to assess the finite sample performance of the proposed test. The power comparison with some existing tests shows some superiority of the proposed test at the chosen alternatives.     

A generalized likelihood ratio test for monitoring profile data

Profile data emerges when the quality of a product or process is characterized by a functional relationship among (input and output) variables. In this paper, we focus on the case where each profile has one response variable Y, one explanatory variable x, and the functional relationship between these two variables can be rather arbitrary. The basic concept can be applied to a much wider case, however. We propose a general method based on the Generalized Likelihood Ratio Test (GLRT) for monitoring of profile data. The proposed method uses nonparametric regression to estimate the on-line profiles and thus does not require any functional form for the profiles. Both Shewhart-type and EWMA-type control charts are considered. The average run length (ARL) performance of the proposed method is studied. It is shown that the proposed GLRT-based control chart can efficiently detect both location and dispersion shifts of the on-line profiles from the baseline profile. An upper control limit (UCL) corresponding to a desired in-control ARL value is constructed.     

The new robust two-sample test for randomly right-censored data

Using a minimum p-value principle, a new two-sample test MIN3 is proposed in the paper. The cumulative distribution function of the MIN3 test statistic is studied and approximated by the Beta distribution of the third kind. Lower percentage points of the distribution of the new test statistic under the null hypothesis are computed. Also the test power for a lot of types of alternative hypotheses (with 0, 1 and 2 point(-s) of the intersection(-s) of survival functions) is studied and we found that the usage of the MIN3 test is a preferred strategy by the Wald and Savage decision-making criteria under risk and uncertainty. The results of application of the MIN3 test are shown for two examples from lifetime data analysis.     

A new computational approach test for one-way ANOVA under heteroscedasticity

This paper proposes a new test statistic based on the computational approach test (CAT) for one-way analysis of variance (ANOVA) under heteroscedasticity. The proposed test was compared with other popular tests according to type I error and power of tests under different combinations of variances, means, number of groups and sample sizes. As a result, it was observed that the proposed test yields better results than other tests in many cases.     

Choice of alpha spending function and time points in clinical trials with one or two interim analyses

One characterization of group sequential methods uses alpha spending functions to allocate the false positive rate throughout a study. We consider and evaluate several such spending functions as well as the time points of the interim analyses at which they apply. In addition, we evaluate the double triangular test as an alternative procedure that allows for early termination of the trial not only due to efficacy differences between treatments, but also due to lack of such differences. We motivate and illustrate our work by reference to the analysis of survival data from a proposed oncology study. Such group sequential procedures with one or two interim analyses are only slightly less powerful than fixed sample trials, but provide for the strong possibility of early stopping. Therefore, in all situations where they can practically be applied, we recommend their routine use in clinical trials. The double triangular test provides a suitable alternative to the group sequential procedures in that they do not provide for early stopping with acceptance of the null hypothesis. Again, there is only a modest loss in power relative to fixed sample tests. Copyright © 2004 John Wiley & Sons, Ltd.     

A new adaptive test for paired data for small to moderate sample sizes

When carrying out data analysis, a practitioner has to decide on a suitable test for hypothesis testing, and as such, would look for a test that has a high relative power. Tests for paired data tests are usually conducted using t-test, Wilcoxon signed-rank test or the sign test. Some adaptive tests have also been suggested in the literature by O'Gorman, who found that no single member of that family performed well for all sample sizes and different tail weights, and hence, he recommended that choice of a member of that family be made depending on both the sample size and the tail weight. In this paper, we propose a new adaptive test. Simulation studies for n=25 and n=50 show that it works well for nearly all tail weights ranging from the light-tailed beta and uniform distributions to t(4) distributions. More precisely, our test has both robustness of level (in keeping the empirical levels close to the nominal level) and efficiency of power. The results of our study contribute to the area of statistical inference.     

A nonparametric test for proving noninferiority in clinical trials with ordered categorical data

We consider the problem of proving noninferiority when the comparison is based on ordered categorical data. We apply a rank test based on the Wilcoxon–Mann–Whitney effect where the asymptotic variance is estimated consistently under the alternative and a small‐sample approximation is given. We give the associated 100(1?α)% confidence interval and propose a formula for sample size determination. Finally, we illustrate the procedure and possible choices of the noninferiority margin using data from a clinical trial. Copyright © 2003 John Wiley & Sons, Ltd.     

Improving interim decisions in randomized trials by exploiting information on short‐term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short‐term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short‐term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long‐term endpoint which enable the incorporation of baseline covariates and multiple short‐term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re‐assessment based on the inverse normal P‐value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.     

A new revised version of McNemar's test for paired binary data

By considering separately B and C, the frequencies of individuals who consistently gave positive or negative answers in before and after responses, a new revised version of McNemar's test is derived. It improves upon Lu's revised formula, which considers B and C together. When both B and C are 0, the new revised version produces the same results as McNemar's test. When one of B and C is 0, the new revised test produces the same results as Lu's version. Compared to Lu's version, the new revised test is a more complete revision of McNemar's test.     

A new two-sample test for choosing between log-rank and Wilcoxon tests with right-censored data

When differences of survival functions are located in early time, a Wilcoxon test is the best test, but when differences of survival functions are located in late time, using a log-rank test is better. Therefore, a researcher needs a stable test in these situations. In this paper, a new two-sample test is proposed and considered. This test is distribution-free. This test is useful for choosing between log-rank and Wilcoxon tests. Its power is roughly the maximal power of the log-rank test and Wilcoxon test.     

A new test for two-sample location problem based on empirical distribution function

We propose a new test for testing the equality of location parameter of two populations based on empirical distribution function (ECDF). The test statistics is obtained as a power divergence between two ECDFs. The test is shown to be distribution free, and its null distribution is obtained. We conducted empirical power comparison of the proposed test with several other available tests in the literature. We found that the proposed test performs better than its competitors considered here under several population structures. We also used two real datasets to illustrate the procedure.     

Uniformly most powerful unbiased test in univariate linear calibration

In this paper, we deal with the hypothesis testing problems for the univariate linear calibration, where a normally distributed response variable and an explanatory variable are involved, and the observations of the response variable corresponding to known values of the explanatory variable are used for making inferences concerning a single unknown value of the explanatory variable. The uniformly most powerful unbiased tests for both one-sided and two-sided hypotheses are constructed and verified. The power behaviour of the proposed tests is numerically compared with that of the existing method, and simulations show that the proposed tests make the powers improved.     

Adaptive and repeated cumulative meta‐analyses of safety data during a new drug development process

During a new drug development process, it is desirable to timely detect potential safety signals. For this purpose, repeated meta‐analyses may be performed sequentially on accumulating safety data. Moreover, if the amount of safety data from the originally planned program is not enough to ensure adequate power to test a specific hypothesis (e.g., the noninferiority hypothesis of an event of interest), the total sample size may be increased by adding new studies to the program. Without appropriate adjustment, it is well known that the type I error rate will be inflated because of repeated analyses and sample size adjustment. In this paper, we discuss potential issues associated with adaptive and repeated cumulative meta‐analyses of safety data conducted during a drug development process. We consider both frequentist and Bayesian approaches. A new drug development example is used to demonstrate the application of the methods. Copyright © 2015 John Wiley & Sons, Ltd.     

A new alternative to the standard F test for clustered data

The data collection process and the inherent population structure are the main causes for clustered data. The observations in a given cluster are correlated, and the magnitude of such correlation is often measured by the intra-cluster correlation coefficient. The intra-cluster correlation can lead to an inflated size of the standard F test in a linear model. In this paper, we propose a solution to this problem. Unlike previous adjustments, our method does not require estimation of the intra-class correlation, which is problematic especially when the number of clusters is small. Our simulation results show that the new method outperforms the existing methods.