Sample size recalculation for binary data in internal pilot study designs

For binary endpoints, the required sample size depends not only on the known values of significance level, power and clinically relevant difference but also on the overall event rate. However, the overall event rate may vary considerably between studies and, as a consequence, the assumptions made in the planning phase on this nuisance parameter are to a great extent uncertain. The internal pilot study design is an appealing strategy to deal with this problem. Here, the overall event probability is estimated during the ongoing trial based on the pooled data of both treatment groups and, if necessary, the sample size is adjusted accordingly. From a regulatory viewpoint, besides preserving blindness it is required that eventual consequences for the Type I error rate should be explained. We present analytical computations of the actual Type I error rate for the internal pilot study design with binary endpoints and compare them with the actual level of the chi‐square test for the fixed sample size design. A method is given that permits control of the specified significance level for the chi‐square test under blinded sample size recalculation. Furthermore, the properties of the procedure with respect to power and expected sample size are assessed. Throughout the paper, both the situation of equal sample size per group and unequal allocation ratio are considered. The method is illustrated with application to a clinical trial in depression. Copyright © 2004 John Wiley & Sons Ltd.     

Sample Size Calculation and Blinded Sample Size Recalculation in Clinical Trials Where the Treatment Effect is Measured by the Relative Risk

In clinical trials with binary endpoints, the required sample size does not depend only on the specified type I error rate, the desired power and the treatment effect but also on the overall event rate which, however, is usually uncertain. The internal pilot study design has been proposed to overcome this difficulty. Here, nuisance parameters required for sample size calculation are re-estimated during the ongoing trial and the sample size is recalculated accordingly. We performed extensive simulation studies to investigate the characteristics of the internal pilot study design for two-group superiority trials where the treatment effect is captured by the relative risk. As the performance of the sample size recalculation procedure crucially depends on the accuracy of the applied sample size formula, we firstly explored the precision of three approximate sample size formulae proposed in the literature for this situation. It turned out that the unequal variance asymptotic normal formula outperforms the other two, especially in case of unbalanced sample size allocation. Using this formula for sample size recalculation in the internal pilot study design assures that the desired power is achieved even if the overall rate is mis-specified in the planning phase. The maximum inflation of the type I error rate observed for the internal pilot study design is small and lies below the maximum excess that occurred for the fixed sample size design.     

Effects of the generalized Box–Cox transformation on Type I error rate and power of Hotelling's T 2

Most multivariate statistical techniques rely on the assumption of multivariate normality. The effects of nonnormality on multivariate tests are assumed to be negligible when variance–covariance matrices and sample sizes are equal. Therefore, in practice, investigators usually do not attempt to assess multivariate normality. In this simulation study, the effects of skewed and leptokurtic multivariate data on the Type I error and power of Hotelling's T ² were examined by manipulating distribution, sample size, and variance–covariance matrix. The empirical Type I error rate and power of Hotelling's T ² were calculated before and after the application of generalized Box–Cox transformation. The findings demonstrated that even when variance–covariance matrices and sample sizes are equal, small to moderate changes in power still can be observed.     

Sample size determination for p-charts

Two types of decision errors can be made when using a quality control chart for non-conforming units (p-chart). A Type I error occurs when the process is not out of control but a search for an assignable cause is performed unnecessarily. A Type II error occurs when the process is out of control but a search for an assignable cause is not performed. The probability of a Type I error is under direct control of the decision-maker while the probability of a Type II error depends, in part, on the sample size. A simple sample size formula is presented for determining the required sample size for a p-chart with specified probabilities of Type I and Type II errors.     

A Bayesian sequential design with binary outcome

Several researchers have proposed solutions to control type I error rate in sequential designs. The use of Bayesian sequential design becomes more common; however, these designs are subject to inflation of the type I error rate. We propose a Bayesian sequential design for binary outcome using an alpha‐spending function to control the overall type I error rate. Algorithms are presented for calculating critical values and power for the proposed designs. We also propose a new stopping rule for futility. Sensitivity analysis is implemented for assessing the effects of varying the parameters of the prior distribution and maximum total sample size on critical values. Alpha‐spending functions are compared using power and actual sample size through simulations. Further simulations show that, when total sample size is fixed, the proposed design has greater power than the traditional Bayesian sequential design, which sets equal stopping bounds at all interim analyses. We also find that the proposed design with the new stopping for futility rule results in greater power and can stop earlier with a smaller actual sample size, compared with the traditional stopping rule for futility when all other conditions are held constant. Finally, we apply the proposed method to a real data set and compare the results with traditional designs.     

Sample sizes for trials involving multiple correlated must-win comparisons

In Clinical trials involving multiple comparisons of interest, the importance of controlling the trial Type I error is well-understood and well-documented. Moreover, when these comparisons are themselves correlated, methodologies exist for accounting for the correlation in the trial design, when calculating the trial significance levels. However, less well-documented is the fact that there are some circumstances where multiple comparisons affect the Type II error rather than the Type I error, and failure to account for this, can result in a reduction in the overall trial power. In this paper, we describe sample size calculations for clinical trials involving multiple correlated comparisons, where all the comparisons must be statistically significant for the trial to provide evidence of effect, and show how such calculations have to account for multiplicity in the Type II error. For the situation of two comparisons, we provide a result which assumes a bivariate Normal distribution. For the general case of two or more comparisons we provide a solution using inflation factors to increase the sample size relative to the case of a single outcome. We begin with a simple case of two comparisons assuming a bivariate Normal distribution, show how to factor in correlation between comparisons and then generalise our findings to situations with two or more comparisons. These methods are easy to apply, and we demonstrate how accounting for the multiplicity in the Type II error leads, at most, to modest increases in the sample size.     

Application of Anbar's Approach to Hypothesis Testing to Detect the Difference between Two Proportions

In this paper, Anbar's (1983) approach for estimating a difference between two binomial proportions is discussed with respect to a hypothesis testing problem. Such an approach results in two possible testing strategies. While the results of the tests are expected to agree for a large sample size when two proportions are equal, the tests are shown to perform quite differently in terms of their probabilities of a Type I error for selected sample sizes. Moreover, the tests can lead to different conclusions, which is illustrated via a simple example; and the probability of such cases can be relatively large. In an attempt to improve the tests while preserving their relative simplicity feature, a modified test is proposed. The performance of this test and a conventional test based on normal approximation is assessed. It is shown that the modified Anbar's test better controls the probability of a Type I error for moderate sample sizes.     

Exact methods of testing the homogeneity of prevalences for correlated binary data

Correlated binary data arise in many ophthalmological and otolaryngological clinical trials. To test the homogeneity of prevalences among different groups is an important issue when conducting these trials. The equal correlation coefficients model proposed by Donner in 1989 is a popular model handling correlated binary data. The asymptotic chi-square test works well when the sample size is large. However, it would fail to maintain the type I error rate when the sample size is relatively small. In this paper, we propose several exact methods to deal with small sample scenarios. Their performances are compared with respect to type I error rate and power. The ‘M approach’ and the ‘E + M approach’ seem to outperform the others. A real work example is given to further explain how these approaches work. Finally, the computational efficiency of the exact methods is discussed as a pressing issue of future work.     

Linear models analysis of small samples of categorized ordinal response data

A study of the small sample properties of linear models analyses of contingency tables with ordered response categories is presented. Analytical and empirical study of the linear models approach of Williams and Grizzle suggests that modification would improve its performance in small sample situations. Subsequent simulation studies of the modified procedure show that for small samples it has acceptable performance with respect to Type I errors and is robust with respect to response distribution form and inequality among the sample sizes in the contingency table.     

Sample Size Re-Estimation Based on Two-Stage Analysis of Variance: Interim Analysis of Clinical Trials

Planning and conducting interim analysis are important steps for long-term clinical trials. In this article, the concept of conditional power is combined with the classic analysis of variance (ANOVA) for a study of two-stage sample size re-estimation based on interim analysis. The overall Type I and Type II errors would be inflated by interim analysis. We compared the effects on re-estimating sample sizes with and without the adjustment of Type I and Type II error rates due to interim analysis.     

Approximating Fisher's information for the functional measurement error model

For the functional measurement error model, the true, unobservable explanatory variables when treated as nuisance parameters yield an increase in the number of nuisance parameters corresponding to an increase in sample size. Fisher's information may not exist for all parameters under this scenario. We propose a simple but effective method of deriving Fisher's information by approximating the design matrix of explanatory variables with a quantile design matrix. We illustrate the application of our method with a numerical example. Adaptation of this method shows very good performance for the prediction problem.     

Designing clinical trials with uncertain estimates of variability

An estimated sample size is a function of three components: the required power, the predetermined Type I error rate, and the specified effect size. For Normal data the standardized effect size is taken as the difference between two means divided by an estimate of the population standard deviation. However, in early phase trials one may not have a good estimate of the population variance as it is often based on the results of a few relatively small trials. The imprecision of this estimate should be taken into account in sample size calculations. When estimating a trial sample size this paper recommends that one should investigate the sensitivity of the trial to the assumptions made about the variance and consider being adaptive in one's trial design. Copyright © 2004 John Wiley & Sons Ltd.     

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim

Interest in confirmatory adaptive combined phase II/III studies with treatment selection has increased in the past few years. These studies start comparing several treatments with a control. One (or more) treatment(s) is then selected after the first stage based on the available information at an interim analysis, including interim data from the ongoing trial, external information and expert knowledge. Recruitment continues, but now only for the selected treatment(s) and the control, possibly in combination with a sample size reassessment. The final analysis of the selected treatment(s) includes the patients from both stages and is performed such that the overall Type I error rate is strictly controlled, thus providing confirmatory evidence of efficacy at the final analysis. In this paper we describe two approaches to control the Type I error rate in adaptive designs with sample size reassessment and/or treatment selection. The first method adjusts the critical value using a simulation-based approach, which incorporates the number of patients at an interim analysis, the true response rates, the treatment selection rule, etc. We discuss the underlying assumptions of simulation-based procedures and give several examples where the Type I error rate is not controlled if some of the assumptions are violated. The second method is an adaptive Bonferroni-Holm test procedure based on conditional error rates of the individual treatment-control comparisons. We show that this procedure controls the Type I error rate, even if a deviation from a pre-planned adaptation rule or the time point of such a decision is necessary.     

Sample size re‐estimation incorporating prior information on a nuisance parameter

Prior information is often incorporated informally when planning a clinical trial. Here, we present an approach on how to incorporate prior information, such as data from historical clinical trials, into the nuisance parameter–based sample size re‐estimation in a design with an internal pilot study. We focus on trials with continuous endpoints in which the outcome variance is the nuisance parameter. For planning and analyzing the trial, frequentist methods are considered. Moreover, the external information on the variance is summarized by the Bayesian meta‐analytic‐predictive approach. To incorporate external information into the sample size re‐estimation, we propose to update the meta‐analytic‐predictive prior based on the results of the internal pilot study and to re‐estimate the sample size using an estimator from the posterior. By means of a simulation study, we compare the operating characteristics such as power and sample size distribution of the proposed procedure with the traditional sample size re‐estimation approach that uses the pooled variance estimator. The simulation study shows that, if no prior‐data conflict is present, incorporating external information into the sample size re‐estimation improves the operating characteristics compared to the traditional approach. In the case of a prior‐data conflict, that is, when the variance of the ongoing clinical trial is unequal to the prior location, the performance of the traditional sample size re‐estimation procedure is in general superior, even when the prior information is robustified. When considering to include prior information in sample size re‐estimation, the potential gains should be balanced against the risks.     

Type I error and power in trials with one interim futility analysis

Futility analysis reduces the opportunity to commit Type I error. For a superiority study testing a two‐sided hypothesis, an interim futility analysis can substantially reduce the overall Type I error while keeping the overall power relatively intact. In this paper, we quantify the extent of the reduction for both one‐sided and two‐sided futility analysis. We argue that, because of the reduction, we should be allowed to set the significance level for the final analysis at a level higher than the allowable Type I error rate for the study. We propose a method to find the significance level for the final analysis. We illustrate the proposed methodology and show that a design employing a futility analysis can reduce the sample size, and therefore reduce the exposure of patients to unnecessary risk and lower the cost of a clinical trial. Copyright © 2004 John Wiley & Sons, Ltd.     

Controlling type 1 error rate for sequential,bioequivalence studies with crossover designs

Sample size reestimation in a crossover, bioequivalence study can be a useful adaptive design tool, particularly when the intrasubject variability of the drug formulation under investigation is not well understood. When sample size reestimation is done based on an interim estimate of the intrasubject variability and bioequivalence is tested using the pooled estimate of intrasubject variability, type 1 error inflation will occur. Type 1 error inflation is caused by the pooled estimate being a biased estimator of the intrasubject variability. The type 1 error inflation and bias of the pooled estimator of variability are well characterized in the setting of a two‐arm, parallel study. The purpose of this work is to extend this characterization to the setting of a crossover, bioequivalence study with sample size reestimation and to propose an estimator of the intrasubject variability that will prevent type 1 error inflation.     

Power of the adjusted Q statistic to evaluate heterogeneity in meta-analyses of cluster randomized trials

Because of its simplicity, the Q statistic is frequently used to test the heterogeneity of the estimated intervention effect in meta-analyses of individually randomized trials. However, it is inappropriate to apply it directly to the meta-analyses of cluster randomized trials without taking clustering effects into account. We consider the properties of the adjusted Q statistic for testing heterogeneity in the meta-analyses of cluster randomized trials with binary outcomes. We also derive an analytic expression for the power of this statistic to detect heterogeneity in meta-analyses, which can be useful when planning a meta-analysis. A simulation study is used to assess the performance of the adjusted Q statistic, in terms of its Type I error rate and power. The simulation results are compared to that obtained from the proposed formula. It is found that the adjusted Q statistic has a Type I error rate close to the nominal level of 5%, as compared to the unadjusted Q statistic commonly used to test for heterogeneity in the meta-analyses of individually randomized trials with an inflated Type I error rate. Data from a meta-analysis of four cluster randomized trials are used to illustrate the procedures.     

Repeated Confidence Intervals Under Fractional Brownian Motion in Long-Term Clinical Trials

Repeated confidence interval (RCI) is an important tool for design and monitoring of group sequential trials according to which we do not need to stop the trial with planned statistical stopping rules. In this article, we derive RCIs when data from each stage of the trial are not independent thus it is no longer a Brownian motion (BM) process. Under this assumption, a larger class of stochastic processes fractional Brownian motion (FBM) is considered. Comparisons of RCI width and sample size requirement are made to those under Brownian motion for different analysis times, Type I error rates and number of interim analysis. Power family spending functions including Pocock, O'Brien-Fleming design types are considered for these simulations. Interim data from BHAT and oncology trials is used to illustrate how to derive RCIs under FBM for efficacy and futility monitoring.     

A Geometric Examination of Linear Model Assumptions

From a geometric perspective, linear model theory relies on a single assumption, that (‘corrected’) data vector directions are uniformly distributed in Euclidean space. We use this perspective to explore pictorially the effects of violations of the traditional assumptions (normality, independence and homogeneity of variance) on the Type I error rate. First, for several non‐normal distributions we draw geometric pictures and carry out simulations to show how the effects of non‐normality diminish with increased parent distribution symmetry and continuity, and increased sample size. Second, we explore the effects of dependencies on Type I error rate. Third, we use simulation and geometry to investigate the effect of heterogeneity of variance on Type I error rate. We conclude, in a fresh way, that independence and homogeneity of variance are more important assumptions than normality. The practical implication is that statisticians and authors of statistical computing packages need to pay more attention to the correctness of these assumptions than to normality.     

Sample size calculation for testing non-inferiority and equivalence under Poisson distribution

When counting the number of chemical parts in air pollution studies or when comparing the occurrence of congenital malformations between a uranium mining town and a control population, we often assume Poisson distribution for the number of these rare events. Some discussions on sample size calculation under Poisson model appear elsewhere, but all these focus on the case of testing equality rather than testing equivalence. We discuss sample size and power calculation on the basis of exact distribution under Poisson models for testing non-inferiority and equivalence with respect to the mean incidence rate ratio. On the basis of large sample theory, we further develop an approximate sample size calculation formula using the normal approximation of a proposed test statistic for testing non-inferiority and an approximate power calculation formula for testing equivalence. We find that using these approximation formulae tends to produce an underestimate of the minimum required sample size calculated from using the exact test procedure. On the other hand, we find that the power corresponding to the approximate sample sizes can be actually accurate (with respect to Type I error and power) when we apply the asymptotic test procedure based on the normal distribution. We tabulate in a variety of situations the minimum mean incidence needed in the standard (or the control) population, that can easily be employed to calculate the minimum required sample size from each comparison group for testing non-inferiority and equivalence between two Poisson populations.