Robust inference from multiple test statistics via permutations: a better alternative to the single test statistic approach for randomized trials

Formal inference in randomized clinical trials is based on controlling the type I error rate associated with a single pre‐specified statistic. The deficiency of using just one method of analysis is that it depends on assumptions that may not be met. For robust inference, we propose pre‐specifying multiple test statistics and relying on the minimum p‐value for testing the null hypothesis of no treatment effect. The null hypothesis associated with the various test statistics is that the treatment groups are indistinguishable. The critical value for hypothesis testing comes from permutation distributions. Rejection of the null hypothesis when the smallest p‐value is less than the critical value controls the type I error rate at its designated value. Even if one of the candidate test statistics has low power, the adverse effect on the power of the minimum p‐value statistic is not much. Its use is illustrated with examples. We conclude that it is better to rely on the minimum p‐value rather than a single statistic particularly when that single statistic is the logrank test, because of the cost and complexity of many survival trials. Copyright © 2013 John Wiley & Sons, Ltd.     

More Powerful Tests from Confidence Interval p Values

In this article the problem of comparing two independent binomial populations is considered. It is shown that the test based on the confidence interval p value of Berger and Boos often is uniformly more powerful than the standard unconditional test. This test also requires less computational time.     

Blinded sample size re-estimation in clinical trials comparing several treatments

An important question that arises in clinical trials is how many additional observations, if any, are required beyond those originally planned. This has satisfactorily been answered in the case of two-treatment double-blind clinical experiments. However, one may be interested in comparing a new treatment with its competitors, which may be more than one. This problem is addressed in this investigation involving responses from arbitrary distributions, in which the mean and the variance are not functionally related. First, a solution in determining the initial sample size for specified level of significance and power at a specified alternative is obtained. Then it is shown that when the initial sample size is large, the nominal level of significance and the power at the pre-specified alternative are fairly robust for the proposed sample size re-estimation procedure. An application of the results is made to the blood coagulation functionality problem considered by Kropf et al. [Multiple comparisons of treatments with stable multivariate tests in a two-stage adaptive design, including a test for non-inferiority, Biom. J. 42(8) (2000), pp. 951–965].     

Response-adaptive trial designs with accelerated Thompson sampling

In a clinical trial, sometimes it is desirable to allocate as many patients as possible to the best treatment, in particular, when a trial for a rare disease may contain a considerable portion of the whole target population. The Gittins index rule is a powerful tool for sequentially allocating patients to the best treatment based on the responses of patients already treated. However, its application in clinical trials is limited due to technical complexity and lack of randomness. Thompson sampling is an appealing approach, since it makes a compromise between optimal treatment allocation and randomness with some desirable optimal properties in the machine learning context. However, in clinical trial settings, multiple simulation studies have shown disappointing results with Thompson samplers. We consider how to improve short-run performance of Thompson sampling and propose a novel acceleration approach. This approach can also be applied to situations when patients can only be allocated by batch and is very easy to implement without using complex algorithms. A simulation study showed that this approach could improve the performance of Thompson sampling in terms of average total response rate. An application to a redesign of a preference trial to maximize patient's satisfaction is also presented.     

A course for clinical trial personnel in clinical study designs, randomization, allocation schedules, and interactive response systems

This continuing education course for professionals involved in all areas of clinical trials integrates concepts related to the role of randomization in the scientific process. The course includes two interactive lecture and discussion sections and a workshop practicum. The first interactive lecture introduces basic clinical trial issues and statistical principles such as bias, blinding, randomization, control groups, and the importance of formulating clear and discriminating clinical and statistical hypotheses. It then focuses on the most commonly used clinical study designs and the corresponding patient randomization schemes. The second interactive lecture focuses on the implementation of randomization of patients and drug supply through allocation and component ID schedules. The workshop practicum, conducted in small groups, enables students to apply the lecture concepts to real clinical studies. Flexibility was built into the workshop practicum materials to allow the course content to be customized to specific audiences, and the interactive lecture sessions can be stretched to cover more advanced topics according to class interest and time availability.     

Blinded sample size recalculation for clinical trials with normal data and baseline adjusted analysis

Baseline adjusted analyses are commonly encountered in practice, and regulatory guidelines endorse this practice. Sample size calculations for this kind of analyses require knowledge of the magnitude of nuisance parameters that are usually not given when the results of clinical trials are reported in the literature. It is therefore quite natural to start with a preliminary calculated sample size based on the sparse information available in the planning phase and to re‐estimate the value of the nuisance parameters (and with it the sample size) when a portion of the planned number of patients have completed the study. We investigate the characteristics of this internal pilot study design when an analysis of covariance with normally distributed outcome and one random covariate is applied. For this purpose we first assess the accuracy of four approximate sample size formulae within the fixed sample size design. Then the performance of the recalculation procedure with respect to its actual Type I error rate and power characteristics is examined. The results of simulation studies show that this approach has favorable properties with respect to the Type I error rate and power. Together with its simplicity, these features should make it attractive for practical application. Copyright © 2009 John Wiley & Sons, Ltd.     

The potential for bias in reporting of industry-sponsored clinical trials

Concerns about potentially misleading reporting of pharmaceutical industry research have surfaced many times. The potential for duality (and thereby conflict) of interest is only too clear when you consider the sums of money required for the discovery, development and commercialization of new medicines. As the ability of major, mid-size and small pharmaceutical companies to innovate has waned, as evidenced by the seemingly relentless decline in the numbers of new medicines approved by Food and Drug Administration and European Medicines Agency year-on-year, not only has the cost per new approved medicine risen: so too has the public and media concern about the extent to which the pharmaceutical industry is open and honest about the efficacy, safety and quality of the drugs we manufacture and sell. In 2005 an Editorial in Journal of the American Medical Association made clear that, so great was their concern about misleading reporting of industry-sponsored studies, henceforth no article would be published that was not also guaranteed by independent statistical analysis. We examine the precursors to this Editorial, as well as its immediate and lasting effects for statisticians, for the manner in which statistical analysis is carried out, and for the industry more generally.     

Evaluation of single and mixed verbal operant arrangements for teaching mands and tacts

The purpose of this series of experiments was to evaluate the effects of mixed mand-tact arrangements on the acquisition of mands and tacts in preschool-aged children. In Experiment 1, the effects of three training arrangements (mand-only training, tact-only training, and mand-tact training) were investigated with 3 typically developing children. Rates of acquisition in single (mand-only and tact-only) versus mixed (mand-tact) presentation were comparable, in contrast to earlier investigations. Experiment 2 attempted to clarify the equivocal findings of Experiment 1 by directly replicating the Carroll and Hesse (1987) investigation with 2 typically developing children. Results again demonstrated no clear benefit of mixed verbal operant training on tact acquisition. In Experiment 3, these same arrangements were evaluated with a boy with autism and included assessments to determine that a relevant establishing operation was in effect prior to each mand training session. Experiment 3 again failed to demonstrate the facilitative effects of mand-tact training on the acquisition of mands or tacts. Taken together, the data from these three experiments fail to support the improved efficiency of mand-tact training suggested by prior studies. Findings are discussed in the context of future research investigating mixed verbal operant arrangements.     

Adaptive Percolation Using Subjective Likelihoods

A phenomenon that I call “adaptive percolation” commonly arises in biology, business, economics, defense, finance, manufacturing, and the social sciences. Here one wishes to select a handful of entities from a large pool of entities via a process of screening through a hierarchy of sieves. The process is not unlike the percolation of a liquid through a porous medium. The probability model developed here is based on a nested and adaptive Bayesian approach that results in the product of beta-binomial distributions with common parameters. The common parameters happen to be the observed data. I call this the percolated beta-binomial distribution . The model turns out to be a slight generalization of the probabilistic model used in percolation theory. The generalization is a consequence of using a subjectively specified likelihood function to construct a probability model. The notion of using likelihoods for constructing probability models is not a part of the conventional toolkit of applied probabilists. To the best of my knowledge, a use of the product of beta-binomial distributions as a probability model for Bernoulli trials appears to be new. The development of the material of this article is illustrated via data from the 2009 astronaut selection program, which motivated this work.     

Statistical considerations associated with a comprehensive regulatory framework to address the unmet need for new antibacterial therapies

At present, there are situations in antibiotic drug development where the low number of enrollable patients with key problem pathogens makes it impossible to conduct fully powered non‐inferiority trials in the traditional way. Recent regulatory changes have begun to address this situation. In parallel, statistical issues regarding the application of alternative techniques, balancing the unmet need with the level of certainty in the approval process, and the use of additional sources of data are critical areas to increase development feasibility. Although such approaches increase uncertainty compared with a traditional development program, this will be necessary to allow new agents to be made available. Identification of these risks and explicit discussion around requirements in these areas should help clarify the situation, and hence, the feasibility of developing drugs to treat the most concerning pathogens before the unmet need becomes even more acute than at present. Copyright © 2014 John Wiley & Sons, Ltd.