Evaluating dynamic treatment strategies: does it have to be more costly?

Dynamic treatment strategies are designed to change treatments over time in response to intermediate outcomes. They can be deployed for primary treatment as well as for the introduction of adjuvant treatment or other treatment‐enhancing interventions. When treatment interventions are delayed until needed, more cost‐efficient strategies will result. Sequential multiple assignment randomized (SMAR) trials allow for unbiased estimation of the marginal effects of different sequences of history‐dependent treatment decisions. Because a single SMAR trial enables evaluation of many different dynamic regimes at once, it is naturally thought to require larger sample sizes than the parallel randomized trial. In this paper, we compare power between SMAR trials studying a regime, where treatment boosting enters when triggered by an observed event, versus the parallel design, where a treatment boost is consistently prescribed over the entire study period. In some settings, we found that the dynamic design yields the more efficient trial for the detection of treatment activity. We develop one particular trial to compare a dynamic nursing intervention with telemonitoring for the enhancement of medication adherence in epilepsy patients. To this end, we derive from the SMAR trial data either an average of conditional treatment effects (‘conditional estimator’) or the population‐averaged (‘marginal’) estimator of the dynamic regimes. Analytical sample size calculations for the parallel design and the conditional estimator are compared with simulated results for the population‐averaged estimator. We conclude that in specific settings, well‐chosen SMAR designs may require fewer data for the development of more cost‐efficient treatment strategies than parallel designs. Copyright © 2012 John Wiley & Sons, Ltd.     

Nested combination tests with a time‐to‐event endpoint using a short‐term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family‐wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short‐term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short‐term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.     

The 'improved' brown and forsythe test for mean equality: some things can't be fixed

Mehrotra (1997) presented an ‘;improved’ Brown and Forsythe (1974) statistic which is designed to provide a valid test of mean equality in independent groups designs when variances are heterogeneous. In particular, the usual Brown and Fosythe procedure was modified by using a Satterthwaite approximation for numerator degrees of freedom instead of the usual value of number of groups minus one. Mehrotra then, through Monte Carlo methods, demonstrated that the ‘improved’ method resulted in a robust test of significance in cases where the usual Brown and Forsythe method did not. Accordingly, this ‘improved’ procedure was recommended. We show that under conditions likely to be encountered in applied settings, that is, conditions involving heterogeneous variances as well as nonnormal data, the ‘improved’ Brown and Forsythe procedure results in depressed or inflated rates of Type I error in unbalanced designs. Previous findings indicate, however, that one can obtain a robust test by adopting a heteroscedastic statistic with the robust estimators, rather than the usual least squares estimators, and further improvement can be expected when critical significance values are obtained through bootstrapping methods.     

Characterization of dose‐response for count data using a generalized MCP‐Mod approach in an adaptive dose‐ranging trial

Understanding the dose–response relationship is a key objective in Phase II clinical development. Yet, designing a dose‐ranging trial is a challenging task, as it requires identifying the therapeutic window and the shape of the dose–response curve for a new drug on the basis of a limited number of doses. Adaptive designs have been proposed as a solution to improve both quality and efficiency of Phase II trials as they give the possibility to select the dose to be tested as the trial goes. In this article, we present a ‘shapebased’ two‐stage adaptive trial design where the doses to be tested in the second stage are determined based on the correlation observed between efficacy of the doses tested in the first stage and a set of pre‐specified candidate dose–response profiles. At the end of the trial, the data are analyzed using the generalized MCP‐Mod approach in order to account for model uncertainty. A simulation study shows that this approach gives more precise estimates of a desired target dose (e.g. ED70) than a single‐stage (fixed‐dose) design and performs as well as a two‐stage D‐optimal design. We present the results of an adaptive model‐based dose‐ranging trial in multiple sclerosis that motivated this research and was conducted using the presented methodology. Copyright © 2015 John Wiley & Sons, Ltd.     

Improving interim decisions in randomized trials by exploiting information on short‐term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short‐term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short‐term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long‐term endpoint which enable the incorporation of baseline covariates and multiple short‐term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re‐assessment based on the inverse normal P‐value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.     

Sample size calculation for the Power Model for dose proportionality studies

There are several approaches to assess or demonstrate pharmacokinetic dose proportionality. One statistical method is the traditional ANOVA model, where dose proportionality is evaluated using the bioequivalence limits. A more informative method is the mixed effects Power Model, where dose proportionality is assessed using a decision rule for the estimated slope. Here we propose analytical derivations of sample sizes for various designs (including crossover, incomplete block and parallel group designs) to be analysed according to the Power Model.     

Optimizing trial design in pharmacogenetics research: comparing a fixed parallel group,group sequential,and adaptive selection design on sample size requirements

Two‐stage clinical trial designs may be efficient in pharmacogenetics research when there is some but inconclusive evidence of effect modification by a genomic marker. Two‐stage designs allow to stop early for efficacy or futility and can offer the additional opportunity to enrich the study population to a specific patient subgroup after an interim analysis. This study compared sample size requirements for fixed parallel group, group sequential, and adaptive selection designs with equal overall power and control of the family‐wise type I error rate. The designs were evaluated across scenarios that defined the effect sizes in the marker positive and marker negative subgroups and the prevalence of marker positive patients in the overall study population. Effect sizes were chosen to reflect realistic planning scenarios, where at least some effect is present in the marker negative subgroup. In addition, scenarios were considered in which the assumed ‘true’ subgroup effects (i.e., the postulated effects) differed from those hypothesized at the planning stage. As expected, both two‐stage designs generally required fewer patients than a fixed parallel group design, and the advantage increased as the difference between subgroups increased. The adaptive selection design added little further reduction in sample size, as compared with the group sequential design, when the postulated effect sizes were equal to those hypothesized at the planning stage. However, when the postulated effects deviated strongly in favor of enrichment, the comparative advantage of the adaptive selection design increased, which precisely reflects the adaptive nature of the design. Copyright © 2013 John Wiley & Sons, Ltd.     

Cumulative cohort design for dose-finding

We introduce a new design for dose-finding in the context of toxicity studies for which it is assumed that toxicity increases with dose. The goal is to identify the maximum tolerated dose, which is taken to be the dose associated with a prespecified “target” toxicity rate. The decision to decrease, increase or repeat a dose for the next subject depends on how far an estimated toxicity rate at the current dose is from the target. The size of the window within which the current dose will be repeated is obtained based on the theory of Markov chains as applied to group up-and-down designs. But whereas the treatment allocation rule in Markovian group up-and-down designs is only based on information from the current cohort of subjects, the treatment allocation rule for the proposed design is based on the cumulative information at the current dose. We then consider an extension of this new design for clinical trials in which the subject's outcome is not known immediately. The new design is compared to the continual reassessment method.     

Model robust extrapolation designs

We seek designs which are optimal in some sense for extrapolation when the true regression function is in a certain class of regression functions. More precisely, the class is defined to be the collection of regression functions such that its (h + 1)-th derivative is bounded. The class can be viewed as representing possible departures from an ‘ideal’ model and thus describes a model robust setting. The estimates are restricted to be linear and the designs are restricted to be with minimal number of points. The design and estimate sought is minimax for mean square error. The optimal designs for cases X = [0, ∞] and X = [-1, 1], where X is the place where observations can be taken, are discussed.     

Adaptive graph‐based multiple testing procedures

Multiple testing procedures defined by directed, weighted graphs have recently been proposed as an intuitive visual tool for constructing multiple testing strategies that reflect the often complex contextual relations between hypotheses in clinical trials. Many well‐known sequentially rejective tests, such as (parallel) gatekeeping tests or hierarchical testing procedures are special cases of the graph based tests. We generalize these graph‐based multiple testing procedures to adaptive trial designs with an interim analysis. These designs permit mid‐trial design modifications based on unblinded interim data as well as external information, while providing strong family wise error rate control. To maintain the familywise error rate, it is not required to prespecify the adaption rule in detail. Because the adaptive test does not require knowledge of the multivariate distribution of test statistics, it is applicable in a wide range of scenarios including trials with multiple treatment comparisons, endpoints or subgroups, or combinations thereof. Examples of adaptations are dropping of treatment arms, selection of subpopulations, and sample size reassessment. If, in the interim analysis, it is decided to continue the trial as planned, the adaptive test reduces to the originally planned multiple testing procedure. Only if adaptations are actually implemented, an adjusted test needs to be applied. The procedure is illustrated with a case study and its operating characteristics are investigated by simulations. Copyright © 2014 John Wiley & Sons, Ltd.     

Optimal PPS Sampling with Vanishing Auxiliary Variables – with Applications in Microscopy

Recently, non‐uniform sampling has been suggested in microscopy to increase efficiency. More precisely, proportional to size (PPS) sampling has been introduced, where the probability of sampling a unit in the population is proportional to the value of an auxiliary variable. In the microscopy application, the sampling units are fields of view, and the auxiliary variables are easily observed approximations to the variables of interest. Unfortunately, often some auxiliary variables vanish, that is, are zero‐valued. Consequently, part of the population is inaccessible in PPS sampling. We propose a modification of the design based on a stratification idea, for which an optimal solution can be found, using a model‐assisted approach. The new optimal design also applies to the case where ‘vanish’ refers to missing auxiliary variables and has independent interest in sampling theory. We verify robustness of the new approach by numerical results, and we use real data to illustrate the applicability.     

Ultimate Extinction of the Promiscuous Bisexual Galton—Watson Metapopulation

A variant of a sexual Gallon–Watson process is considered. At each generation the population is partitioned among n‘hosts’ (population patches) and individual members mate at random only with others within the same host. This is appropriate for many macroparasite systems, and at low parasite loads it gives rise to a depressed rate of reproduction relative to an asexual system, due to the possibility that females are unmated. It is shown that stochasticity mitigates against this effect, so that for small initial populations the probability of ultimate extinction (the complement of an ‘epidemic’) displays a tradeoff as a function of n between the strength of fluctuations which overcome this ‘mating’ probability, and the probability of the subpopulation in one host being ‘rescued’ by that in another. Complementary approximations are developed for the extinction probability: an asymptotically exact approximation at large n, and for small n a short‐time probability that is exact in the limit where the mean number of offspring per parent is large.     

Testing multiple primary endpoints in clinical trials with sample size adaptation

In this paper, we propose a design that uses a short‐term endpoint for accelerated approval at interim analysis and a long‐term endpoint for full approval at final analysis with sample size adaptation based on the long‐term endpoint. Two sample size adaptation rules are compared: an adaptation rule to maintain the conditional power at a prespecified level and a step function type adaptation rule to better address the bias issue. Three testing procedures are proposed: alpha splitting between the two endpoints; alpha exhaustive between the endpoints; and alpha exhaustive with improved critical value based on correlation. Family‐wise error rate is proved to be strongly controlled for the two endpoints, sample size adaptation, and two analysis time points with the proposed designs. We show that using alpha exhaustive designs greatly improve the power when both endpoints are effective, and the power difference between the two adaptation rules is minimal. The proposed design can be extended to more general settings. Copyright © 2015 John Wiley & Sons, Ltd.     

A Five-Decision Testing Procedure to Infer the Value of a Unidimensional Parameter

ABSTRACT

A statistical test can be seen as a procedure to produce a decision based on observed data, where some decisions consist of rejecting a hypothesis (yielding a significant result) and some do not, and where one controls the probability to make a wrong rejection at some prespecified significance level. Whereas traditional hypothesis testing involves only two possible decisions (to reject or not a null hypothesis), Kaiser’s directional two-sided test as well as the more recently introduced testing procedure of Jones and Tukey, each equivalent to running two one-sided tests, involve three possible decisions to infer the value of a unidimensional parameter. The latter procedure assumes that a point null hypothesis is impossible (e.g., that two treatments cannot have exactly the same effect), allowing a gain of statistical power. There are, however, situations where a point hypothesis is indeed plausible, for example, when considering hypotheses derived from Einstein’s theories. In this article, we introduce a five-decision rule testing procedure, equivalent to running a traditional two-sided test in addition to two one-sided tests, which combines the advantages of the testing procedures of Kaiser (no assumption on a point hypothesis being impossible) and Jones and Tukey (higher power), allowing for a nonnegligible (typically 20%) reduction of the sample size needed to reach a given statistical power to get a significant result, compared to the traditional approach.     

A Multiplicative Alo-group Based Hierarchical Decision Model and Application

Pairwise comparison matrix (PCM) is a popular technique used in multi-criteria decision making. The abelian linearly ordered group (alo-group) is a powerful tool for the discussion of PCMs. In this article, a criterion for acceptable consistency of PCM is introduced, which is independent of the scale and can be intuitively interpreted. The relation of the introduced criterion with the weak consistency is investigated. Then, a multiplicative alo-group based hierarchical decision model is proposed. The following approaches are included: (1) the introduced criterion for acceptable consistency is used to check whether or not a PCM is acceptable; (2) the row’s geometric mean method is used for deriving the local priorities of a multiplicative PCM; (3) a Hierarchy Composition Rule derived from the weighted mean is used for computing the criterion/subcriterion’s weights with regard to the total goal; and (4) the weighted geometric mean is used as the aggregation rule, where the alternative’s local priorities are min-normalized. The proposed model has the property of preserving rank. Moreover, it has counterparts in the additive case. Finally, the model is applied to a layout planning problem of an aircraft maintenance base with a computer-based software.     

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim

Interest in confirmatory adaptive combined phase II/III studies with treatment selection has increased in the past few years. These studies start comparing several treatments with a control. One (or more) treatment(s) is then selected after the first stage based on the available information at an interim analysis, including interim data from the ongoing trial, external information and expert knowledge. Recruitment continues, but now only for the selected treatment(s) and the control, possibly in combination with a sample size reassessment. The final analysis of the selected treatment(s) includes the patients from both stages and is performed such that the overall Type I error rate is strictly controlled, thus providing confirmatory evidence of efficacy at the final analysis. In this paper we describe two approaches to control the Type I error rate in adaptive designs with sample size reassessment and/or treatment selection. The first method adjusts the critical value using a simulation-based approach, which incorporates the number of patients at an interim analysis, the true response rates, the treatment selection rule, etc. We discuss the underlying assumptions of simulation-based procedures and give several examples where the Type I error rate is not controlled if some of the assumptions are violated. The second method is an adaptive Bonferroni-Holm test procedure based on conditional error rates of the individual treatment-control comparisons. We show that this procedure controls the Type I error rate, even if a deviation from a pre-planned adaptation rule or the time point of such a decision is necessary.     

Estimation of Stratified Mark‐Specific Proportional Hazards Models with Missing Marks

Abstract. An objective of randomized placebo‐controlled preventive HIV vaccine efficacy trials is to assess the relationship between the vaccine effect to prevent infection and the genetic distance of the exposing HIV to the HIV strain represented in the vaccine construct. Motivated by this objective, recently a mark‐specific proportional hazards (PH) model with a continuum of competing risks has been studied, where the genetic distance of the transmitting strain is the continuous ‘mark’ defined and observable only in failures. A high percentage of genetic marks of interest may be missing for a variety of reasons, predominantly because rapid evolution of HIV sequences after transmission before a blood sample is drawn from which HIV sequences are measured. This research investigates the stratified mark‐specific PH model with missing marks where the baseline functions may vary with strata. We develop two consistent estimation approaches, the first based on the inverse probability weighted complete‐case (IPW) technique, and the second based on augmenting the IPW estimator by incorporating auxiliary information predictive of the mark. We investigate the asymptotic properties and finite‐sample performance of the two estimators, and show that the augmented IPW estimator, which satisfies a double robustness property, is more efficient.     

A statistical testing framework for evaluating the quality of measurement processes

In this paper we address the evaluation of measurement process quality. We mainly focus on the evaluation procedure, as far as it is based on the numerical outcomes for the measurement of a single physical quantity. We challenge the approach where the ‘exact’ value of the observed quantity is compared with the error interval obtained from the measurements under test and we propose a procedure where reference measurements are used as ‘gold standard’. To this purpose, we designed a specific t-test procedure, explained here. We also describe and discuss a numerical simulation experiment demonstrating the behaviour of our procedure.     

The pitfall of instrumental variables in big data: What the rule of thumb can't give you

ABSTRACT

Background: Instrumental variables (IVs) have become much easier to find in the “Big data era” which has increased the number of applications of the Two-Stage Least Squares model (TSLS). With the increased availability of IVs, the possibility that these IVs are weak has increased. Prior work has suggested a ‘rule of thumb’ that IVs with a first stage F statistic at least ten will avoid a relative bias in point estimates greater than 10%. We investigated whether or not this threshold was also an efficient guarantee of low false rejection rates of the null hypothesis test in TSLS applications with many IVs.

Objective: To test how the ‘rule of thumb’ for weak instruments performs in predicting low false rejection rates in the TSLS model when the number of IVs is large.

Method: We used a Monte Carlo approach to create 28 original data sets for different models with the number of IVs varying from 3 to 30. For each model, we generated 2000 observations for each iteration and conducted 50,000 iterations to reach convergence in rejection rates. The point estimate was set to 0, and probabilities of rejecting this hypothesis were recorded for each model as a measurement of false rejection rate. The relationship between the endogenous variable and IVs was carefully adjusted to let the F statistics for the first stage model equal ten, thus simulating the ‘rule of thumb.’

Results: We found that the false rejection rates (type I errors) increased when the number of IVs in the TSLS model increased while holding the F statistics for the first stage model equal to 10. The false rejection rate exceeds 10% when TLSL has 24 IVs and exceed 15% when TLSL has 30 IVs.

Conclusion: When more instrumental variables were applied in the model, the ‘rule of thumb’ was no longer an efficient guarantee for good performance in hypothesis testing. A more restricted margin for F statistics is recommended to replace the ‘rule of thumb,’ especially when the number of instrumental variables is large.     

Investigation of the properties of the “switching decision rule” described in the Japanese guideline for human bioequivalence studies

In a human bioequivalence (BE) study, the conclusion of BE is usually based on the ratio of geometric means of pharmacokinetic parameters between a test and a reference products. The “Guideline for Bioequivalence Studies of Generic Products” (2012) issued by the Japanese health authority and other similar guidelines across the world require a 90% confidence interval (CI) of the ratio to fall entirely within the range of 0.8 to 1.25 for the conclusion of BE. If prerequisite conditions are satisfied, the Japanese guideline provides for a secondary BE criterion that requires the point estimate of the ratio to fall within the range of 0.9 to 1.11. We investigated the statistical properties of the “switching decision rule” wherein the secondary criterion is applied only when the CI criterion fails. The behavior of the switching decision rule differed from either of its component criteria and displayed an apparent type I error rate inflation when the prerequisite conditions were not considered. The degree of inflation became greater as the true variability increased in comparison to the assumed variability used in the sample size calculation. To our knowledge, this is the first report in which the overall behavior of the combination of the two component criteria was investigated. The implications of the in vitro tests on human BE and the accuracy of the intra‐subject variability have impacts on appropriate planning and interpretation of BE studies utilizing the switching decision rule.