Designing field experiments which are subject to representation bias

The term 'representation bias' is used to describe the disparities that exist between treatment effects estimated from field experiments, and those effects that would be seen if treatments were used in the field. In this paper we are specifically concerned with representation bias caused by disease inoculum travelling between plots, or out of the experimental area altogether. The scope for such bias is maximized in the case of airborne spread diseases. This paper extends the work of Deardon et al. (2004), using simulation methods to explore the relationship between design and representation bias. In doing so, we illustrate the importance of plot size and spacing, as well as treatment-to-plot allocation. We examine a novel class of designs, incomplete column designs, to develop an understanding of the mechanisms behind representation bias. We also introduce general methods of designing field trials, which can be used to limit representation bias by carefully controlling treatment to block allocation in both incomplete column and incomplete randomized block designs. Finally, we show how the commonly used practice of sampling from the centres of plots, rather than entire plots, can also help to control representation bias.     

Selection bias for treatments with positive Phase 2 results

In drug development, treatments are most often selected at Phase 2 for further development when an initial trial of a new treatment produces a result that is considered positive. This selection due to a positive result means, however, that an estimator of the treatment effect, which does not take account of the selection is likely to over‐estimate the true treatment effect (ie, will be biased). This bias can be large and researchers may face a disappointingly lower estimated treatment effect in further trials. In this paper, we review a number of methods that have been proposed to correct for this bias and introduce three new methods. We present results from applying the various methods to two examples and consider extensions of the examples. We assess the methods with respect to bias of estimation of the treatment effect and compare the probabilities that a bias‐corrected treatment effect estimate will exceed a decision threshold. Following previous work, we also compare average power for the situation where a Phase 3 trial is launched given that the bias‐corrected observed Phase 2 treatment effect exceeds a launch threshold. Finally, we discuss our findings and potential application of the bias correction methods.     

Error variance bias in neighbour balance and evenness of distribution designs

Neighbour balance and evenness of distribution designs help to address user concerns in the two‐dimensional layout of agricultural field trials. This is done by minimising the occurrence of pairwise treatment plot neighbours and ensuring that the replications of treatments are spread out across rows and columns of a trial. Such considerations result in a restriction on the normal randomisation process for a row‐column design which can lead to error variance bias. In this paper, uniformity trial data is used to assess the degree of this bias for both resolvable and non‐resolvable designs. Comparisons are made with a similar investigation using Linear Variance spatial designs.     

Confidence bands for the difference of two survival functions under the additive risk model

In many clinical studies, a commonly encountered problem is to compare the survival probabilities of two treatments for a given patient with a certain set of covariates, and there is often a need to make adjustments for other covariates that may affect outcomes. One approach is to plot the difference between the two subject-specific predicted survival estimates with a simultaneous confidence band. Such a band will provide useful information about when these two treatments differ and which treatment has a better survival probability. In this paper, we show how to construct such a band based on the additive risk model and we use the martingale central limit theorem to derive its asymptotic distribution. The proposed method is evaluated from a simulation study and is illustrated with two real examples.     

NEAREST NEIGHBOUR ANALYSIS OF UNEQUALLY REPLICATED TRIALS

There has been much work on the use of neighbouring plots to control environmental variation in the analysis of agricultural field experiments. In particular, the Residual Maximum Likelihood Neighbour (REMLN) analysis of Gleeson&Cullis (1987) appears very promising. The application of the REMLN analysis to an unequally replicated field trial augmented with an additional variety planted every six plots in a grid system is here compared with a covariance (COV) analysis using the neighbouring grid or check plot values as the covariate. The results indicate that the REMLN analysis gives more accurate estimates of treatment contrasts than the COV analyses, but that the estimate of treatment means can be biased. The bias depends on the mean of the check plot. This bias can be removed by adjusting the estimates of the treatment means such that their average equals the average of the raw means rather than that of the raw data.     

Intent‐to‐treat analysis in the presence of off‐treatment or missing data

Intent‐to‐treat (ITT) analysis is viewed as the analysis of a clinical trial that provides the least bias, but difficult issues can arise. Common analysis methods such as mixed‐effects and proportional hazards models are usually labeled as ITT analysis, but in practice they can often be inconsistent with a strict interpretation of the ITT principle. In trials where effective medications are available to patients withdrawing from treatment, ITT analysis can mask important therapeutic effects of the intervention studied in the trial. Analysis of on‐treatment data may be subject to bias, but can address efficacy objectives when combined with careful review of the pattern of withdrawals across treatments particularly for those patients withdrawing due to lack of efficacy and adverse events. Copyright © 2010 John Wiley & Sons, Ltd.     

VALID RANDOMISATION FOR THE FIRST-DIFFERENCE ANALYSIS

A general method is presented for randomising a block design while preserving the neighbour relationships between treatments. The randomisation possesses validity properties for the first-difference analysis introduced by Besag & Kempton (1986). The estimators of treatment differences are unbiased, and the paper shows how to calculate quadratic estimators of their variance, which are unbiased if treatment effects and plot effects are additive. Simplifications, which appear when the design is neighbour-balanced, are described and illustrated.     

Point estimates and confidence regions for sequential trials involving selection

A number of authors have proposed clinical trial designs involving the comparison of several experimental treatments with a control treatment in two or more stages. At the end of the first stage, the most promising experimental treatment is selected, and all other experimental treatments are dropped from the trial. Provided it is good enough, the selected experimental treatment is then compared with the control treatment in one or more subsequent stages. The analysis of data from such a trial is problematic because of the treatment selection and the possibility of stopping at interim analyses. These aspects lead to bias in the maximum-likelihood estimate of the advantage of the selected experimental treatment over the control and to inaccurate coverage for the associated confidence interval. In this paper, we evaluate the bias of the maximum-likelihood estimate and propose a bias-adjusted estimate. We also propose an approach to the construction of a confidence region for the vector of advantages of the experimental treatments over the control based on an ordering of the sample space. These regions are shown to have accurate coverage, although they are also shown to be necessarily unbounded. Confidence intervals for the advantage of the selected treatment are obtained from the confidence regions and are shown to have more accurate coverage than the standard confidence interval based upon the maximum-likelihood estimate and its asymptotic standard error.     

Optimal Circular Block Designs for Neighbouring Competition Effects

Competition or interference occurs when the responses to treatments in experimental units are affected by the treatments in neighbouring units. This may contribute to variability in experimental results and lead to substantial losses in efficiency. The study of a competing situation needs designs in which the competing units appear in a predetermined pattern. This paper deals with optimality aspects of circular block designs for studying the competition among treatments applied to neighbouring experimental units. The model considered is a four-way classified model consisting of direct effect of the treatment applied to a particular plot, the effect of those treatments applied to the immediate left and right neighbouring units and the block effect. Conditions have been obtained for the block design to be universally optimal for estimating direct and neighbour effects. Some classes of balanced and strongly balanced complete block designs have been identified to be universally optimal for the estimation of direct, left and right neighbour effects and a list of universally optimal designs for v<20 and r<100 has been prepared.     

Analysis of progression-free survival in oncology trials: some common statistical issues

With the advent of ever more effective second and third line cancer treatments and the growing use of 'crossover' trial designs in oncology, in which patients switch to the alternate randomized treatment upon disease progression, progression-free survival (PFS) is an increasingly important endpoint in oncologic drug development. However, several concerns exist regarding the use of PFS as a basis to compare treatments. Unlike survival, the exact time of progression is unknown, so progression times might be over-estimated and, consequently, bias may be introduced when comparing treatments. Further, it is not uncommon for randomized therapy to be stopped prior to progression being documented due to toxicity or the initiation of additional anti-cancer therapy; in such cases patients are frequently not followed further for progression and, consequently, are right-censored in the analysis. This article reviews these issues and concludes that concerns relating to the exact timing of progression are generally overstated, with analysis techniques and simple alternative endpoints available to either remove bias entirely or at least provide reassurance via supportive analyses that bias is not present. Further, it is concluded that the regularly recommended manoeuvre to censor PFS time at dropout due to toxicity or upon the initiation of additional anti-cancer therapy is likely to favour the more toxic, less efficacious treatment and so should be avoided whenever possible.     

Joint modeling tumor burden and time to event data in oncology trials

The tumor burden (TB) process is postulated to be the primary mechanism through which most anticancer treatments provide benefit. In phase II oncology trials, the biologic effects of a therapeutic agent are often analyzed using conventional endpoints for best response, such as objective response rate and progression‐free survival, both of which causes loss of information. On the other hand, graphical methods including spider plot and waterfall plot lack any statistical inference when there is more than one treatment arm. Therefore, longitudinal analysis of TB data is well recognized as a better approach for treatment evaluation. However, longitudinal TB process suffers from informative missingness because of progression or death. We propose to analyze the treatment effect on tumor growth kinetics using a joint modeling framework accounting for the informative missing mechanism. Our approach is illustrated by multisetting simulation studies and an application to a nonsmall‐cell lung cancer data set. The proposed analyses can be performed in early‐phase clinical trials to better characterize treatment effect and thereby inform decision‐making. Copyright © 2014 John Wiley & Sons, Ltd.     

A graphical model selection tool for mixed models

ABSTRACT

Model selection can be defined as the task of estimating the performance of different models in order to choose the most parsimonious one, among a potentially very large set of candidate statistical models. We propose a graphical representation to be considered as an extension to the class of mixed models of the deviance plot proposed in the literature within the framework of classical and generalized linear models. This graphical representation allows, once a reduced number of models have been selected, to identify important covariates focusing only on the fixed effects component, assuming the random part properly specified. Nevertheless, we suggest also a standalone figure representing the residual random variance ratio: a cross-evaluation of the two graphical representations will allow to derive some conclusions on the random part specification of the model and a more accurate selection of the final model.     

Landmark estimation of survival and treatment effects in observational studies

Clinical studies aimed at identifying effective treatments to reduce the risk of disease or death often require long term follow-up of participants in order to observe a sufficient number of events to precisely estimate the treatment effect. In such studies, observing the outcome of interest during follow-up may be difficult and high rates of censoring may be observed which often leads to reduced power when applying straightforward statistical methods developed for time-to-event data. Alternative methods have been proposed to take advantage of auxiliary information that may potentially improve efficiency when estimating marginal survival and improve power when testing for a treatment effect. Recently, Parast et al. (J Am Stat Assoc 109(505):384–394, 2014) proposed a landmark estimation procedure for the estimation of survival and treatment effects in a randomized clinical trial setting and demonstrated that significant gains in efficiency and power could be obtained by incorporating intermediate event information as well as baseline covariates. However, the procedure requires the assumption that the potential outcomes for each individual under treatment and control are independent of treatment group assignment which is unlikely to hold in an observational study setting. In this paper we develop the landmark estimation procedure for use in an observational setting. In particular, we incorporate inverse probability of treatment weights (IPTW) in the landmark estimation procedure to account for selection bias on observed baseline (pretreatment) covariates. We demonstrate that consistent estimates of survival and treatment effects can be obtained by using IPTW and that there is improved efficiency by using auxiliary intermediate event and baseline information. We compare our proposed estimates to those obtained using the Kaplan–Meier estimator, the original landmark estimation procedure, and the IPTW Kaplan–Meier estimator. We illustrate our resulting reduction in bias and gains in efficiency through a simulation study and apply our procedure to an AIDS dataset to examine the effect of previous antiretroviral therapy on survival.     

Conditional estimation using prior information in 2‐stage group sequential designs assuming asymptotic normality when the trial terminated early

Two‐stage designs are widely used to determine whether a clinical trial should be terminated early. In such trials, a maximum likelihood estimate is often adopted to describe the difference in efficacy between the experimental and reference treatments; however, this method is known to display conditional bias. To reduce such bias, a conditional mean‐adjusted estimator (CMAE) has been proposed, although the remaining bias may be nonnegligible when a trial is stopped for efficacy at the interim analysis. We propose a new estimator for adjusting the conditional bias of the treatment effect by extending the idea of the CMAE. This estimator is calculated by weighting the maximum likelihood estimate obtained at the interim analysis and the effect size prespecified when calculating the sample size. We evaluate the performance of the proposed estimator through analytical and simulation studies in various settings in which a trial is stopped for efficacy or futility at the interim analysis. We find that the conditional bias of the proposed estimator is smaller than that of the CMAE when the information time at the interim analysis is small. In addition, the mean‐squared error of the proposed estimator is also smaller than that of the CMAE. In conclusion, we recommend the use of the proposed estimator for trials that are terminated early for efficacy or futility.     

The Analysis of Designed Experiments and Longitudinal Data by Using Smoothing Splines

In designed experiments and in particular longitudinal studies, the aim may be to assess the effect of a quantitative variable such as time on treatment effects. Modelling treatment effects can be complex in the presence of other sources of variation. Three examples are presented to illustrate an approach to analysis in such cases. The first example is a longitudinal experiment on the growth of cows under a factorial treatment structure where serial correlation and variance heterogeneity complicate the analysis. The second example involves the calibration of optical density and the concentration of a protein DNase in the presence of sampling variation and variance heterogeneity. The final example is a multienvironment agricultural field experiment in which a yield–seeding rate relationship is required for several varieties of lupins. Spatial variation within environments, heterogeneity between environments and variation between varieties all need to be incorporated in the analysis. In this paper, the cubic smoothing spline is used in conjunction with fixed and random effects, random coefficients and variance modelling to provide simultaneous modelling of trends and covariance structure. The key result that allows coherent and flexible empirical model building in complex situations is the linear mixed model representation of the cubic smoothing spline. An extension is proposed in which trend is partitioned into smooth and non-smooth components. Estimation and inference, the analysis of the three examples and a discussion of extensions and unresolved issues are also presented.     

Improving the causal treatment effect estimation with propensity scores by the bootstrap

AStA Advances in Statistical Analysis - When observational studies are used to establish the causal effects of treatments, the estimated effect is affected by treatment selection bias. The inverse...     

Model averaging for treatment effect estimation in subgroups

In many clinical trials, biological, pharmacological, or clinical information is used to define candidate subgroups of patients that might have a differential treatment effect. Once the trial results are available, interest will focus on subgroups with an increased treatment effect. Estimating a treatment effect for these groups, together with an adequate uncertainty statement is challenging, owing to the resulting “random high” / selection bias. In this paper, we will investigate Bayesian model averaging to address this problem. The general motivation for the use of model averaging is to realize that subgroup selection can be viewed as model selection, so that methods to deal with model selection uncertainty, such as model averaging, can be used also in this setting. Simulations are used to evaluate the performance of the proposed approach. We illustrate it on an example early‐phase clinical trial.     

Subgroup analysis and interpretation for phase 3 confirmatory trials: White paper of the EFSPI/PSI working group on subgroup analysis

Subgroup by treatment interaction assessments are routinely performed when analysing clinical trials and are particularly important for phase 3 trials where the results may affect regulatory labelling. Interpretation of such interactions is particularly difficult, as on one hand the subgroup finding can be due to chance, but equally such analyses are known to have a low chance of detecting differential treatment effects across subgroup levels, so may overlook important differences in therapeutic efficacy. EMA have therefore issued draft guidance on the use of subgroup analyses in this setting. Although this guidance provided clear proposals on the importance of pre‐specification of likely subgroup effects and how to use this when interpreting trial results, it is less clear which analysis methods would be reasonable, and how to interpret apparent subgroup effects in terms of whether further evaluation or action is necessary. A PSI/EFSPI Working Group has therefore been investigating a focused set of analysis approaches to assess treatment effect heterogeneity across subgroups in confirmatory clinical trials that take account of the number of subgroups explored and also investigating the ability of each method to detect such subgroup heterogeneity. This evaluation has shown that the plotting of standardised effects, bias‐adjusted bootstrapping method and SIDES method all perform more favourably than traditional approaches such as investigating all subgroup‐by‐treatment interactions individually or applying a global test of interaction. Therefore, these approaches should be considered to aid interpretation and provide context for observed results from subgroup analyses conducted for phase 3 clinical trials.     

Example of Block Designs for Plant and Tree Breeding Trials

Experimental designs which use extensive blocking and which are particularly useful in plant and tree breeding trials are discussed. They can be constructed either to accommodate field restrictions or take advantage of favourable plot layouts. Computer software is available to generate these design types for use in practice. Examples cover latinized row-column designs, t -latinized and partially-latinized designs and designs with unequal block sizes.     

Inferences on the difference between future observations for comparing two treatments

The comparison of two treatments with normally distributed data is considered. Inferences are considered based upon the difference between single potential future observations from each of the two treatments, which provides a useful and easily interpretable assessment of the difference between the two treatments. These methodologies combine information from a standard confidence interval analysis of the difference between the two treatment means, with information available from standard prediction intervals of future observations. Win-probabilities, which are the probabilities that a future observation from one treatment will be superior to a future observation from the other treatment, are a special case of these methodologies. The theoretical derivation of these methodologies is based upon inferences about the non-centrality parameter of a non-central t-distribution. Equal and unequal variance situations are addressed, and extensions to groups of future observations from the two treatments are also considered. Some examples and discussions of the methodologies are presented.