Adaptive designs for pivotal trials: discussion points from the PSI Adaptive Design Expert Group

The Committee for Medicinal Products for Human Use (CHMP) is currently preparing a guideline on 'methodological issues in confirmatory clinical trials with flexible design and analysis plan'. PSI (Statisticians in the Pharmaceutical Industry) sponsored a meeting of pharmaceutical statisticians with an interest in the area to share experiences and identify potential opportunities for adaptive designs in late-phase clinical drug development. This article outlines the issues raised, resulting discussions and consensus views reached. Adaptive designs have potential utility in late-phase clinical development. Sample size re-estimation seems to be valuable and widely accepted, but should be made independent of the observed treatment effect where possible. Where unblinding is necessary, careful consideration needs to be given to preserving the integrity of the trial. An area where adaptive designs can be particularly beneficial is to allow dose selection in pivotal trials via adding/dropping treatment arms; for example, combining phase II and III of the drug development program. The more adaptations made during a late-phase clinical trial, the less likely that the clinical trial would be considered as a confirmatory trial. In all cases it would be advisable to consult with regulatory agencies at the protocol design stage. All involved should remain open to scientifically valid opportunities to improve drug development.     

European Federation of Statisticians in the Pharmaceutical Industry's position on access to clinical trial data

The European Federation of Statisticians in the Pharmaceutical Industry (EFSPI) believes access to clinical trial data should be implemented in a way that supports good research, avoids misuse of such data, lies within the scope of the original informed consent and fully protects patient confidentiality. In principle, EFSPI supports responsible data sharing. EFSPI acknowledges it is in the interest of patients that their data are handled in a strictly confidential manner to avoid misuse under all possible circumstances. It is also in the interest of the altruistic nature of patients participating in trials that such data will be used for further development of science as much as possible applying good statistical principles. This paper summarises EFSPI's position on access to clinical trial data. The position was developed during the European Medicines Agency (EMA) advisory process and before the draft EMA policy on publication and access to clinical trial data was released for consultation; however, the EFSPI's position remains unchanged following the release of the draft policy. Finally, EFSPI supports a need for further guidance to be provided on important technical aspects relating to re‐analyses and additional analyses of clinical trial data, for example, multiplicity, meta‐analysis, subgroup analyses and publication bias. Copyright © 2013 John Wiley & Sons, Ltd.     

Automatic selection of the proper family for simultaneous confidence intervals

Statisticians often employ simultaneous confidence intervals to reduce the likelihood of their drawing false conclusions when they must make a number of comparisons. To do this properly, it is necessary to consider the family of comparisons over which simultaneous confidence must be assured. Sometimes it is not clear what family of comparisons is appropriate. We describe how computer software can monitor the types of contrasts a user examines, and select the smallest family of contrasts that is likely to be of interest. We also describe how to calculate simultaneous confidence intervals for these families using a hybrid of the Bonferroni and Scheffé methods. Our method is especially suitable for problems with discrete and continuous predictors.     

Evaluation of alternative confidence intervals to address non-inferiority through the stratified difference between proportions

The confidence interval (CI) for the difference between two proportions has been an important and active research topic, especially in the context of non-inferiority hypothesis testing. Issues concerning the Type 1 error rate, power, coverage rate and aberrations have been extensively studied for non-stratified cases. However, stratified confidence intervals are frequently used in non-inferiority trials and similar settings. In this paper, several methods for stratified confidence intervals for the difference between two proportions, including existing methods and novel extensions from unstratified CIs, are evaluated across different scenarios. When sparsity across the strata is not a concern, adding imputed observations to the stratification analysis can strengthen Type-1 error control without substantial loss of power. When sparseness of data is a concern, most of the evaluated methods fail to control Type-1 error; the modified stratified t-test CI is an exception. We recommend the modified stratified t-test CI as the most useful and flexible method across the respective scenarios; the modified stratified Wald CI may be useful in settings where sparsity is unlikely. These findings substantially contribute to the application of stratified CIs for non-inferiority testing of differences between two proportions.     

Simultaneous Confidence Intervals for Ratios of Fixed Effect Parameters in Linear Mixed Models

In multiple comparisons of fixed effect parameters in linear mixed models, treatment effects can be reported as relative changes or ratios. Simultaneous confidence intervals for such ratios had been previously proposed based on Bonferroni adjustments or multivariate normal quantiles accounting for the correlation among the multiple contrasts. We propose Fieller-type intervals using multivariate t quantiles and the application of Markov chain Monte Carlo techniques to sample from the joint posterior distribution and construct percentile-based simultaneous intervals. The methods are compared in a simulation study including bioassay problems with random intercepts and slopes, repeated measurements designs, and multicenter clinical trials.     

Issues in applying recent CPMP ‘Points to Consider’ and FDA guidance documents with biostatistical implications

The International Conference on Harmonisation guideline ‘Statistical Principles for Clinical Trials’ was adopted by the Committee for Proprietary Medicinal Products (CPMP) in March 1998, and consequently is operational in Europe. Since then more detailed guidance on selected topics has been issued by the CPMP in the form of ‘Points to Consider’ documents. The intent of these was to give guidance particularly to non‐statistical reviewers within regulatory authorities, although of course they also provide a good source of information for pharmaceutical industry statisticians. In addition, the Food and Drug Administration has recently issued a draft guideline on data monitoring committees. In November 2002 a one‐day discussion forum was held in London by Statisticians in the Pharmaceutical Industry (PSI). The aim of the meeting was to discuss how statisticians were responding to some of the issues covered in these new guidelines, and to document consensus views where they existed. The forum was attended by industry, academic and regulatory statisticians. This paper outlines the questions raised, resulting discussions and consensus views reached. It is clear from the guidelines and discussions at the workshop that the statistical analysis strategy must be planned during the design phase of a clinical trial and carefully documented. Once the study is complete the analysis strategy should be thoughtfully executed and the findings reported. Copyright © 2003 John Wiley & Sons, Ltd.     

Assessing Statistical Results: Magnitude,Precision, and Model Uncertainty

ABSTRACT

Evaluating the importance and the strength of empirical evidence requires asking three questions: First, what are the practical implications of the findings? Second, how precise are the estimates? Confidence intervals provide an intuitive way to communicate precision. Although nontechnical audiences often misinterpret confidence intervals (CIs), I argue that the result is less dangerous than the misunderstandings that arise from hypothesis tests. Third, is the model correctly specified? The validity of point estimates and CIs depends on the soundness of the underlying model.     

Missing data: Discussion points from the PSI missing data expert group

The Points to Consider Document on Missing Data was adopted by the Committee of Health and Medicinal Products (CHMP) in December 2001. In September 2007 the CHMP issued a recommendation to review the document, with particular emphasis on summarizing and critically appraising the pattern of drop‐outs, explaining the role and limitations of the ‘last observation carried forward’ method and describing the CHMP's cautionary stance on the use of mixed models. In preparation for the release of the updated guidance document, statisticians in the Pharmaceutical Industry held a one‐day expert group meeting in September 2008. Topics that were debated included minimizing the extent of missing data and understanding the missing data mechanism, defining the principles for handling missing data and understanding the assumptions underlying different analysis methods. A clear message from the meeting was that at present, biostatisticians tend only to react to missing data. Limited pro‐active planning is undertaken when designing clinical trials. Missing data mechanisms for a trial need to be considered during the planning phase and the impact on the objectives assessed. Another area for improvement is in the understanding of the pattern of missing data observed during a trial and thus the missing data mechanism via the plotting of data; for example, use of Kaplan–Meier curves looking at time to withdrawal. Copyright © 2009 John Wiley & Sons, Ltd.     

Confidence intervals and bands for the binormal ROC curve revisited

Two types of confidence intervals (CIs) and confidence bands (CBs) for the receiver operating characteristic (ROC) curve are studied: pointwise CIs and simultaneous CBs. An optimized version of the pointwise CI with the shortest width is developed. A new ellipse-envelope simultaneous CB for the ROC curve is suggested as an adaptation of the Working-Hotelling-type CB implemented in a paper by Ma and Hall (1993). Statistical simulations show that our ellipse-envelope CB covers the true ROC curve with a probability close to nominal while the coverage probability of the Ma and Hall CB is significantly smaller. Simulations also show that our CI for the area under the ROC curve is close to nominal while the coverage probability of the CI suggested by Hanley and McNail (1982) uniformly overestimates the nominal value. Two examples illustrate our simultaneous ROC bands: radiation dose estimation from time to vomiting and discrimination of breast cancer from benign abnormalities using electrical impedance measurements.     

Simultaneous confidence interval construction for many-to-one comparisons of proportion differences based on correlated paired data

In some medical researches such as ophthalmological, orthopaedic and otolaryngologic studies, it is often of interest to compare multiple groups with a control using data collected from paired organs of patients. The major difficulty in performing the data analysis is to adjust the multiplicity between the comparison of multiple groups, and the correlation within the same patient''s paired organs. In this article, we construct asymptotic simultaneous confidence intervals (SCIs) for many-to-one comparisons of proportion differences adjusting for multiplicity and the correlation. The coverage probabilities and widths of the proposed CIs are evaluated by Monte Carlo simulation studies. The methods are illustrated by a real data example.     

Conditional approval: discussion points from the PSI conditional approval expert group

The European Agency for the Evaluation of Medicinal Products has recently completed the consultation of a draft guidance on how to implement conditional approval. This route of application is available for orphan drugs, emergency situations and serious debilitating or life-threatening diseases. Although there has been limited experience in implementing conditional approval to date, PSI (Statisticians in the Pharmaceutical Industry) sponsored a meeting of pharmaceutical statisticians with an interest in the area to discuss potential issues. This article outlines the issues raised and resulting discussions, based on the group's interpretation of the legislation. Conditional approval seems to fit well with the accepted regulatory strategy in HIV. In oncology, conditional approval may be most likely when (a) compelling phase II data are available using accepted clinical outcomes (e.g. progression/recurrence-free survival or overall survival) and Phase III has been planned or started, or (b) when data are available using a surrogate endpoint for clinical outcome (e.g. response rate or biochemical measures) from a single-arm study in rare tumours with high response, compared with historical data. The use of interim analyses in Phase III for supporting conditional approval raises some challenging issues regarding dissemination of information, maintenance of blinding, potential introduction of bias, ethics, switching, etc.     

Exact confidence intervals for the shape parameter of the gamma distribution

In this paper exact confidence intervals (CIs) for the shape parameter of the gamma distribution are constructed using the method of Bølviken and Skovlund [Confidence intervals from Monte Carlo tests. J Amer Statist Assoc. 1996;91:1071–1078]. The CIs which are based on the maximum likelihood estimator or the moment estimator are compared to bootstrap CIs via a simulation study.     

Performance of some multiple testing procedures to compare three doses of a test drug and placebo

A study design with two or more doses of a test drug and placebo is frequently used in clinical drug development. Multiplicity issues arise when there are multiple comparisons between doses of test drug and placebo, and also when there are comparisons of doses with one another. An appropriate analysis strategy needs to be specified in advance to avoid spurious results through insufficient control of Type I error, as well as to avoid the loss of power due to excessively conservative adjustments for multiplicity. For evaluation of alternative strategies with possibly complex management of multiplicity, we compare the performance of several testing procedures through the simulated data that represent various patterns of treatment differences. The purpose is to identify which methods perform better or more robustly than the others and under what conditions. Copyright © 2005 John Wiley & Sons, Ltd.     

Evaluating the Performance of Simultaneous Stepwise Confidence Intervals for the Difference between two Poisson Rates

We consider the problem of simultaneously estimating Poisson rate differences via applications of the Hsu and Berger stepwise confidence interval method (termed HBM), where comparisons to a common reference group are performed. We discuss continuity-corrected confidence intervals (CIs) and investigate the HBM performance with a moment-based CI, and uncorrected and corrected for continuity Wald and Pooled confidence intervals (CIs). Using simulations, we compare nine individual CIs in terms of coverage probability and the HBM with nine intervals in terms of family-wise error rate (FWER) and overall and local power. The simulations show that these statistical properties depend highly on parameter settings.     

A statistician's perspective on biomarkers in drug development

Biomarkers play an increasingly important role in many aspects of pharmaceutical discovery and development, including personalized medicine and the assessment of safety data, with heavy reliance being placed on their delivery. Statisticians have a fundamental role to play in ensuring that biomarkers and the data they generate are used appropriately and to address relevant objectives such as the estimation of biological effects or the forecast of outcomes so that claims of predictivity or surrogacy are only made based upon sound scientific arguments. This includes ensuring that studies are designed to answer specific and pertinent questions, that the analyses performed account for all levels and sources of variability and that the conclusions drawn are robust in the presence of multiplicity and confounding factors, especially as many biomarkers are multidimensional or may be an indirect measure of the clinical outcome. In all of these areas, as in any area of drug development, statistical best practice incorporating both scientific rigor and a practical understanding of the situation should be followed. This article is intended as an introduction for statisticians embarking upon biomarker-based work and discusses these issues from a practising statistician's perspective with reference to examples.     

Empirical Bayes Confidence Intervals for Means of Natural Exponential Family-Quadratic Variance Function Distributions with Application to Small Area Estimation

Abstract.  The paper develops empirical Bayes (EB) confidence intervals for population means with distributions belonging to the natural exponential family-quadratic variance function (NEF-QVF) family when the sample size for a particular population is moderate or large. The basis for such development is to find an interval centred around the posterior mean which meets the target coverage probability asymptotically, and then show that the difference between the coverage probabilities of the Bayes and EB intervals is negligible up to a certain order. The approach taken is Edgeworth expansion so that the sample sizes from the different populations need not be significantly large. The proposed intervals meet the target coverage probabilities asymptotically, and are easy to construct. We illustrate use of these intervals in the context of small area estimation both through real and simulated data. The proposed intervals are different from the bootstrap intervals. The latter can be applied quite generally, but the order of accuracy of these intervals in meeting the desired coverage probability is unknown.     

Proposed best practice for projects that involve modelling and simulation

Modelling and simulation has been used in many ways when developing new treatments. To be useful and credible, it is generally agreed that modelling and simulation should be undertaken according to some kind of best practice. A number of authors have suggested elements required for best practice in modelling and simulation. Elements that have been suggested include the pre‐specification of goals, assumptions, methods, and outputs. However, a project that involves modelling and simulation could be simple or complex and could be of relatively low or high importance to the project. It has been argued that the level of detail and the strictness of pre‐specification should be allowed to vary, depending on the complexity and importance of the project. This best practice document does not prescribe how to develop a statistical model. Rather, it describes the elements required for the specification of a project and requires that the practitioner justify in the specification the omission of any of the elements and, in addition, justify the level of detail provided about each element. This document is an initiative of the Special Interest Group for modelling and simulation. The Special Interest Group for modelling and simulation is a body open to members of Statisticians in the Pharmaceutical Industry and the European Federation of Statisticians in the Pharmaceutical Industry. Examples of a very detailed specification and a less detailed specification are included as appendices.     

Confidence bands for the difference of two survival functions under the additive risk model

In many clinical studies, a commonly encountered problem is to compare the survival probabilities of two treatments for a given patient with a certain set of covariates, and there is often a need to make adjustments for other covariates that may affect outcomes. One approach is to plot the difference between the two subject-specific predicted survival estimates with a simultaneous confidence band. Such a band will provide useful information about when these two treatments differ and which treatment has a better survival probability. In this paper, we show how to construct such a band based on the additive risk model and we use the martingale central limit theorem to derive its asymptotic distribution. The proposed method is evaluated from a simulation study and is illustrated with two real examples.     

Divergence-based confidence intervals in false-positive misclassification model

In this article, we introduce minimum divergence estimators of parameters of a binary response model when data are subject to false-positive misclassification and obtained using a double-sampling plan. Under this set up, the problem of goodness-of-fit is considered and divergence-based confidence intervals (CIs) for a population proportion parameter are derived. A simulation experiment is carried out to compare the coverage probabilities of the new CIs. An application to real data is also given.