Decision‐making in drug development using a composite definition of success

Evidence‐based quantitative methodologies have been proposed to inform decision‐making in drug development, such as metrics to make go/no‐go decisions or predictions of success, identified with statistical significance of future clinical trials. While these methodologies appropriately address some critical questions on the potential of a drug, they either consider the past evidence without predicting the outcome of the future trials or focus only on efficacy, failing to account for the multifaceted aspects of a successful drug development. As quantitative benefit‐risk assessments could enhance decision‐making, we propose a more comprehensive approach using a composite definition of success based not only on the statistical significance of the treatment effect on the primary endpoint but also on its clinical relevance and on a favorable benefit‐risk balance in the next pivotal studies. For one drug, we can thus study several development strategies before starting the pivotal trials by comparing their predictive probability of success. The predictions are based on the available evidence from the previous trials, to which new hypotheses on the future development could be added. The resulting predictive probability of composite success provides a useful summary to support the discussions of the decision‐makers. We present a fictive, but realistic, example in major depressive disorder inspired by a real decision‐making case.     

On estimands and the analysis of adverse events in the presence of varying follow‐up times within the benefit assessment of therapies

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit‐risk ratio. The statistical analysis of AEs is complicated by the fact that the follow‐up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow‐up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta‐analyses of AE data and sketch possible solutions.     

Moving statistics beyond the individual clinical trial: applying decision science to optimize a clinical development plan

This paper describes how simple decision science techniques can be used to optimize the clinical development plan for a given compound. Using a case study from the stroke therapeutic area it is shown how methods such as decision trees can be utilized to describe, and adjudicate on, individual development plans. Terminology pertinent to decision sciences is described and areas where it is recommended statisticians should focus are highlighted. Copyright © 2005 John Wiley & Sons, Ltd.     

Statisticians in European regulatory agencies

The UK body of statisticians in the pharmaceutical industry, PSI, has called on heads of European regulatory agencies responsible for assessing applications for marketing authorizations for new medicines in the EU to employ full time statisticians. In order to assess the present situation a survey has been conducted to identify the number of agencies employing one or more full time statisticians. Out of 29 responding agencies, 12 employed one or more statisticians on a full time basis, whereas 17 did not. Among these 17, 7 involved external experts on a regular basis, 5 involved external statisticians on a case‐by‐case basis, whereas 5 never involved external statistical expertise. Failure to involve statisticians in the assessment of efficacy and safety of medicines does not automatically lead to reports of low quality or invalid assessment of benefit‐risk. However, in depth knowledge of statistical methodology is often necessary to uncover weaknesses and potentially biased efficacy estimates. This might be of importance for the final opinion on granting a marketing authorization, and statistical review should therefore be conducted by those who are professionally expert in the area. A positive trend toward an increased involvement of statistical expertise in the European network of regulatory agencies is observed. Copyright © 2009 John Wiley & Sons, Ltd.     

Bayesian joint modelling of benefit and risk in drug development

To gain regulatory approval, a new medicine must demonstrate that its benefits outweigh any potential risks, ie, that the benefit‐risk balance is favourable towards the new medicine. For transparency and clarity of the decision, a structured and consistent approach to benefit‐risk assessment that quantifies uncertainties and accounts for underlying dependencies is desirable. This paper proposes two approaches to benefit‐risk evaluation, both based on the idea of joint modelling of mixed outcomes that are potentially dependent at the subject level. Using Bayesian inference, the two approaches offer interpretability and efficiency to enhance qualitative frameworks. Simulation studies show that accounting for correlation leads to a more accurate assessment of the strength of evidence to support benefit‐risk profiles of interest. Several graphical approaches are proposed that can be used to communicate the benefit‐risk balance to project teams. Finally, the two approaches are illustrated in a case study using real clinical trial data.     

Leading beyond regulatory approval: Opportunities for statisticians to optimize evidence generation and impact clinical practice

The role and value of statistical contributions in drug development up to the point of health authority approval are well understood. But health authority approval is only a true ‘win’ if the evidence enables access and adoption into clinical practice. In today's complex and evolving healthcare environment, there is additional strategic evidence generation, communication, and decision support that can benefit from statistical contributions. In this article, we describe the history of medical affairs in the context of drug development, the factors driving post-approval evidence generation needs, and the opportunities for statisticians to optimize evidence generation for stakeholders beyond health authorities in order to ensure that new medicines reach appropriate patients.     

International Statistical Activities

The growth of the health care industry places increasing strain on available resources. As in other areas of social policy, health statisticians and health data are increasingly expected to provide keys to rational decision making. To accomplish this goal, the statistician and decision-maker need to interact to an increasing degree. The current issues in health policy and the statistician's contribution to policy analysis are discussed in the context of the National Center for Health Statistics experience. While the article focuses on health, it has implications for statisticians and policy formation in other fields.     

Utility values in health technology assessments: a statistician's perspective

This paper provides an introduction to utilities for statisticians working mainly in clinical research who have not had experience of health technology assessment work. Utility is the numeric valuation applied to a health state based on the preference of being in that state relative to perfect health. Utilities are often combined with survival data in health economic modelling to obtain quality‐adjusted life years. There are several methods available for deriving the preference weights and the health states to which they are applied, and combining them to estimate utilities, and the clinical statistician has valuable skills that can be applied in ensuring the robustness of the trial design, data collection and analyses to obtain and handle this data. In addition to raising awareness of the subject and providing source references, the paper outlines the concepts and approaches around utilities using examples, discusses some of the key issues, and proposes areas where statisticians can collaborate with health economic colleagues to improve the quality of this important element of health technology assessment. Copyright © 2014 John Wiley & Sons, Ltd.     

Global benefit-risk assessment in designing clinical trials and some statistical considerations of the method

When characterizing a therapy, the efficacy and the safety are two major aspects under consideration. In prescribing a therapy to a patient, a clinician puts the two aspects together and makes a decision based on a consolidated thought process. The global benefit-risk (GBR) measures proposed by Chuang-Stein et al. (Stat. Med. 1991; 10:1349-1359) are useful in facilitating the thinking, and creating the framework for making statistical comparisons based on benefit-risk point of view. This article describes how a GBR linear score was defined and used as the primary outcome measure in a clinical trial design. The robustness of the definitions of 'benefit' and 'risk' are evaluated using different criteria. The sensitivity of the pre-specified weights is also analyzed using alternative weights; one of those was determined by the relative to an identified distribution integral transformation approach (Biometrics 1958; 14:18-38). Statistical considerations are illustrated using pooled data from clinical trials studying antidepressant. The pros and cons for using GBR assessments in the setting of clinical trials are discussed.     

Statistical approaches for conducting network meta-analysis in drug development

We introduce health technology assessment and evidence synthesis briefly, and then concentrate on the statistical approaches used for conducting network meta-analysis (NMA) in the development and approval of new health technologies. NMA is an extension of standard meta-analysis where indirect as well as direct information is combined and can be seen as similar to the analysis of incomplete-block designs. We illustrate it with an example involving three treatments, using fixed-effects and random-effects models, and using frequentist and Bayesian approaches. As most statisticians in the pharmaceutical industry are familiar with SAS? software for analyzing clinical trials, we provide example code for each of the methods we illustrate. One issue that has been overlooked in the literature is the choice of constraints applied to random effects, and we show how this affects the estimates and standard errors and propose a symmetric set of constraints that is equivalent to most current practice. Finally, we discuss the role of statisticians in planning and carrying out NMAs and the strategy for dealing with important issues such as heterogeneity.     

REGULATORY UPDATES: ICH E14 draft consensus guideline on clinical evaluation of QT/QTc interval prolongation

ICH E14 calls for public comment by epidemiologists and statisticians on the practical implications of thresholds to be used for regulatory decision‐making. Readers involved in QT/QTc assessment in drug development and those with an interest in this area are encouraged to give the topic some thought and to be prepared to engage in public debate on the proposed ICH E14 guidance in late 2004 and early 2005. Copyright © 2004 John Wiley & Sons Ltd.     

Subgroup analyses in cost‐effectiveness analyses to support health technology assessments

‘Success’ in drug development is bringing to patients a new medicine that has an acceptable benefit–risk profile and that is also cost‐effective. Cost‐effectiveness means that the incremental clinical benefit is deemed worth paying for by a healthcare system, and it has an important role in enabling manufacturers to obtain new medicines to patients as soon as possible following regulatory approval. Subgroup analyses are increasingly being utilised by decision‐makers in the determination of the cost‐effectiveness of new medicines when making recommendations. This paper highlights the statistical considerations when using subgroup analyses to support cost‐effectiveness for a health technology assessment. The key principles recommended for subgroup analyses supporting clinical effectiveness published by Paget et al. are evaluated with respect to subgroup analyses supporting cost‐effectiveness. A health technology assessment case study is included to highlight the importance of subgroup analyses when incorporated into cost‐effectiveness analyses. In summary, we recommend planning subgroup analyses for cost‐effectiveness analyses early in the drug development process and adhering to good statistical principles when using subgroup analyses in this context. In particular, we consider it important to provide transparency in how subgroups are defined, be able to demonstrate the robustness of the subgroup results and be able to quantify the uncertainty in the subgroup analyses of cost‐effectiveness. Copyright © 2014 John Wiley & Sons, Ltd.     

Statistical problems in environmental research

Growing concern about the health effects of exposure to pollutants and other chemicals in the environment has stimulated new research to detect and quantify environmental hazards. This research has generated many interesting and challenging methodological problems for statisticians. One type of statistical research develops new methods for the design and analysis of individual studies. Because current research of this type is too diverse to summarize in a single article, we discuss current work in two areas of application: the carcinogen bioassay in small rodents and epidemiologic studies of air pollution. To assess the risk of a potentially harmful agent, one must frequently combine evidence from different and often quite dissimilar studies. Hence, this paper also discusses the central role of data synthesis in risk assessment, reviews some of the relevant statistical literature, and considers the role of statisticians in evaluating and combining evidence from diverse sources.     

Subgroup analyses of clinical effectiveness to support health technology assessments

Subgroup analysis is an integral part of access and reimbursement dossiers, in particular health technology assessment (HTA), and their HTA recommendations are often limited to subpopulations. HTA recommendations for subpopulations are not always clear and without controversies. In this paper, we review several HTA guidelines regarding subgroup analyses. We describe good statistical principles for subgroup analyses of clinical effectiveness to support HTAs and include case examples where HTA recommendations were given to subpopulations only. Unlike regulatory submissions, pharmaceutical statisticians in most companies have had limited involvement in the planning, design and preparation of HTA/payers submissions. We hope to change this by highlighting how pharmaceutical statisticians should contribute to payers' submissions. This includes early engagement in reimbursement strategy discussions to influence the design, analysis and interpretation of phase III randomized clinical trials as well as meta-analyses/network meta-analyses. The focus on this paper is on subgroup analyses relating to clinical effectiveness as we believe this is the first key step of statistical involvement and influence in the preparation of HTA and reimbursement submissions.     

Three-Factor Cartographic Representation

Statistics and statisticians have contributed to industry. Attention has been given to the appropriate training of statisticians for careers in industry. Statistics is used with benefit to industry in quality control, experimental design in research and development, and management decisions. Statistics can benefit from industry's assistance, the subject of this article. Five premises are set forth, three of them suggesting problems for statistics and industry. They relate to public understanding of statistics, the recruitment of students to statistics, the recruitment of statisticians to industry, and the relevance of research in statistics to the needs of industry. Thirteen recommendations are made on what industry can do for statistics and suggestions are made on how the American Statistical Association and the American Society for Quality Control can provide leadership in meeting the problems posed with the help of industry.     

Modelling and simulation in the pharmaceutical industry--some reflections

Modelling and simulation (M&S) is increasingly being applied in (clinical) drug development. It provides an opportune area for the community of pharmaceutical statisticians to pursue. In this article, we highlight useful principles behind the application of M&S. We claim that M&S should be focussed on decisions, tailored to its purpose and based in applied sciences, not relying entirely on data-driven statistical analysis. Further, M&S should be a continuous process making use of diverse information sources and applying Bayesian and frequentist methodology, as appropriate. In addition to forming a basis for analysing decision options, M&S provides a framework that can facilitate communication between stakeholders. Besides the discussion on modelling philosophy, we also describe how standard simulation practice can be ineffective and how simulation efficiency can often be greatly improved.     

Ethical Numbers: Ethics Training in U.S. Graduate Statistics Programs, 2013–2014

As important members of research teams, statisticians bear an ethical responsibility to analyze, interpret, and report data honestly and objectively. One way of reinforcing ethical responsibilities is through required courses covering a variety of ethics-related topics at the graduate level. We assessed ethics requirements for graduate-level statistics training programs in the United States for the 2013–2014 academic year using the websites of 88 universities, examining 103 biostatistics programs, and 136 statistics degree programs. We categorized programs’ ethics training requirements as required or not required. Thirty-one (35.1%) universities required an ethics course for at least some degree students. Sixty-two (25.5%) degree programs required an ethics course for at least some students. The majority (77.4%) of required courses were worth 0 or 1 credit. Of the 177 programs without an ethics requirement, 19 (10.7%) listed an ethics elective. Although a single ethics course is insufficient for instilling an ethical approach to science, degree programs that model expectations through coursework point to the value of ethics in science. More training programs should prepare statisticians to consider the ethical dimensions of their work through required coursework. Supplementary materials for this article are available online.     

A statistician's perspective on biomarkers in drug development

Biomarkers play an increasingly important role in many aspects of pharmaceutical discovery and development, including personalized medicine and the assessment of safety data, with heavy reliance being placed on their delivery. Statisticians have a fundamental role to play in ensuring that biomarkers and the data they generate are used appropriately and to address relevant objectives such as the estimation of biological effects or the forecast of outcomes so that claims of predictivity or surrogacy are only made based upon sound scientific arguments. This includes ensuring that studies are designed to answer specific and pertinent questions, that the analyses performed account for all levels and sources of variability and that the conclusions drawn are robust in the presence of multiplicity and confounding factors, especially as many biomarkers are multidimensional or may be an indirect measure of the clinical outcome. In all of these areas, as in any area of drug development, statistical best practice incorporating both scientific rigor and a practical understanding of the situation should be followed. This article is intended as an introduction for statisticians embarking upon biomarker-based work and discusses these issues from a practising statistician's perspective with reference to examples.     

Professional development of statisticians in the pharmaceutical sector: evolution over the past decade and into the future

The adoption of The International Conference on Harmonization Tripartite Guideline: Statistical Principles for Clinical Trials (ICH-E9) has provided a foundation for the application of statistical principles in clinical research and raised awareness of the value of a statistical contribution to the wider pharmaceutical R&D process. In addition, over the past decade globalization of the pharmaceutical R&D process and the measures taken to address reduced productivity and spiralling costs have impacted on the roles and career opportunities for statisticians working in the pharmaceutical sector. This has enhanced the need for continuing professional development to equip statisticians with the skills to fully contribute to creating innovative solutions. In the future, key areas of focus are the establishment of professional standards for statistical work and increasing the collaboration between statisticians working in industry, regulatory agencies and academia. In addition, the diversity of roles and potential career paths for statisticians embarking on a career in the pharmaceutical sector emphasizes the importance of mentoring and coaching. For the more experienced statisticians, there are unprecedented opportunities to lead and innovate.     

Why is Getting Rid of P-Values So Hard? Musings on Science and Statistics

ABSTRACT

The current concerns about reproducibility have focused attention on proper use of statistics across the sciences. This gives statisticians an extraordinary opportunity to change what are widely regarded as statistical practices detrimental to the cause of good science. However, how that should be done is enormously complex, made more difficult by the balkanization of research methods and statistical traditions across scientific subdisciplines. Working within those sciences while also allying with science reform movements—operating simultaneously on the micro and macro levels—are the key to making lasting change in applied science.