Subgroup analyses in cost‐effectiveness analyses to support health technology assessments

‘Success’ in drug development is bringing to patients a new medicine that has an acceptable benefit–risk profile and that is also cost‐effective. Cost‐effectiveness means that the incremental clinical benefit is deemed worth paying for by a healthcare system, and it has an important role in enabling manufacturers to obtain new medicines to patients as soon as possible following regulatory approval. Subgroup analyses are increasingly being utilised by decision‐makers in the determination of the cost‐effectiveness of new medicines when making recommendations. This paper highlights the statistical considerations when using subgroup analyses to support cost‐effectiveness for a health technology assessment. The key principles recommended for subgroup analyses supporting clinical effectiveness published by Paget et al. are evaluated with respect to subgroup analyses supporting cost‐effectiveness. A health technology assessment case study is included to highlight the importance of subgroup analyses when incorporated into cost‐effectiveness analyses. In summary, we recommend planning subgroup analyses for cost‐effectiveness analyses early in the drug development process and adhering to good statistical principles when using subgroup analyses in this context. In particular, we consider it important to provide transparency in how subgroups are defined, be able to demonstrate the robustness of the subgroup results and be able to quantify the uncertainty in the subgroup analyses of cost‐effectiveness. Copyright © 2014 John Wiley & Sons, Ltd.     

Utility values in health technology assessments: a statistician's perspective

This paper provides an introduction to utilities for statisticians working mainly in clinical research who have not had experience of health technology assessment work. Utility is the numeric valuation applied to a health state based on the preference of being in that state relative to perfect health. Utilities are often combined with survival data in health economic modelling to obtain quality‐adjusted life years. There are several methods available for deriving the preference weights and the health states to which they are applied, and combining them to estimate utilities, and the clinical statistician has valuable skills that can be applied in ensuring the robustness of the trial design, data collection and analyses to obtain and handle this data. In addition to raising awareness of the subject and providing source references, the paper outlines the concepts and approaches around utilities using examples, discusses some of the key issues, and proposes areas where statisticians can collaborate with health economic colleagues to improve the quality of this important element of health technology assessment. Copyright © 2014 John Wiley & Sons, Ltd.     

Longitudinal dynamic analyses of cognition in the health and retirement study panel

The purpose of this paper is to highlight some classic issues in the measurement of change and to show how contemporary solutions can be used to deal with some of these issues. Five classic issues will be raised here: (1) Separating individual changes from group differences; (2) options for incomplete longitudinal data over time, (3) options for nonlinear changes over time; (4) measurement invariance in studies of changes over time; and (5) new opportunities for modeling dynamic changes. For each issue we will describe the problem, and then review some contemporary solutions to these problems base on Structural Equation Models (SEM). We will fit these SEM to using existing panel data from the Health & Retirement Study (HRS) cognitive variables. This is not intended as an overly technical treatment, so only a few basic equations are presented, examples will be displayed graphically, and more complete references to the contemporary solutions will be given throughout.     

A critical review of graphics for subgroup analyses in clinical trials

Subgroup analyses are a routine part of clinical trials to investigate whether treatment effects are homogeneous across the study population. Graphical approaches play a key role in subgroup analyses to visualise effect sizes of subgroups, to aid the identification of groups that respond differentially, and to communicate the results to a wider audience. Many existing approaches do not capture the core information and are prone to lead to a misinterpretation of the subgroup effects. In this work, we critically appraise existing visualisation techniques, propose useful extensions to increase their utility and attempt to develop an effective visualisation approach. We focus on forest plots, UpSet plots, Galbraith plots, subpopulation treatment effect pattern plot, and contour plots, and comment on other approaches whose utility is more limited. We illustrate the methods using data from a prostate cancer study.     

Improving the contributions of technology assessment to the health care system of the U.S.A.

Summary Based on 14 case studies of highly effective therapies and the reasons they succeeded less frequently than they could, we propose a variety of steps to improve the health care system of the U.S.A. Whatever proposal emerges from current national debates until innovations are shown to be safe and effective, they should not be supported; when slightly better technologies are much more expensive than other good ones we need to consider appropriate choices carefully; simplified billing and bookkeping would reduce our costs; when a technology is rapidly introduced cautionnary measures may be needed; tracking immunization and repairing their omissions requires a new system; educational programs such as seen effective in hypertension should be applied in other areas such as vaccination; in organ transplantation the nation should consider “presumed consent”; our payment system sometimes creates perverse incentives and therefore needs review; and the preferences of the public in allocation of health resources need to be discovered once the public is informed about the issues. Research supported by Andrew W. Mellon Foundation.     

Choice of alpha spending function and time points in clinical trials with one or two interim analyses

One characterization of group sequential methods uses alpha spending functions to allocate the false positive rate throughout a study. We consider and evaluate several such spending functions as well as the time points of the interim analyses at which they apply. In addition, we evaluate the double triangular test as an alternative procedure that allows for early termination of the trial not only due to efficacy differences between treatments, but also due to lack of such differences. We motivate and illustrate our work by reference to the analysis of survival data from a proposed oncology study. Such group sequential procedures with one or two interim analyses are only slightly less powerful than fixed sample trials, but provide for the strong possibility of early stopping. Therefore, in all situations where they can practically be applied, we recommend their routine use in clinical trials. The double triangular test provides a suitable alternative to the group sequential procedures in that they do not provide for early stopping with acceptance of the null hypothesis. Again, there is only a modest loss in power relative to fixed sample tests. Copyright © 2004 John Wiley & Sons, Ltd.     

Lessons from meta‐analyses of randomized clinical trials for analysis of distributed networks of observational databases

Networks of constellations of longitudinal observational databases, often electronic medical records or transactional insurance claims or both, are increasingly being used for studying the effects of medicinal products in real‐world use. Such databases are frequently configured as distributed networks. That is, patient‐level data are kept behind firewalls and not communicated outside of the data vendor other than in aggregate form. Instead, data are standardized across the network, and queries of the network are executed locally by data partners, and summary results provided to a central research partner(s) for amalgamation, aggregation, and summarization. Such networks can be huge covering years of data on upwards of 100 million patients. Examples of such networks include the FDA Sentinel Network, ASPEN, CNODES, and EU‐ADR. As this is a new emerging field, we note in this paper the conceptual similarities and differences between the analysis of distributed networks and the now well‐established field of meta‐analysis of randomized clinical trials (RCTs). We recommend, wherever appropriate, to apply learnings from meta‐analysis to help guide the development of distributed network analyses of longitudinal observational databases.     

A note on sample size savings with the use of a single well‐controlled clinical trial to support the efficacy of a new drug

In terms of the risk of making a Type I error in evaluating a null hypothesis of equality, requiring two independent confirmatory trials with two‐sided p‐values less than 0.05 is equivalent to requiring one confirmatory trial with two‐sided p‐value less than 0.001 25. Furthermore, the use of a single confirmatory trial is gaining acceptability, with discussion in both ICH E9 and a CPMP Points to Consider document. Given the growing acceptance of this approach, this note provides a formula for the sample size savings that are obtained with the single clinical trial approach depending on the levels of Type I and Type II errors chosen. For two replicate trials each powered at 90%, which corresponds to a single larger trial powered at 81%, an approximate 19% reduction in total sample size is achieved with the single trial approach. Alternatively, a single trial with the same sample size as the total sample size from two smaller trials will have much greater power. For example, in the case where two trials are each powered at 90% for two‐sided α=0.05 yielding an overall power of 81%, a single trial using two‐sided α=0.001 25 would have 91% power. Copyright © 2004 John Wiley & Sons, Ltd.     

Application of Bayesian analyses to doubly randomized delayed start,matched control designs to demonstrate disease modification

Disease modification is a primary therapeutic aim when developing treatments for most chronic progressive diseases. The best treatments do not simply affect disease symptoms but fundamentally improve disease course by slowing, halting, or reversing disease progression. One of many challenges for establishing disease modification relates to the identification of adequate analytic tools to show differences in a disease course following intervention. Traditional approaches rely on the comparisons of slopes or noninferiority margins. However, it has proven difficult to conclusively demonstrate disease modification using such approaches. To address these challenges, we propose a novel adaptation of the delayed start study design that incorporates posterior probabilities identified by hierarchical Bayesian inference approaches to establish evidence for disease modification. Our models compare the size of treatment differences at the end of the delayed start period with those at the end of the early start period. Simulations that compare several models are provided. These include general linear models, repeated measures models, spline models, and model averaging. Our work supports the superiority of model averaging for accurately characterizing complex data that arise in real world applications. This novel approach has been applied to the design of an ongoing, doubly randomized, matched control study that aims to show disease modification in young persons with schizophrenia (the Disease Recovery Evaluation and Modification (DREaM) study). The application of this Bayesian methodology to the DREaM study highlights the value of this approach and demonstrates many practical challenges that must be addressed when implementing this methodology in a real world trial.     

Case study in the Bayesian analysis of a cost‐effectiveness trial in the evaluation of health care technologies: Depression

We present a case study based on a depression study that will illustrate the use of Bayesian statistics in the economic evaluation of cost‐effectiveness data, demonstrate the benefits of the Bayesian approach (whilst honestly recognizing any deficiencies) with respect to frequentist methods, and provide details of using the methods, including computer code where appropriate. Copyright © 2003 John Wiley & Sons, Ltd.     

Application of a stochastic model to response assessments in a study of squamous cell carcinoma of the head and neck

This paper discusses the application of a stochastic model in the analysis of response assessments made at various time points in a clinical trial of patients with squamous cell carcinoma of the head and neck. The transition rates and probabilities during treatment administration are derived using maximum likelihood methods. The results are then compared with the standard analyses used in solid tumour studies. Stochastic modelling is considered to complement the standard analyses, provide a holistic approach and better explain the underlying disease process. Copyright © 2012 John Wiley & Sons, Ltd.     

Testing the equality of diagnostic effectiveness of one measure with respect to k different features

In several cases the same measurement is used as a marker for two or more population features, and it is useful to test whether this measurement has the same diagnostic effectiveness with respect to different features. In this paper we use the area under receiver operating characteristic curve as index for the discriminatory power among continuous variables and population features (eventuality, two or more diseases), and we propose a test to contrast the equality of the diagnostic effectiveness of this measurement.     

Prediction of bankruptcy using support vector machines: an application to bank bankruptcy

The purpose of this study was to apply support vector machines (SVMs) to bank bankruptcy analysis using practical steps. Although the prediction of the financial distress of companies is done using several statistical and machine learning techniques, bank classification and bankruptcy prediction still need to be investigated because few investigations have been conducted in this field of banking. In this study, SVMs were implemented to analyse financial ratios. Data sets from Turkish commercial banks were used. This study shows that SVMs with the Gaussian kernel are capable of extracting useful information from financial data and can be used as part of an early warning system.     

Using multivariate rank sum tests to evaluate effectiveness of computer applications in teaching business statistics

Arguments about using computer facilities in classroom teaching have received a lot of attention over time. Using the computer facilities will be helpful to demonstrate real-world applications, while poor data or inappropriate case studies might hinder the applications of the computer programs in classroom teaching. In this paper, we examine the impacts that using computer programs to teach business statistics have on students in the Krannert School of Management at Purdue University. The results show that students are attracted to the interactive computer programs designed for the business statistics course, and students are more motivated to attend classes when computer programs are applied in teaching. Furthermore, computer programs help students to understand confusing topics, and students feel that teaching them to use computer facilities really improves their own abilities to apply similar programs in analyzing real-world problems.     

The single-index support vector regression model to address the problem of high dimensionality

ABSTRACT

The last few years, the applications of Support Vector Machine (SVM) for solving classification and regression problems have been increasing, due to its high performance and ability to transform the non-linear relationships among variables to linear form by employing the kernel idea (kernel function). In this work, we develop a semi-parametric approach to fit single-index models to deal with high-dimensional problems. To achieve this goal, we use support vector regression (SVR) for estimating the unknown nonparametric link function, while the single-index is determined by using the semi-parametric least squares method (Ichimura 1993). This development enhances the ability of SVR to solve high-dimensional problem. We design a three simulation examples with high-dimensional problems (linear and nonlinear). The simulations demonstrate the superior performance of the proposed method versus the standard SVR method. This is further illustrated by applying the real data.     

Re-randomization tests as sensitivity analyses to confirm immunological noninferiority of an investigational vaccine: Case study

Here we present as case study how re-randomization tests were performed in two randomized, controlled clinical trials as sensitivity analyses, as recommended by the United States Food and Drug Administration in the context of adaptive randomization. This was done to confirm primary conclusions on immunological noninferiority of an investigational new fully liquid presentation of a quadrivalent cross-reacting material conjugate meningococcal vaccine (MenACWY-CRM), over its licensed lyophilized/liquid presentation. In two phase 2b studies (Study #1: NCT03652610; Study #2: NCT03433482), noninferiority of the fully liquid presentation of MenACWY-CRM to the licensed presentation was assessed and demonstrated for immune responses against meningococcal serogroup A (MenA), the only vaccine component modified from lyophilized to liquid in the new presentation. The original vaccine assignment algorithm, with a minimization procedure accounting for center or center within age strata, was used to re-randomize participants belonging to the fully liquid and licensed vaccine groups while keeping antibody responses, covariates and entry order as observed. Test statistics under re-randomization were generated according to the ANCOVA model used in the primary analysis. To confirm immunological noninferiority following re-randomization, the corresponding p-values had to be <0.025. For both studies and all primary objective evaluations, the re-randomization p-values were well below 0.025 (0.0004 for Study #1; 0.0001 for the two co-primary endpoints in Study #2). Re-randomization tests performed to comply with a regulatory request confirmed the primary conclusions of immunological noninferiority for the MenA of the fully liquid compared to the licensed vaccine presentation.     

Sparsely clustered survival data: application to the evaluation of the safety and effectiveness of medical devices

The effectiveness and safety of implantable medical devices is a critical public health concern. We consider analysis of data in which it is of interest to compare devices but some individuals may be implanted with two or more devices. Our motivating example is based on orthopedic devices, where the same individual can be implanted with as many as two devices for the same joint but on different sides of the body, referred to as bilateral cases. Different methods of analysis are considered in a simulation study and real data example, including both marginal and conditional survival models, fitting single and separate models for bilateral and non-bilateral cases, and combining estimates from these two models. The results of simulations suggest that in the context of orthopedic devices, where implants failures are rare, models fit on both bilateral and non-bilateral cases simultaneously could be quite misleading, and that combined estimates from fitting two separate models performed better under homogeneity. A real data example illustrates the issues surrounding analysis of orthopedic device data with bilateral cases. Our findings suggest that research studies of orthopedic devices should at minimum consider fitting separate models to bilateral and non-bilateral cases.     

Alternative sensitivity analyses for regression estimates of treatment effects to unobserved confounding in binary and survival data

Statistical Methods & Applications - Estimates of treatment effects in non-experimental studies are subject to bias owing to unobserved confounding. It is desirable to assess the sensitivity of...     

Comparison of the effectiveness of forecasts obtained by means of selected probability functions with respect to forecast error distributions

The forecasting of sales in a company is one of the crucial challenges that must be faced. Nowadays, there is a large spectrum of methods that enable making reliable forecasts. However, sometimes the nature of time series excludes many well-known and widely used forecasting methods (e.g., econometric models). Therefore, the authors decided to forecast on the basis of a seasonally adjusted median of selected probability distributions. The obtained forecasts were verified by means of distributions of the Theil U² coefficient and unbiasedness coefficient.