A frequentist assessment of Bayesian inclusion probabilities for screening predictors

Bayesian inclusion probabilities have become a popular tool for variable assessment. From a frequentist perspective, it is often difficult to evaluate these probabilities as typically no Type I error rates are considered, neither are any explorations of power of the methods given. This paper considers how a frequentist may evaluate Bayesian inclusion probabilities for screening predictors. This evaluation looks at both unrestricted and restricted model spaces and develops a framework which a frequentist can utilize inclusion probabilities that preserve Type I error rates. Furthermore, this framework is applied to an analysis of the Arabidopsis thaliana with respect to determining quantitative trait loci associated with cotelydon opening angle.     

Outcome-adaptive allocation with natural lead-in for three-group trials with binary outcomes

ABSTRACT

Just as Bayes extensions of the frequentist optimal allocation design have been developed for the two-group case, we provide a Bayes extension of optimal allocation in the three-group case. We use the optimal allocations derived by Jeon and Hu [Optimal adaptive designs for binary response trials with three treatments. Statist Biopharm Res. 2010;2(3):310–318] and estimate success probabilities for each treatment arm using a Bayes estimator. We also introduce a natural lead-in design that allows adaptation to begin as early in the trial as possible. Simulation studies show that the Bayesian adaptive designs simultaneously increase the power and expected number of successfully treated patients compared to the balanced design. And compared to the standard adaptive design, the natural lead-in design introduced in this study produces a higher expected number of successes whilst preserving power.     

Bayesian single-arm phase II trial designs with time-to-event endpoints

For the cancer clinical trials with immunotherapy and molecularly targeted therapy, time-to-event endpoint is often a desired endpoint. In this paper, we present an event-driven approach for Bayesian one-stage and two-stage single-arm phase II trial designs. Two versions of Bayesian one-stage designs were proposed with executable algorithms and meanwhile, we also develop theoretical relationships between the frequentist and Bayesian designs. These findings help investigators who want to design a trial using Bayesian approach have an explicit understanding of how the frequentist properties can be achieved. Moreover, the proposed Bayesian designs using the exact posterior distributions accommodate the single-arm phase II trials with small sample sizes. We also proposed an optimal two-stage approach, which can be regarded as an extension of Simon's two-stage design with the time-to-event endpoint. Comprehensive simulations were conducted to explore the frequentist properties of the proposed Bayesian designs and an R package BayesDesign can be assessed via R CRAN for convenient use of the proposed methods.     

Bayesian single-to-double arm transition design using both short-term and long-term endpoints

The choice between single-arm designs versus randomized double-arm designs has been contentiously debated in the literature of phase II oncology trials. Recently, as a compromise, the single-to-double arm transition design was proposed, combining the two designs into one trial over two stages. Successful implementation of the two-stage transition design requires a suspension period at the end of the first stage to collect the response data of the already enrolled patients. When the evaluation of the primary efficacy endpoint is overly long, the between-stage suspension period may unfavorably prolong the trial duration and cause a delay in treating future eligible patients. To accelerate the trial, we propose a Bayesian single-to-double arm design with short-term endpoints (BSDS), where an intermediate short-term endpoint is used for making early termination decisions at the end of the single-arm stage, followed by an evaluation of the long-term endpoint at the end of the subsequent double-arm stage. Bayesian posterior probabilities are used as the primary decision-making tool at the end of the trial. Design calibration steps are proposed for this Bayesian monitoring process to control the frequentist operating characteristics and minimize the expected sample size. Extensive simulation studies have demonstrated that our design has comparable power and average sample size but a much shorter trial duration than conventional single-to-double arm design. Applications of the design are illustrated using two phase II oncology trials with binary endpoints.     

Inference and Decision Making for 21st-Century Drug Development and Approval

ABSTRACT

The cost and time of pharmaceutical drug development continue to grow at rates that many say are unsustainable. These trends have enormous impact on what treatments get to patients, when they get them and how they are used. The statistical framework for supporting decisions in regulated clinical development of new medicines has followed a traditional path of frequentist methodology. Trials using hypothesis tests of “no treatment effect” are done routinely, and the p-value < 0.05 is often the determinant of what constitutes a “successful” trial. Many drugs fail in clinical development, adding to the cost of new medicines, and some evidence points blame at the deficiencies of the frequentist paradigm. An unknown number effective medicines may have been abandoned because trials were declared “unsuccessful” due to a p-value exceeding 0.05. Recently, the Bayesian paradigm has shown utility in the clinical drug development process for its probability-based inference. We argue for a Bayesian approach that employs data from other trials as a “prior” for Phase 3 trials so that synthesized evidence across trials can be utilized to compute probability statements that are valuable for understanding the magnitude of treatment effect. Such a Bayesian paradigm provides a promising framework for improving statistical inference and regulatory decision making.     

High-dimensional posterior consistency of the Bayesian lasso

ABSTRACT

This paper considers posterior consistency in the context of high-dimensional variable selection using the Bayesian lasso algorithm. In a frequentist setting, consistency is perhaps the most basic property that we expect any reasonable estimator to achieve. However, in a Bayesian setting, consistency is often ignored or taken for granted, especially in more complex hierarchical Bayesian models. In this paper, we have derived sufficient conditions for posterior consistency in the Bayesian lasso model with the orthogonal design, where the number of parameters grows with the sample size.     

The Bayesian and frequentist approaches to testing a one-sided hypothesis about a multivariate mean

This paper compares the Bayesian and frequentist approaches to testing a one-sided hypothesis about a multivariate mean. First, this paper proposes a simple way to assign a Bayesian posterior probability to one-sided hypotheses about a multivariate mean. The approach is to use (almost) the exact posterior probability under the assumption that the data has multivariate normal distribution, under either a conjugate prior in large samples or under a vague Jeffreys prior. This is also approximately the Bayesian posterior probability of the hypothesis based on a suitably flat Dirichlet process prior over an unknown distribution generating the data. Then, the Bayesian approach and a frequentist approach to testing the one-sided hypothesis are compared, with results that show a major difference between Bayesian reasoning and frequentist reasoning. The Bayesian posterior probability can be substantially smaller than the frequentist p-value. A class of example is given where the Bayesian posterior probability is basically 0, while the frequentist p-value is basically 1. The Bayesian posterior probability in these examples seems to be more reasonable. Other drawbacks of the frequentist p-value as a measure of whether the one-sided hypothesis is true are also discussed.     

Efficacy and toxicity monitoring via Bayesian predictive probabilities in phase II clinical trials

Bayesian monitoring strategies based on predictive probabilities are widely used in phase II clinical trials that involve a single efficacy binary variable. The essential idea is to control the predictive probability that the trial will show a conclusive result at the scheduled end of the study, given the information at the interim stage and the prior beliefs. In this paper, we present an extension of this approach to incorporate toxicity considerations in single-arm phase II trials. We consider two binary endpoints representing response and toxicity of the experimental treatment and define the result as successful at the conclusion of the study if the posterior probability of an high efficacy and that of a small toxicity are both sufficiently large. At any interim look, the Multinomial-Dirichlet distribution provides the predictive probability of each possible combination of future efficacy and toxicity outcomes. It is exploited to obtain the predictive probability that the trial will yield a positive outcome, if it continues to the planned end. Different possible interim situations are considered to investigate the behaviour of the proposed predictive rules and the differences with the monitoring strategies based on posterior probabilities are highlighted. Simulation studies are also performed to evaluate the frequentist operating characteristics of the proposed design and to calibrate the design parameters.

     

A note on fiducial model averaging as an alternative to checking Bayesian and frequentist models

Just as frequentist hypothesis tests have been developed to check model assumptions, prior predictive p-values and other Bayesian p-values check prior distributions as well as other model assumptions. These model checks not only suffer from the usual threshold dependence of p-values, but also from the suppression of model uncertainty in subsequent inference. One solution is to transform Bayesian and frequentist p-values for model assessment into a fiducial distribution across the models. Averaging the Bayesian or frequentist posterior distributions with respect to the fiducial distribution can reproduce results from Bayesian model averaging or classical fiducial inference.     

A Hybrid Geometric Phase II/III Clinical Trial Design based on Treatment Failure Time and Toxicity

The problem of comparing several experimental treatments to a standard arises frequently in medical research. Various multi-stage randomized phase II/III designs have been proposed that select one or more promising experimental treatments and compare them to the standard while controlling overall Type I and Type II error rates. This paper addresses phase II/III settings where the joint goals are to increase the average time to treatment failure and control the probability of toxicity while accounting for patient heterogeneity. We are motivated by the desire to construct a feasible design for a trial of four chemotherapy combinations for treating a family of rare pediatric brain tumors. We present a hybrid two-stage design based on two-dimensional treatment effect parameters. A targeted parameter set is constructed from elicited parameter pairs considered to be equally desirable. Bayesian regression models for failure time and the probability of toxicity as functions of treatment and prognostic covariates are used to define two-dimensional covariate-adjusted treatment effect parameter sets. Decisions at each stage of the trial are based on the ratio of posterior probabilities of the alternative and null covariate-adjusted parameter sets. Design parameters are chosen to minimize expected sample size subject to frequentist error constraints. The design is illustrated by application to the brain tumor trial.     

A generalization of Jeffreys' rule for non regular models

ABSTRACT

We propose a generalization of the one-dimensional Jeffreys' rule in order to obtain non informative prior distributions for non regular models, taking into account the comments made by Jeffreys in his article of 1946. These non informatives are parameterization invariant and the Bayesian intervals have good behavior in frequentist inference. In some important cases, we can generate non informative distributions for multi-parameter models with non regular parameters. In non regular models, the Bayesian method offers a satisfactory solution to the inference problem and also avoids the problem that the maximum likelihood estimator has with these models. Finally, we obtain non informative distributions in job-search and deterministic frontier production homogenous models.     

Decision‐making in a phase II clinical trial: a new approach combining Bayesian and frequentist concepts

The aim of a phase II clinical trial is to decide whether or not to develop an experimental therapy further through phase III clinical evaluation. In this paper, we present a Bayesian approach to the phase II trial, although we assume that subsequent phase III clinical trials will have standard frequentist analyses. The decision whether to conduct the phase III trial is based on the posterior predictive probability of a significant result being obtained. This fusion of Bayesian and frequentist techniques accepts the current paradigm for expressing objective evidence of therapeutic value, while optimizing the form of the phase II investigation that leads to it. By using prior information, we can assess whether a phase II study is needed at all, and how much or what sort of evidence is required. The proposed approach is illustrated by the design of a phase II clinical trial of a multi‐drug resistance modulator used in combination with standard chemotherapy in the treatment of metastatic breast cancer. Copyright © 2005 John Wiley & Sons, Ltd     

Bayesian estimation and prediction of the intensity of the power law process

In analyzing failure data pertaining to a repairable system, perhaps the most widely used parametric model is a nonhomogeneous Poisson process with Weibull intensity, more commonly referred to as the Power Law Process (PLP) model. Investigations relating to inference of parameters of the PLP under a frequentist framework abound in the literature. The focus of this article is to supplement those findings from a Bayesian perspective, which has thus far been explored to a limited extent in this context. Main emphasis is on the inference of the intensity function of the PLP. Both estimation and future prediction are considered under traditional as well as more complex censoring schemes. Modern computational tools such as Markov Chain Monte Carlo are exploited efficiently to facilitate the numerical evaluation process. Results from the Bayesian inference are contrasted with the corresponding findings from a frequentist analysis, both from a qualitative and a quantitative viewpoint. The developed methodology is implemented in analyzing interval-censored failure data of equipments in a fleet of marine vessels.     

Hybrid Bayesian inference on HIV viral dynamic models

Modelling of HIV dynamics in AIDS research has greatly improved our understanding of the pathogenesis of HIV-1 infection and guided for the treatment of AIDS patients and evaluation of antiretroviral therapies. Some of the model parameters may have practical meanings with prior knowledge available, but others might not have prior knowledge. Incorporating priors can improve the statistical inference. Although there have been extensive Bayesian and frequentist estimation methods for the viral dynamic models, little work has been done on making simultaneous inference about the Bayesian and frequentist parameters. In this article, we propose a hybrid Bayesian inference approach for viral dynamic nonlinear mixed-effects models using the Bayesian frequentist hybrid theory developed in Yuan [Bayesian frequentist hybrid inference, Ann. Statist. 37 (2009), pp. 2458–2501]. Compared with frequentist inference in a real example and two simulation examples, the hybrid Bayesian approach is able to improve the inference accuracy without compromising the computational load.     

Bayesian D-optimal Accelerated Life Test plans for series systems with competing exponential causes of failure

This paper provides methods of obtaining Bayesian D-optimal Accelerated Life Test (ALT) plans for series systems with independent exponential component lives under the Type-I censoring scheme. Two different Bayesian D-optimality design criteria are considered. For both the criteria, first optimal designs for a given number of experimental points are found by solving a finite-dimensional constrained optimization problem. Next, the global optimality of such an ALT plan is ensured by applying the General Equivalence Theorem. A detailed sensitivity analysis is also carried out to investigate the effect of different planning inputs on the resulting optimal ALT plans. Furthermore, these Bayesian optimal plans are also compared with the corresponding (frequentist) locally D-optimal ALT plans.     

Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses

Testing of a composite null hypothesis versus a composite alternative is considered when both have a related invariance structure. The goal is to develop conditional frequentist tests that allow the reporting of data-dependent error probabilities, error probabilities that have a strict frequentist interpretation and that reflect the actual amount of evidence in the data. The resulting tests are also seen to be Bayesian tests, in the strong sense that the reported frequentist error probabilities are also the posterior probabilities of the hypotheses under default choices of the prior distribution. The new procedures are illustrated in a variety of applications to model selection and multivariate hypothesis testing.     

A dynamic power prior for borrowing historical data in noninferiority trials with binary endpoint

Traditionally, noninferiority hypotheses have been tested using a frequentist method with a fixed margin. Given that information for the control group is often available from previous studies, it is interesting to consider a Bayesian approach in which information is “borrowed” for the control group to improve efficiency. However, construction of an appropriate informative prior can be challenging. In this paper, we consider a hybrid Bayesian approach for testing noninferiority hypotheses in studies with a binary endpoint. To account for heterogeneity between the historical information and the current trial for the control group, a dynamic P value–based power prior parameter is proposed to adjust the amount of information borrowed from the historical data. This approach extends the simple test‐then‐pool method to allow a continuous discounting power parameter. An adjusted α level is also proposed to better control the type I error. Simulations are conducted to investigate the performance of the proposed method and to make comparisons with other methods including test‐then‐pool and hierarchical modeling. The methods are illustrated with data from vaccine clinical trials.     

A comparison of methods forsing loss-based capability index

The process capability index C _pm, sometimes called the loss-based index, has been proposed to the manufacturing industry for measuring process reproduction capability. This index incorporates the variation of production items with respect to the target value and the specification limits preset in the factory. To estimate the loss-based index properly and accurately, certain frequentist and Bayesian perspectives have been proposed to obtain lower confidence bounds (LCBs) for providing minimum process capability. The LCBs not only provide critical information regarding process performance but are also used to determine whether an improvement was made in a capability index and by extension in reducing the fraction of non-conforming items. In this paper, under the assumption of normality, based on frequentist and Bayesian senses, several existing approaches for constructing LCBs of C _pm are presented. Depending on the statistical methods used, we then classify these existing approaches into three categories and compared them in terms of the coverage rates and the mean values of the LCBs via simulations. The relative advantages and disadvantages of these approaches are summarized with some highlights of the relevant findings.     

Frequentist and Bayesian Interval Estimators for the Normal Mean

It is well known that a Bayesian credible interval for a parameter of interest is derived from a prior distribution that appropriately describes the prior information. However, it is less well known that there exists a frequentist approach developed by Pratt (1961 Pratt , J. W. ( 1961 ). Length of confidence intervals . J. Amer. Statist. Assoc. 56 : 549 – 657 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) that also utilizes prior information in the construction of frequentist confidence intervals. This frequentist approach produces confidence intervals that have minimum weighted average expected length, averaged according to some weight function that appropriately describes the prior information. We begin with a simple model as a starting point in comparing these two distinct procedures in interval estimation. Consider X ₁,…, X _n that are independent and identically N(μ, σ²) distributed random variables, where σ² is known, and the parameter of interest is μ. Suppose also that previous experience with similar data sets and/or specific background and expert opinion suggest that μ = 0. Our aim is to: (a) develop two types of Bayesian 1 ? α credible intervals for μ, derived from an appropriate prior cumulative distribution function F(μ) more importantly; (b) compare these Bayesian 1 ? α credible intervals for μ to the frequentist 1 ? α confidence interval for μ derived from Pratt's frequentist approach, in which the weight function corresponds to the prior cumulative distribution function F(μ). We show that the endpoints of the Bayesian 1 ? α credible intervals for μ are very different to the endpoints of the frequentist 1 ? α confidence interval for μ, when the prior information strongly suggests that μ = 0 and the data supports the uncertain prior information about μ. In addition, we assess the performance of these intervals by analyzing their coverage probability properties and expected lengths.     

Frequentist-Bayesian Monte Carlo testing

Abstract

Conventional methods for statistical hypothesis testing has historically been categorized as frequentist or Bayesian. But, a third option based on a reconciling hybrid frequentist-Bayesian framework is quickly emerging. Although prominent, there are applications where the exact hybrid test is not computable. For such cases, the present paper introduces a straightforward Monte Carlo procedure for performing frequentist-Bayesian testing.