Modeling with Mixtures of Linear Regressions

Consider data (x ₁,y ₁),...,(x _n,y _n), where each x _i may be vector valued, and the distribution of y _i given x _i is a mixture of linear regressions. This provides a generalization of mixture models which do not include covariates in the mixture formulation. This mixture of linear regressions formulation has appeared in the computer science literature under the name Hierarchical Mixtures of Experts model.This model has been considered from both frequentist and Bayesian viewpoints. We focus on the Bayesian formulation. Previously, estimation of the mixture of linear regression model has been done through straightforward Gibbs sampling with latent variables. This paper contributes to this field in three major areas. First, we provide a theoretical underpinning to the Bayesian implementation by demonstrating consistency of the posterior distribution. This demonstration is done by extending results in Barron, Schervish and Wasserman (Annals of Statistics 27: 536–561, 1999) on bracketing entropy to the regression setting. Second, we demonstrate through examples that straightforward Gibbs sampling may fail to effectively explore the posterior distribution and provide alternative algorithms that are more accurate. Third, we demonstrate the usefulness of the mixture of linear regressions framework in Bayesian robust regression. The methods described in the paper are applied to two examples.     

Evaluating fit in functional data analysis using model embeddings

The author proposes a general method for evaluating the fit of a model for functional data. His approach consists of embedding the proposed model into a larger family of models, assuming the true process generating the data is within the larger family, and then computing a posterior distribution for the Kullback‐Leibler distance between the true and the proposed models. The technique is illustrated on biomechanical data reported by Ramsay, Flanagan & Wang (1995). It is developed in detail for hierarchical polynomial models such as those found in Lindley & Smith (1972), and is also generally applicable to longitudinal data analysis where polynomials are fit to many individuals.     

Comparison of response adaptive randomization features in multiarm clinical trials with control

We investigate multiple features of response adaptive randomization (RAR) in the context of a multiple arm randomized trial with control, where the primary goal is the identification of the best arm for use in a broader patient population. We maintain constant control allocation and vary the length of time until RAR is started, interim frequency, the underlying quantity used to calculate the randomization probabilities, and a threshold resulting in temporary arm dropping. We evaluate the designs on five metrics measuring benefit to the internal trial population, the future external population, and statistical estimation. Our results indicate these features have minimal interaction within the space explored, with preference for earlier activation of RAR, more frequent interim analyses, randomizing in proportion to the probability each arm is the best, and aggressive thresholding for temporarily dropping arms. The results illustrate useful principles for maximizing the benefit of RAR in practice.     

Smooth estimates of normal mixtures

Posterior distributions for mixture models often have multiple modes, particularly near the boundaries of the parameter space where the component variances are small. This multimodality results in predictive densities that are extremely rough. The authors propose an adjustment of the standard normal‐inverse‐gamma prior structure that directly controls the ratio of the largest component variance to the smallest component variance. The prior adjustment smooths out modes near the boundary of the parameter space, producing more reasonable estimates of the predictive density.