Boosted coefficient models

Regression methods typically construct a mapping from the covariates into the real numbers. Here, however, we consider regression problems where the task is to form a mapping from the covariates into a set of (univariate) real-valued functions. Examples are given by conditional density estimation, hazard regression and regression with a functional response. Our approach starts by modeling the function of interest using a sum of B-spline basis functions. To model dependence on the covariates, the coefficients of this expansion are each modeled as functions of the covariates. We propose to estimate these coefficient functions using boosted tree models. Algorithms are provided for the above three situations, and real data sets are used to investigate their performance. The results indicate that the proposed methodology performs well. In addition, it is both straightforward, and capable of handling a large number of covariates.