Feature significance in generalized additive models

This paper develops inference for the significance of features such as peaks and valleys observed in additive modeling through an extension of the SiZer-type methodology of Chaudhuri and Marron (1999) and Godtliebsen et al. (2002, 2004) to the case where the outcome is discrete. We consider the problem of determining the significance of features such as peaks or valleys in observed covariate effects both for the case of additive modeling where the main predictor of interest is univariate as well as the problem of studying the significance of features such as peaks, inclines, ridges and valleys when the main predictor of interest is geographical location. We work with low rank radial spline smoothers to allow to the handling of sparse designs and large sample sizes. Reducing the problem to a Generalised Linear Mixed Model (GLMM) framework enables derivation of simulation-based critical value approximations and guards against the problem of multiple inferences over a range of predictor values. Such a reduction also allows for easy adjustment for confounders including those which have an unknown or complex effect on the outcome. A simulation study indicates that our method has satisfactory power. Finally, we illustrate our methodology on several data sets.     

Estimating generalized semiparametric additive models using parameter cascading

Elimination of nuisance parameters is a central but difficult problem in statistical inference. We propose the parameter cascading method to estimate statistical models that involve nuisance parameters, structural parameters, and complexity parameters. The parameter cascading method has several unique aspects. First, we consider functional relationships between parameters, quantitatively described using analytical derivatives. These functional relationships can be explicit or implicit, and in the latter case the Implicit Function Theorem is applied to obtain the required derivatives. Second, we can express the gradients and Hessian matrices analytically, which is essential for fast and stable computation. Third, we develop the unconditional variance estimates for parameters, which include the uncertainty coming from other parameter estimates. The parameter cascading method is demonstrated by estimating generalized semiparametric additive models (GSAMs), where the response variable is allowed to be from any distribution. The practical necessity and empirical performance of the parameter cascading method are illustrated through a simulation study, and two applied example, one on finding the effect of air pollution on public health, and the other on the management of a retirement fund. The results demonstrate that the parameter cascading method is a good alternative to traditional methods.     

Conditional variance estimation via nonparametric generalized additive models

Generalized additive models provide a way of circumventing curse of dimension in a wide range of nonparametric regression problem. In this paper, we present a multiplicative model for conditional variance functions where one can apply a generalized additive regression method. This approach extends Fan and Yao (1998) to multivariate cases with a multiplicative structure. In this approach, we use squared residuals instead of using log-transformed squared residuals. This idea gives a smaller variance than Yu (2017) when the variance of squared error is smaller than the variance of log-transformed squared error. We provide estimators based on quasi-likelihood and an iterative algorithm based on smooth backfitting for generalized additive models. We also provide some asymptotic properties of estimators and the convergence of proposed algorithm. A numerical study shows the empirical evidence of the theory.     

Derivative estimation and testing in generalized additive models

Estimation and testing procedures for generalized additive (interaction) models are developed. We present extensions of several existing procedures for additive models when the link is the identity. This set of methods includes estimation of all component functions and their derivatives, testing functional forms and in particular variable selection. Theorems and simulation results are presented for the fundamentally new procedures. These comprise of, in particular, the introduction of local polynomial smoothing for this kind of models and the testing, including variable selection. Our method is straightforward to implement and the simulation studies show good performance in even small data sets.     

On Markov-switching periodic ARMA models

In this work, we propose a generalization of the classical Markov-switching ARMA models to the periodic time-varying case. Specifically, we propose a Markov-switching periodic ARMA (MS-PARMA) model. In addition of capturing regime switching often encountered during the study of many economic time series, this new model also captures the periodicity feature in the autocorrelation structure. We first provide some probabilistic properties of this class of models, namely the strict periodic stationarity and the existence of higher-order moments. We thus propose a procedure for computing the autocovariance function where we show that the autocovariances of the MS-PARMA model satisfy a system of equations similar to the PARMA Yule–Walker equations. We propose also an easily implemented algorithm which can be used to obtain parameter estimates for the MS-PARMA model. Finally, a simulation study of the performance of the proposed estimation method is provided.     

Efficient semiparametric estimation in generalized partially linear additive models

In this paper we study semiparametric generalized additive models in which some part of the additive function is linear. We study the semiparametric efficiency under this regression model for the exponential family. We also present an asymptotically efficient estimation procedure based on the generalized profile likelihood approach.     

Application of generalized additive models to butterfly transect count data

We investigate the use of generalized additive models for describing patterns in butterfly transect counts during the flight period. Models were applied to sets of simulated data and to transect counts from the British Butterfly Monitoring Scheme (BMS) recorded at a large number of sites in the UK. The models successfully described patterns in counts in a range of species with different life cycles and the approach can be used to estimate an index of butterfly abundance allowing for missing counts. The method could be extended to include other factors such as temperature, sunshine, windspeed and time of day, and to examine potential biases arising from variation in these factors.     

A generalized ARFIMA process with Markov-switching fractional differencing parameter

We propose a general class of Markov-switching-ARFIMA (MS-ARFIMA) processes in order to combine strands of long memory and Markov-switching literature. Although the coverage of this class of models is broad, we show that these models can be easily estimated with the Durbin–Levinson–Viterbi algorithm proposed. This algorithm combines the Durbin–Levinson and Viterbi procedures. A Monte Carlo experiment reveals that the finite sample performance of the proposed algorithm for a simple mixture model of Markov-switching mean and ARFIMA(1, d, 1) process is satisfactory. We apply the MS-ARFIMA models to the US real interest rates and the Nile river level data, respectively. The results are all highly consistent with the conjectures made or empirical results found in the literature. Particularly, we confirm the conjecture in Beran and Terrin [J. Beran and N. Terrin, Testing for a change of the long-memory parameter. Biometrika 83 (1996), pp. 627–638.] that the observations 1 to about 100 of the Nile river data seem to be more independent than the subsequent observations, and the value of differencing parameter is lower for the first 100 observations than for the subsequent data.     

A comparison of methods for the fitting of generalized additive models

There are several procedures for fitting generalized additive models, i.e. regression models for an exponential family response where the influence of each single covariates is assumed to have unknown, potentially non-linear shape. Simulated data are used to compare a smoothing parameter optimization approach for selection of smoothness and of covariates, a stepwise approach, a mixed model approach, and a procedure based on boosting techniques. In particular it is investigated how the performance of procedures is linked to amount of information, type of response, total number of covariates, number of influential covariates, and extent of non-linearity. Measures for comparison are prediction performance, identification of influential covariates, and smoothness of fitted functions. One result is that the mixed model approach returns sparse fits with frequently over-smoothed functions, while the functions are less smooth for the boosting approach and variable selection is less strict. The other approaches are in between with respect to these measures. The boosting procedure is seen to perform very well when little information is available and/or when a large number of covariates is to be investigated. It is somewhat surprising that in scenarios with low information the fitting of a linear model, even with stepwise variable selection, has not much advantage over the fitting of an additive model when the true underlying structure is linear. In cases with more information the prediction performance of all procedures is very similar. So, in difficult data situations the boosting approach can be recommended, in others the procedures can be chosen conditional on the aim of the analysis.     

Bayesian inference for generalized additive mixed models based on Markov random field priors

Most regression problems in practice require flexible semiparametric forms of the predictor for modelling the dependence of responses on covariates. Moreover, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal or spatial data. We present a unified approach for Bayesian inference via Markov chain Monte Carlo simulation in generalized additive and semiparametric mixed models. Different types of covariates, such as the usual covariates with fixed effects, metrical covariates with non-linear effects, unstructured random effects, trend and seasonal components in longitudinal data and spatial covariates, are all treated within the same general framework by assigning appropriate Markov random field priors with different forms and degrees of smoothness. We applied the approach in several case-studies and consulting cases, showing that the methods are also computationally feasible in problems with many covariates and large data sets. In this paper, we choose two typical applications.     

Sparse additive models

Summary.  We present a new class of methods for high dimensional non-parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non-parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin. They are also closely related to the COSSO model of Lin and Zhang but decouple smoothing and sparsity, enabling the use of arbitrary non-parametric smoothers. We give an analysis of the theoretical properties of sparse additive models and present empirical results on synthetic and real data, showing that they can be effective in fitting sparse non-parametric models in high dimensional data.     

Bootstrap hypothesis testing in generalized additive models for comparing curves of treatments in longitudinal studies

The study of the effect of a treatment may involve the evaluation of a variable at a number of moments. When assuming a smooth curve for the mean response along time, estimation can be afforded by spline regression, in the context of generalized additive models. The novelty of our work lies in the construction of hypothesis tests to compare two curves of treatments in any interval of time for several types of response variables. The within-subject correlation is not modeled but is considered to obtain valid inferences by the use of bootstrap. We propose both semiparametric and nonparametric bootstrap approaches, based on resampling vectors of residuals or responses, respectively. Simulation studies revealed a good performance of the tests, considering, for the outcome, different distribution functions in the exponential family and varying the correlation between observations along time. We show that the sizes of bootstrap tests are close to the nominal value, with tests based on a standardized statistic having slightly better size properties. The power increases as the distance between curves increases and decreases when correlation gets higher. The usefulness of these statistical tools was confirmed using real data, thus allowing to detect changes in fish behavior when exposed to the toxin microcystin-RR.     

Generalized additive mixed models

Following the extension from linear mixed models to additive mixed models, extension from generalized linear mixed models to generalized additive mixed models is made, Algorithms are developed to compute the MLE's of the nonlinear effects and the covariance structures based on the penalized marginal likelihood. Convergence of the algorithms and selection of the smooth param¬eters are discussed.     

Fourier smoother and additive models

Suppose the observations (t_i,y_i), i = 1,… n, follow the model where g_j are unknown functions. The estimation of the additive components can be done by approximating g_j, with a function made up of the sum of a linear fit and a truncated Fourier series of cosines and minimizing a penalized least-squares loss function over the coefficients. This finite-dimensional basis approximation, when fitting an additive model with r predictors, has the advantage of reducing the computations drastically, since it does not require the use of the backfitting algorithm. The cross-validation (CV) [or generalized cross-validation (GCV)] for the additive fit is calculated in a further 0(n) operations. A search path in the r-dimensional space of degrees of freedom is proposed along which the CV (GCV) continuously decreases. The path ends when an increase in the degrees of freedom of any of the predictors yields an increase in CV (GCV). This procedure is illustrated on a meteorological data set.     

Interquantile shrinkage in additive models

In this paper, we investigate the commonality of nonparametric component functions among different quantile levels in additive regression models. We propose two fused adaptive group Least Absolute Shrinkage and Selection Operator penalties to shrink the difference of functions between neighbouring quantile levels. The proposed methodology is able to simultaneously estimate the nonparametric functions and identify the quantile regions where functions are unvarying, and thus is expected to perform better than standard additive quantile regression when there exists a region of quantile levels on which the functions are unvarying. Under some regularity conditions, the proposed penalised estimators can theoretically achieve the optimal rate of convergence and identify the true varying/unvarying regions consistently. Simulation studies and a real data application show that the proposed methods yield good numerical results.     

Isotone additive latent variable models

For manifest variables with additive noise and for a given number of latent variables with an assumed distribution, we propose to nonparametrically estimate the association between latent and manifest variables. Our estimation is a two step procedure: first it employs standard factor analysis to estimate the latent variables as theoretical quantiles of the assumed distribution; second, it employs the additive models’ backfitting procedure to estimate the monotone nonlinear associations between latent and manifest variables. The estimated fit may suggest a different latent distribution or point to nonlinear associations. We show on simulated data how, based on mean squared errors, the nonparametric estimation improves on factor analysis. We then employ the new estimator on real data to illustrate its use for exploratory data analysis.     

Transformed generalized linear models

The estimation of data transformation is very useful to yield response variables satisfying closely a normal linear model. Generalized linear models enable the fitting of models to a wide range of data types. These models are based on exponential dispersion models. We propose a new class of transformed generalized linear models to extend the Box and Cox models and the generalized linear models. We use the generalized linear model framework to fit these models and discuss maximum likelihood estimation and inference. We give a simple formula to estimate the parameter that index the transformation of the response variable for a subclass of models. We also give a simple formula to estimate the r$r$th moment of the original dependent variable. We explore the possibility of using these models to time series data to extend the generalized autoregressive moving average models discussed by Benjamin et al. [Generalized autoregressive moving average models. J. Amer. Statist. Assoc. 98, 214–223]. The usefulness of these models is illustrated in a simulation study and in applications to three real data sets.     

Estimating additive models with missing responses

ABSTRACT

For multivariate regressors, the Nadaraya–Watson regression estimator suffers from the well-known curse of dimensionality. Additive models overcome this drawback. To estimate the additive components, it is usually assumed that we observe all the data. However, in many applied statistical analysis missing data occur. In this paper, we study the effect of missing responses on the additive components estimation. The estimators are based on marginal integration adapted to the missing situation. The proposed estimators turn out to be consistent under mild assumptions. A simulation study allows to compare the behavior of our procedures, under different scenarios.     

Semiparametric mixture of additive regression models

In this article, we propose a semiparametric mixture of additive regression models, in which the regression functions are additive and non parametric while the mixing proportions and variances are constant. Compared with the mixture of linear regression models, the proposed methodology is more flexible in modeling the non linear relationship between the response and covariate. A two-step procedure based on the spline-backfitted kernel method is derived for computation. Moreover, we establish the asymptotic normality of the resultant estimators and examine their good performance through a numerical example.     

Generalized additive models for longitudinal data

We introduce a class of models for longitudinal data by extending the generalized estimating equations approach of Liang and Zeger (1986) to incorporate the flexibility of nonparametric smoothing. The algorithm provides a unified estimation procedure for marginal distributions from the exponential family. We propose pointwise standard-error bands and approximate likelihood-ratio and score tests for inference. The algorithm is formally derived by using the penalized quasilikelihood framework. Convergence of the estimating equations and consistency of the resulting solutions are discussed. We illustrate the algorithm with data on the population dynamics of Colorado potato beetles on potato plants.