Smoothing spline Gaussian regression: more scalable computation via efficient approximation

Summary.  Smoothing splines via the penalized least squares method provide versatile and effective nonparametric models for regression with Gaussian responses. The computation of smoothing splines is generally of the order O ( n ³), n being the sample size, which severely limits its practical applicability. We study more scalable computation of smoothing spline regression via certain low dimensional approximations that are asymptotically as efficient. A simple algorithm is presented and the Bayes model that is associated with the approximations is derived, with the latter guiding the porting of Bayesian confidence intervals. The practical choice of the dimension of the approximating space is determined through simulation studies, and empirical comparisons of the approximations with the exact solution are presented. Also evaluated is a simple modification of the generalized cross-validation method for smoothing parameter selection, which to a large extent fixes the occasional undersmoothing problem that is suffered by generalized cross-validation.     

Inference in smoothing spline analysis of variance

Summary. Smoothing spline analysis of variance decomposes a multivariate function into additive components. This decomposition not only provides an efficient way to model a multivariate function but also leads to meaningful inference by testing whether a certain component equals 0. No formal procedure is yet available to test such a hypothesis. We propose an asymptotic method based on the likelihood ratio to test whether a functional component is 0. This test allows us to choose an optimal model and to compare groups of curves. We first develop the general theory by exploiting the connection between mixed effects models and smoothing splines. We then apply this to compare two groups of curves and to select an optimal model in a two-dimensional problem. A small simulation is used to assess the finite sample performance of the likelihood ratio test.     

Smoothing Time Series with Local Polynomial Regression on Time

Time series smoothers estimate the level of a time series at time t as its conditional expectation given present, past and future observations, with the smoothed value depending on the estimated time series model. Alternatively, local polynomial regressions on time can be used to estimate the level, with the implied smoothed value depending on the weight function and the bandwidth in the local linear least squares fit. In this article we compare the two smoothing approaches and describe their similarities. Through simulations, we assess the increase in the mean square error that results when approximating the estimated optimal time series smoother with the local regression estimate of the level.     

Weighted empirical adaptive variance estimators for correlated data regression

Estimating equations based on marginal generalized linear models are useful for regression modelling of correlated data, but inference and testing require reliable estimates of standard errors. We introduce a class of variance estimators based on the weighted empirical variance of the estimating functions and show that an adaptive choice of weights allows reliable estimation both asymptotically and by simulation in finite samples. Connections with previous bootstrap and jackknife methods are explored. The effect of reliable variance estimation is illustrated in data on health effects of air pollution in King County, Washington.     

Trend smoothness achieved by penalized least squares with the smoothing parameter chosen by optimality criteria

This work presents a study about the smoothness attained by the methods more frequently used to choose the smoothing parameter in the context of splines: Cross Validation, Generalized Cross Validation, and corrected Akaike and Bayesian Information Criteria, implemented with Penalized Least Squares. It is concluded that the amount of smoothness strongly depends on the length of the series and on the type of underlying trend, while the presence of seasonality even though statistically significant is less relevant. The intrinsic variability of the series is not statistically significant and its effect is taken into account only through the smoothing parameter.     

Estimation and inference for varying coefficient partially nonlinear models

In this paper, we extend the varying coefficient partially linear model to the varying coefficient partially nonlinear model in which the linear part of the varying coefficient partially linear model is replaced by a nonlinear function of the covariates. A profile nonlinear least squares estimation procedure for the parameter vector and the coefficient function vector of the varying coefficient partially nonlinear model is proposed and the asymptotic properties of the resulting estimators are established. We further propose a generalized likelihood ratio (GLR) test to check whether or not the varying coefficients in the model are constant. The asymptotic null distribution of the GLR statistic is derived and a residual-based bootstrap procedure is also suggested to derive the p-value of the GLR test. Some simulations are conducted to assess the performance of the proposed estimating and testing procedures and the results show that both the procedures perform well in finite samples. Furthermore, a real data example is given to demonstrate the application of the proposed model and its estimating and testing procedures.     

Modelling functional additive quantile regression using support vector machines approach

This work deals with conditional quantiles estimation when several functional covariates are involved, via a support vector machines nonparametric methodology. We establish weak consistency of this estimator. To fit the additive components, we use an ordinary backfitting procedure combined with an iterative reweighted least-squares procedure to solve the penalised minimisation problem. This procedure makes it possible to derive a split sample method for choosing the hyper-parameters of the model. The performances of the proposed technique, in terms of forecast accuracy, are evaluated through simulation and a real dataset study.     

Estimation of variance components in the mixed effects models: A comparison between analysis of variance and spectral decomposition

The mixed effects models with two variance components are often used to analyze longitudinal data. For these models, we compare two approaches to estimating the variance components, the analysis of variance approach and the spectral decomposition approach. We establish a necessary and sufficient condition for the two approaches to yield identical estimates, and some sufficient conditions for the superiority of one approach over the other, under the mean squared error criterion. Applications of the methods to circular models and longitudinal data are discussed. Furthermore, simulation results indicate that better estimates of variance components do not necessarily imply higher power of the tests or shorter confidence intervals.