Smoothing a Time Series by Segments of the Data Range

We consider the problem of estimating a trend with different amounts of smoothness for segments of a time series subjected to different variability regimes. We propose using an unobserved components model to consider the existence of at least two data segments. We first fix some desired percentages of smoothness for the trend segments and deduce the corresponding smoothing parameters involved. Once the size of each segment is chosen, the smoothing formulas here derived produce trend estimates for all segments with the desired smoothness as well as their corresponding estimated variances. Empirical examples from demography and economics illustrate our proposal.     

Theory & Methods: Krige,Smooth, Both or Neither?

Both kriging and non-parametric regression smoothing can model a non-stationary regression function with spatially correlated errors. However comparisons have mainly been based on ordinary kriging and smoothing with uncorrelated errors. Ordinary kriging attributes smoothness of the response to spatial autocorrelation whereas non-parametric regression attributes trends to a smooth regression function. For spatial processes it is reasonable to suppose that the response is due to both trend and autocorrelation. This paper reviews methodology for non-parametric regression with autocorrelated errors which is a natural compromise between the two methods. Re-analysis of the one-dimensional stationary spatial data of Laslett (1994) and a clearly non-stationary time series demonstrates the rather surprising result that for these data, ordinary kriging outperforms more computationally intensive models including both universal kriging and correlated splines for spatial prediction. For estimating the regression function, non-parametric regression provides adaptive estimation, but the autocorrelation must be accounted for in selecting the smoothing parameter.     

A model-averaging approach for smoothing spline regression

ABSTRACT

This article considers nonparametric regression problems and develops a model-averaging procedure for smoothing spline regression problems. Unlike most smoothing parameter selection studies determining an optimum smoothing parameter, our focus here is on the prediction accuracy for the true conditional mean of Y given a predictor X. Our method consists of two steps. The first step is to construct a class of smoothing spline regression models based on nonparametric bootstrap samples, each with an appropriate smoothing parameter. The second step is to average bootstrap smoothing spline estimates of different smoothness to form a final improved estimate. To minimize the prediction error, we estimate the model weights using a delete-one-out cross-validation procedure. A simulation study has been performed by using a program written in R. The simulation study provides a comparison of the most well known cross-validation (CV), generalized cross-validation (GCV), and the proposed method. This new method is straightforward to implement, and gives reliable performances in simulations.     

Trend estimation of multivariate time series with controlled smoothness

This paper extends the univariate time series smoothing approach provided by penalized least squares to a multivariate setting, thus allowing for joint estimation of several time series trends. The theoretical results are valid for the general multivariate case, but particular emphasis is placed on the bivariate situation from an applied point of view. The proposal is based on a vector signal-plus-noise representation of the observed data that requires the first two sample moments and specifying only one smoothing constant. A measure of the amount of smoothness of an estimated trend is introduced so that an analyst can set in advance a desired percentage of smoothness to be achieved by the trend estimate. The required smoothing constant is determined by the chosen percentage of smoothness. Closed form expressions for the smoothed estimated vector and its variance-covariance matrix are derived from a straightforward application of generalized least squares, thus providing best linear unbiased estimates for the trends. A detailed algorithm applicable for estimating bivariate time series trends is also presented and justified. The theoretical results are supported by a simulation study and two real applications. One corresponds to Mexican and US macroeconomic data within the context of business cycle analysis, and the other one to environmental data pertaining to a monitored site in Scotland.     

A non-iterative optimization method for smoothness in penalized spline regression

Typically, an optimal smoothing parameter in a penalized spline regression is determined by minimizing an information criterion, such as one of the C _p , CV and GCV criteria. Since an explicit solution to the minimization problem for an information criterion cannot be obtained, it is necessary to carry out an iterative procedure to search for the optimal smoothing parameter. In order to avoid such extra calculation, a non-iterative optimization method for smoothness in penalized spline regression is proposed using the formulation of generalized ridge regression. By conducting numerical simulations, we verify that our method has better performance than other methods which optimize the number of basis functions and the single smoothing parameter by means of the CV or GCV criteria.     

ON CONFIDENCE INTERVALS FOR GENERALIZED ADDITIVE MODELS BASED ON PENALIZED REGRESSION SPLINES

Generalized additive models represented using low rank penalized regression splines, estimated by penalized likelihood maximisation and with smoothness selected by generalized cross validation or similar criteria, provide a computationally efficient general framework for practical smooth modelling. Various authors have proposed approximate Bayesian interval estimates for such models, based on extensions of the work of Wahba, G. (1983) [Bayesian confidence intervals for the cross validated smoothing spline. J. R. Statist. Soc. B 45 , 133–150] and Silverman, B.W. (1985) [Some aspects of the spline smoothing approach to nonparametric regression curve fitting. J. R. Statist. Soc. B 47 , 1–52] on smoothing spline models of Gaussian data, but testing of such intervals has been rather limited and there is little supporting theory for the approximations used in the generalized case. This paper aims to improve this situation by providing simulation tests and obtaining asymptotic results supporting the approximations employed for the generalized case. The simulation results suggest that while across‐the‐model performance is good, component‐wise coverage probabilities are not as reliable. Since this is likely to result from the neglect of smoothing parameter variability, a simple and efficient simulation method is proposed to account for smoothing parameter uncertainty: this is demonstrated to substantially improve the performance of component‐wise intervals.     

Penalized Spline Estimation for Varying-Coefficient Models

Varying-coefficient models are useful extensions of classical linear models. They arise from multivariate nonparametric regression, nonlinear time series modeling and forecasting, longitudinal data analysis, and others. This article proposes the penalized spline estimation for the varying-coefficient models. Assuming a fixed but potentially large number of knots, the penalized spline estimators are shown to be strong consistency and asymptotic normality. A systematic optimization algorithm for the selection of multiple smoothing parameters is developed. One of the advantages of the penalized spline estimation is that it can accommodate varying degrees of smoothness among coefficient functions due to multiple smoothing parameters being used. Some simulation studies are presented to illustrate the proposed methods.     

Smoothing spline growth curves with covariates

We adapt the interactive spline model of Wahba. to growth curves o with covariates. The smoothing spline formulation permits a nonpara-metric representation of the growth curves. In the limit when the discretization error is small relative to the estimation error, the resulting growth curve estimates often depend only weakly on the number and locations of the knots. The smoothness parameter is determined from the data by minimizing an empirical estimate of the expected error. We show that the risk estimate of Craven and Wahba is a weighted goodness of fit estimate, A modified loss estimate is given, where a2 is replaced by its unbiased estimate.     

Penalized least squares smoothing of two-dimensional mortality tables with imposed smoothness

This paper presents a method to estimate mortality trends of two-dimensional mortality tables. Comparability of mortality trends for two or more of such tables is enhanced by applying penalized least squares and imposing a desired percentage of smoothness to be attained by the trends. The smoothing procedure is basically determined by the smoothing parameters that are related to the percentage of smoothness. To quantify smoothness, we employ an index defined first for the one-dimensional case and then generalized to the two-dimensional one. The proposed method is applied to data from member countries of the OECD. We establish as goal the smoothed mortality surface for one of those countries and compare it with some other mortality surfaces smoothed with the same percentage of two-dimensional smoothness. Our aim is to be able to see whether convergence exists in the mortality trends of the countries under study, in both year and age dimensions.     

Nonparametric estimation of discrete hazard functions

A smoothing procedure for discrete time failure data is proposed which allows for the inclusion of covariates. This purely nonparametric method is based on discrete or continuous kernel smoothing techniques that gives a compromise between the data and smoothness. The method may be used as an exploratory tool to uncover the underlying structure or as an alternative to parametric methods when prediction is the primary objective. Confidence intervals are considered and alternative techniques of cross validation based choices of smoothing parameters are investigated.     

Wavelet estimation in time-varying coefficient time series models with measurement errors

The article studies a time-varying coefficient time series model in which some of the covariates are measured with additive errors. In order to overcome the bias of estimator of the coefficient functions when measurement errors are ignored, we propose a modified least squares estimator based on wavelet procedures. The advantage of the wavelet method is to avoid the restrictive smoothness requirement for varying-coefficient functions of the traditional smoothing approaches, such as kernel and local polynomial methods. The asymptotic properties of the proposed wavelet estimators are established under the α-mixing conditions and without specifying the error distribution. These results can be used to make asymptotically valid statistical inference.     

Interval estimation for the breakpoint in segmented regression: a smoothed score‐based approach

This paper is concerned with interval estimation for the breakpoint parameter in segmented regression. We present score‐type confidence intervals derived from the score statistic itself and from the recently proposed gradient statistic. Due to lack of regularity conditions of the score, non‐smoothness and non‐monotonicity, naive application of the score‐based statistics is unfeasible and we propose to exploit the smoothed score obtained via induced smoothing. We compare our proposals with the traditional methods based on the Wald and the likelihood ratio statistics via simulations and an analysis of a real dataset: results show that the smoothed score‐like statistics perform in practice somewhat better than competitors, even when the model is not correctly specified.     

LOCAL LINEAR FORECASTS USING CUBIC SMOOTHING SPLINES

This paper shows how cubic smoothing splines fitted to univariate time series data can be used to obtain local linear forecasts. The approach is based on a stochastic state‐space model which allows the use of likelihoods for estimating the smoothing parameter, and which enables easy construction of prediction intervals. The paper shows that the model is a special case of an ARIMA(0, 2, 2) model; it provides a simple upper bound for the smoothing parameter to ensure an invertible model; and it demonstrates that the spline model is not a special case of Holt's local linear trend method. The paper compares the spline forecasts with Holt's forecasts and those obtained from the full ARIMA(0, 2, 2) model, showing that the restricted parameter space does not impair forecast performance. The advantage of this approach over a full ARIMA(0, 2, 2) model is that it gives a smooth trend estimate as well as a linear forecast function.     

Spline smoothing over difficult regions

Summary. It is occasionally necessary to smooth data over domains in R ² with complex irregular boundaries or interior holes. Traditional methods of smoothing which rely on the Euclidean metric or which measure smoothness over the entire real plane may then be inappropriate. This paper introduces a bivariate spline smoothing function defined as the minimizer of a penalized sum-of-squares functional. The roughness penalty is based on a partial differential operator and is integrated only over the problem domain by using finite element analysis. The method is motivated by and applied to two sample smoothing problems and is compared with the thin plate spline.     

A note on asymptotics for quantile smoothing splines

When cubic smoothing splines are used to estimate the conditional quantile function, thereby balancing fidelity to the data with a smoothness requirement, the resulting curve is the solution to a quadratic program. Using this quadratic characterization and through comparison with the sample conditional quan-tiles, we show strong consistency and asymptotic normality for the quantile smoothing spline.     

Adaptive exponentielle glättung erster ordnung

In the paper we introduce an adaptive procedure of the first order exponential smoothing. In this procedure we get a sequence of estimation converging in mean square to the unknown smoothing parameter and an asymptotically optimum prediction in the sense of the least square error. In case of breaks in the structure of time series we recommend a modification of the procedure.     

Parametric Inference in Stationary Time Series Models with Dependent Errors

This article is concerned with inference for the parameter vector in stationary time series models based on the frequency domain maximum likelihood estimator. The traditional method consistently estimates the asymptotic covariance matrix of the parameter estimator and usually assumes the independence of the innovation process. For dependent innovations, the asymptotic covariance matrix of the estimator depends on the fourth‐order cumulants of the unobserved innovation process, a consistent estimation of which is a difficult task. In this article, we propose a novel self‐normalization‐based approach to constructing a confidence region for the parameter vector in such models. The proposed procedure involves no smoothing parameter, and is widely applicable to a large class of long/short memory time series models with weakly dependent innovations. In simulation studies, we demonstrate favourable finite sample performance of our method in comparison with the traditional method and a residual block bootstrap approach.     

Bayesian inference for semiparametric regression using a Fourier representation

This paper presents the Bayesian analysis of a semiparametric regression model that consists of parametric and nonparametric components. The nonparametric component is represented with a Fourier series where the Fourier coefficients are assumed a priori to have zero means and to decay to 0 in probability at either algebraic or geometric rates. The rate of decay controls the smoothness of the response function. The posterior analysis automatically selects the amount of smoothing that is coherent with the model and data. Posterior probabilities of the parametric and semiparametric models provide a method for testing the parametric model against a non-specific alternative. The Bayes estimator's mean integrated squared error compares favourably with the theoretically optimal estimator for kernel regression.     

Adaptive modelling of dose–response relationships using smoothing splines

We consider the use of smoothing splines for the adaptive modelling of dose–response relationships. A smoothing spline is a nonparametric estimator of a function that is a compromise between the fit to the data and the degree of smoothness and thus provides a flexible way of modelling dose–response data. In conjunction with decision rules for which doses to continue with after an interim analysis, it can be used to give an adaptive way of modelling the relationship between dose and response. We fit smoothing splines using the generalized cross‐validation criterion for deciding on the degree of smoothness and we use estimated bootstrap percentiles of the predicted values for each dose to decide upon which doses to continue with after an interim analysis. We compare this approach with a corresponding adaptive analysis of variance approach based upon new simulations of the scenarios previously used by the PhRMA Working Group on Adaptive Dose‐Ranging Studies. The results obtained for the adaptive modelling of dose–response data using smoothing splines are mostly comparable with those previously obtained by the PhRMA Working Group for the Bayesian Normal Dynamic Linear model (GADA) procedure. These methods may be useful for carrying out adaptations, detecting dose–response relationships and identifying clinically relevant doses. Copyright © 2009 John Wiley & Sons, Ltd.     

Global optimization of the generalized cross-validation criterion

Generalized cross-validation is a method for choosing the smoothing parameter in smoothing splines and related regularization problems. This method requires the global minimization of the generalized cross-validation function. In this paper an algorithm based on interval analysis is presented to find the globally optimal value for the smoothing parameter, and a numerical example illustrates the performance of the algorithm.