A Unified View of Nonparametric Trend-Cycle Predictors Via Reproducing Kernel Hilbert Spaces

We provide a common approach for studying several nonparametric estimators used for smoothing functional time series data. Linear filters based on different building assumptions are transformed into kernel functions via reproducing kernel Hilbert spaces. For each estimator, we identify a density function or second order kernel, from which a hierarchy of higher order estimators is derived. These are shown to give excellent representations for the currently applied symmetric filters. In particular, we derive equivalent kernels of smoothing splines in Sobolev and polynomial spaces. The asymmetric weights are obtained by adapting the kernel functions to the length of the various filters, and a theoretical and empirical comparison is made with the classical estimators used in real time analysis. The former are shown to be superior in terms of signal passing, noise suppression and speed of convergence to the symmetric filter.     

Diagnostic measures for kernel ridge regression on reproducing kernel Hilbert space

The aim of this paper is to define and develop diagnostic measures with respect to kernel ridge regression in a reproducing kernel Hilbert space (RKHS). To identify influential observations, we define a particular version of Cook’s distance for the kernel ridge regression model in RKHS, which is conceptually consistent with Cook’s distance in a classical regression model. Then, by using the perturbation formula for the regularized conditional expectation of the outcome in RKHS, we develop an approximate version of Cook”s distance in RKHS because the original definition requires intensive computations. Such an approximated Cook”s distance is represented in terms of basic building blocks such as residuals and leverages of the kernel ridge regression. The results of the simulation and real application demonstrate that our diagnostic measure successfully detects potentially influential observations on estimators in kernel ridge regression.     

On testing for multivariate ARCH effects in vector time series models

Using a spectral approach, the authors propose tests to detect multivariate ARCH effects in the residuals from a multivariate regression model. The tests are based on a comparison, via a quadratic norm, between the uniform density and a kernel‐based spectral density estimator of the squared residuals and cross products of residuals. The proposed tests are consistent under an arbitrary fixed alternative. The authors present a new application of the test due to Hosking (1980) which is seen to be a special case of their approach involving the truncated uniform kernel. However, they typically obtain more powerful procedures when using a different weighting. The authors consider especially the procedure of Robinson (1991) for choosing the smoothing parameter of the spectral density estimator. They also introduce a generalized version of the test for ARCH effects due to Ling & Li (1997). They investigate the finite‐sample performance of their tests and compare them to existing tests including those of Ling & Li (1997) and the residual‐based diagnostics of Tse (2002).Finally, they present a financial application.     

Nonparametric maximum likelihood estimation for shifted curves

The analysis of a sample of curves can be done by self-modelling regression methods. Within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set-up. We derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model. The nonparametric maximum likelihood estimator that we find turns out to be a Nadaraya–Watson-like estimator, but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function, since the score equation automatically selects the shape and the smoothing parameter for the estimation. We apply the method to a sample of electrophoretic spectra to illustrate how it works.     

Understanding exponential smoothing via kernel regression

Exponential smoothing is the most common model-free means of forecasting a future realization of a time series. It requires the specification of a smoothing factor which is usually chosen from the data to minimize the average squared residual of previous one-step-ahead forecasts. In this paper we show that exponential smoothing can be put into a nonparametric regression framework and gain some interesting insights into its performance through this interpretation. We also use theoretical developments from the kernel regression field to derive, for the first time, asymptotic properties of exponential smoothing forecasters.     

Nonparametric estimation of the hazard function under dependence conditions

The estimation of the hazard rate has a great number of practical appli¬cations in dependence situations (seismicity analysis, reliability, economics), Based on kernel estimates of the density and the distribution function, we study the properties of the nonparametric estimator of the hazard function as-sociated with a strongly mixing time series. We prove consistency and asymp¬totic normality properties, and a cross-validation method for the smoothing parameter selection is studied. Some simulations and a practical application to real data are also shown.     

Robust non-parametric smoothing of non-stationary time series

Motivated by the need of extracting local trends and low frequency components in non-stationary time series, this paper discusses methods of robust non-parametric smoothing. Basic approach is the combination of the parametric M-estimation with kernel and local polynomial regression methods. The result is an iterative estimator that retains a linear structure, but has kernel weights also in the direction of the prediction errors. The design of smoothing coefficients is carried out with robust cross-validation criteria and rules of thumb. The method works well both to remove the influence of patches of outliers and to detect the local breaks and persistent structural change in time series.     

Global and local statistical properties of fixed-length nonparametric smoothers

The main purpose of this study is to analyze the global and local statistical properties of nonparametric smoothers subject to a priori fixed length restriction. In order to do so, we introduce a set of local statistical measures based on their weighting system shapes and weight values. In this way, the local statistical measures of bias, variance and mean square error are intrinsic to the smoothers and independent of the data to which they will be applied on. One major advantage of the statistical measures relative to the classical spectral ones is their easiness of calculation. However, in this paper we use both in a complementary manner. The smoothers studied are based on two broad classes of weighting generating functions, local polynomials and probability distributions. We consider within the first class, the locally weighted regression smoother (loess) of degree 1 and 2 (L1 and L2), the cubic smoothing spline (CSS), and the Henderson smoothing linear filter (H); and in the second class, the Gaussian kernel (GK). The weighting systems of these estimators depend on a smoothing parameter that traditionally, is estimated by means of data dependent optimization criteria. However, by imposing to all of them the condition of an equal number of weights, it will be shown that some of their optimal statistical properties are no longer valid. Without any loss of generality, the analysis is carried out for 13- and 9-term lengths because these are the most often selected for the Henderson filters in the context of monthly time series decomposition. We would like to thank an Associate Editor and an anonymous referee for their valuable comments on an earlier version of this paper. Financing from MURST is also gratefully acknowledged.     

Generalized Mahalanobis depth in the reproducing kernel Hilbert space

In this paper, Mahalanobis depth (MHD) in the Reproducing Kernel Hilbert Space (RKHS) is proposed. First, we extend the notion of MHD to a generalized version, i.e., the generalized MHD (GHMD), to make it suitable for the small sample with singular covariance matrix. We prove that GMHD is consistent with MHD when the sample has a full-rank covariance matrix. Second, we further extend GMHD to RKHS, i.e, the kernel mapped GMHD (kmGMHD), and discuss its main properties. Numeric results show that kmGMHD can give a better depth interpretation for the sample with special shape, such as a non-convex sample set. Our proposed kmGMHD can be potentially used as a robust tool for outliers detection and data classification. In addition, we also discuss the influence of parameters on the shape of the central regions.     

Optimal smoothing parameters for multivariate fized and adaptive kernel methods

Except in special cases optimum smoothing parameters of kernel methods are difficult to obtain for small samples, and large sample results are often used. Simulation is used to obtain finite sample optimum smoothing parameters and mean integrated square errors for the bivariate normal density. For this example, comparison is made of finite and asymptotic results, and of fixed and adaptive kernel methods. Further comparisons are made of fixed and adaptive methods by considering four other different types of density. Finally, some examples are given.     

A new reproducing kernel-based nonlinear dimension reduction method for survival data

Based on the theories of sliced inverse regression (SIR) and reproducing kernel Hilbert space (RKHS), a new approach RDSIR (RKHS-based Double SIR) to nonlinear dimension reduction for survival data is proposed. An isometric isomorphism is constructed based on the RKHS property, then the nonlinear function in the RKHS can be represented by the inner product of two elements that reside in the isomorphic feature space. Due to the censorship of survival data, double slicing is used to estimate the weight function to adjust for the censoring bias. The nonlinear sufficient dimension reduction (SDR) subspace is estimated by a generalized eigen-decomposition problem. The asymptotic property of the estimator is established based on the perturbation theory. Finally, the performance of RDSIR is illustrated on simulated and real data. The numerical results show that RDSIR is comparable with the linear SDR method. Most importantly, RDSIR can also effectively extract nonlinearity from survival data.     

Efficient inference for parameters of unobservable periodic autoregressive time series

This paper considers estimating the model coefficients when the observed periodic autoregressive time series is contaminated by a trend. The proposed Yule–Walker estimators are obtained by a two-step procedure. In the first step, the trend is estimated by a weighted local polynomial, and the residuals are obtained by subtracting the trend estimates from the observations; in the second step, the model coefficients are estimated by the well-known Yule–Walker method via the residuals. It is shown that under certain conditions such Yule–Walker estimators are oracally efficient, i.e., they are asymptotically equivalent to those obtained from periodic autoregressive time series without a trend. An easy-to-use implementation procedure is provided. The performance of the estimators is illustrated by simulation studies and real data analysis. In particular, the simulation studies show that the proposed estimator outperforms that obtained from the residuals when the trend is estimated by kernel smoothing without taking the heteroscedasticity into consideration.     

A Note on a Specification Test for Time Series Models Based on Spectral Density Estimation

In a recent paper, Paparoditis [Scand. J. Statist. 27 (2000) 143] proposed a new goodness‐of‐fit test for time series models based on spectral density estimation. The test statistic is based on the distance between a kernel estimator of the ratio of the true and the hypothesized spectral density and the expected value of the estimator under the null and provides a quantification of how well the parametric density fits the sample spectral density. In this paper, we give a detailed asymptotic analysis of the corresponding procedure under fixed alternatives.     

Testing Semiparametric Hypotheses in Locally Stationary Processes

In this paper, we investigate the problem of testing semiparametric hypotheses in locally stationary processes. The proposed method is based on an empirical version of the L₂‐distance between the true time varying spectral density and its best approximation under the null hypothesis. As this approach only requires estimation of integrals of the time varying spectral density and its square, we do not have to choose a smoothing bandwidth for the local estimation of the spectral density – in contrast to most other procedures discussed in the literature. Asymptotic normality of the test statistic is derived both under the null hypothesis and the alternative. We also propose a bootstrap procedure to obtain critical values in the case of small sample sizes. Additionally, we investigate the finite sample properties of the new method and compare it with the currently available procedures by means of a simulation study. Finally, we illustrate the performance of the new test in two data examples, one regarding log returns of the S&P 500 and the other a well‐known series of weekly egg prices.     

On consistent testing for serial correlation in seasonal time series models

The author considers serial correlation testing in seasonal time series models. He proposes a test statistic based on a spectral approach. Many tests of this type rely on kernel-based spectral density estimators that assign larger weights to low order lags than to high ones. Under seasonality, however, large autocorrelations may occur at seasonal lags that classical kernel estimators cannot take into account. The author thus proposes a test statistic that relies on the spectral density estimator of Shin (2004), whose weighting scheme is more adapted to this context. The distribution of his test statistic is derived under the null hypothesis and he studies its behaviour under fixed and local alternatives. He establishes the consistency of the test under a general fixed alternative. He also makes recommendations for the choice of the smoothing parameters. His simulation results suggest that his test is more powerful against seasonality than alternative procedures based on classical weighting schemes. He illustrates his procedure with monthly statistics on employment among young Americans.     

The Scale Invariant Wigner Spectrum Estimation of Gaussian Locally Self-Similar Processes

We study locally self-similar processes (LSSPs) in Silverman’s sense. By deriving the minimum mean-square optimal kernel within Cohen’s class counterpart of time–frequency representations, we obtain an optimal estimation for the scale invariant Wigner spectrum (SIWS) of Gaussian LSSPs. The class of estimators is completely characterized in terms of kernels, so the optimal kernel minimizes the mean-square error of the estimation. We obtain the SIWS estimation for two cases: global and local, where in the local case, the kernel is allowed to vary with time and frequency. We also introduce two generalizations of LSSPs: the locally self-similar chirp process and the multicomponent LSSP, and obtain their optimal kernels. Finally, the performance and accuracy of the estimation is studied via simulation.     

Using recursive algorithms for the efficient identification of smoothing spline ANOVA models

In this paper we present a unified discussion of different approaches to the identification of smoothing spline analysis of variance (ANOVA) models: (i) the “classical” approach (in the line of Wahba in Spline Models for Observational Data, 1990; Gu in Smoothing Spline ANOVA Models, 2002; Storlie et al. in Stat. Sin., 2011) and (ii) the State-Dependent Regression (SDR) approach of Young in Nonlinear Dynamics and Statistics (2001). The latter is a nonparametric approach which is very similar to smoothing splines and kernel regression methods, but based on recursive filtering and smoothing estimation (the Kalman filter combined with fixed interval smoothing). We will show that SDR can be effectively combined with the “classical” approach to obtain a more accurate and efficient estimation of smoothing spline ANOVA models to be applied for emulation purposes. We will also show that such an approach can compare favorably with kriging.     

Local Box–Cox transformation on time-varying parametric models for smoothing estimation of conditional CDF with longitudinal data

Nonparametric estimation and inferences of conditional distribution functions with longitudinal data have important applications in biomedical studies, such as epidemiological studies and longitudinal clinical trials. Estimation approaches without any structural assumptions may lead to inadequate and numerically unstable estimators in practice. We propose in this paper a nonparametric approach based on time-varying parametric models for estimating the conditional distribution functions with a longitudinal sample. Our model assumes that the conditional distribution of the outcome variable at each given time point can be approximated by a parametric model after local Box–Cox transformation. Our estimation is based on a two-step smoothing method, in which we first obtain the raw estimators of the conditional distribution functions at a set of disjoint time points, and then compute the final estimators at any time by smoothing the raw estimators. Applications of our two-step estimation method have been demonstrated through a large epidemiological study of childhood growth and blood pressure. Finite sample properties of our procedures are investigated through a simulation study. Application and simulation results show that smoothing estimation from time-variant parametric models outperforms the existing kernel smoothing estimator by producing narrower pointwise bootstrap confidence band and smaller root mean squared error.     

Two-step estimation of functional linear models with applications to longitudinal data

Functional linear models are useful in longitudinal data analysis. They include many classical and recently proposed statistical models for longitudinal data and other functional data. Recently, smoothing spline and kernel methods have been proposed for estimating their coefficient functions nonparametrically but these methods are either intensive in computation or inefficient in performance. To overcome these drawbacks, in this paper, a simple and powerful two-step alternative is proposed. In particular, the implementation of the proposed approach via local polynomial smoothing is discussed. Methods for estimating standard deviations of estimated coefficient functions are also proposed. Some asymptotic results for the local polynomial estimators are established. Two longitudinal data sets, one of which involves time-dependent covariates, are used to demonstrate the approach proposed. Simulation studies show that our two-step approach improves the kernel method proposed by Hoover and co-workers in several aspects such as accuracy, computational time and visual appeal of the estimators.     

On Fan's adaptive Neyman tests for comparing two spectral densities

In this paper, we consider tests for assessing whether two stationary and independent time series have the same spectral densities (or same autocovariance functions). Both frequency domain and time domain test statistics for this purpose are reviewed. The adaptive Neyman tests are then introduced and their performances are investigated. Our tests are adaptive, that is, they are constructed completely by the data and do not involve any unknown smoothing parameters. Simulation studies show that our proposed tests are at least comparable to the current tests in most cases. Furthermore, our tests are much more powerful in some cases, such as against the long orders of autoregressive moving average (ARMA) models such as seasonal ARMA series.