Bootstrap inference in local polynomial regression of time series

In this paper we consider the inferential aspect of the nonparametric estimation of a conditional function , where X _t,m represents the vector containing the m conditioning lagged values of the series. Here is an arbitrary measurable function. The local polynomial estimator of order p is used for the estimation of the function g, and of its partial derivatives up to a total order p. We consider α-mixing processes, and we propose the use of a particular resampling method, the local polynomial bootstrap, for the approximation of the sampling distribution of the estimator. After analyzing the consistency of the proposed method, we present a simulation study which gives evidence of its finite sample behaviour.     

Robust recursive estimation in nonlinear time series

Aase (1983) has dealt with recursive estimation in nonlinear time series of autoregressive type including its asymptotic properties. This contribution modifies the results for the case of nonlinear time series with outliers using the principle of M-estimation from robust statistics. Strong consistency of the robust recursive estimates is preserved under corresponding assumptions. Several types of such estimates are compared by means of a numerical simulation.     

A recursive approach to parameter estimation in regression and time series models

In this paper we discuss the recursive (or on line) estimation in (i) regression and (ii) autoregressive integrated moving average (ARIMA) time series models. The adopted approach uses Kalman filtering techniques to calculate estimates recursively. This approach is used for the estimation of constant as well as time varying parameters. In the first section of the paper we consider the linear regression model. We discuss recursive estimation both for constant and time varying parameters. For constant parameters, Kalman filtering specializes to recursive least squares. In general, we allow the parameters to vary according to an autoregressive integrated moving average process and update the parameter estimates recursively. Since the stochastic model for the parameter changes will "be rarely known, simplifying assumptions have to be made. In particular we assume a random walk model for the time varying parameters and show how to determine whether the parameters are changing over time. This is illustrated with an example.     

Predicting hospital census using time series regression methods

We first describe the time series modeling problem in a general way. Then some specific assumptions and observations which are pertinent to the application of these models are made. We next propose a specific approach to the modeling problem, one which yields efficient, easily calculated estimators of all parameters (under the stated assumptions). Finally, the technique is applied to the problem of modeling the census of a particular hospital.     

Nonparametric regression estimation in a null recurrent time series

We derive an asymptotic theory of nonparametric estimation for a time series regression model Z_t=f(X_t)+W_t, where {X_t} and {Z_t} are observed nonstationary processes, and {W_t} is an unobserved stationary process. The class of nonstationary processes allowed for {X_t} is a subclass of the class of null recurrent Markov chains. This subclass contains the random walk, unit root processes and nonlinear processes. The process {W_t} is assumed to be linear and stationary.     

Automatic polynomial wavelet regression

In Oh, Naveau and Lee (2001) a simple method is proposed for reducing the bias at the boundaries for wavelet thresholding regression. The idea is to model the regression function as a sum of wavelet basis functions and a low-order polynomial. The latter is expected to account for the boundary problem. Practical implementation of this method requires the choice of the order of the low-order polynomial, as well as the wavelet thresholding value. This paper proposes two automatic methods for making such choices. Finite sample performances of these two methods are evaluated via numerical experiments.     

Adaptive wavelet series estimation in separable nonparametric regression models

It is well-known that multivariate curve estimation suffers from the curse of dimensionality. However, reasonable estimators are possible, even in several dimensions, under appropriate restrictions on the complexity of the curve. In the present paper we explore how much appropriate wavelet estimators can exploit a typical restriction on the curve such as additivity. We first propose an adaptive and simultaneous estimation procedure for all additive components in additive regression models and discuss rate of convergence results and data-dependent truncation rules for wavelet series estimators. To speed up computation we then introduce a wavelet version of functional ANOVA algorithm for additive regression models and propose a regularization algorithm which guarantees an adaptive solution to the multivariate estimation problem. Some simulations indicate that wavelets methods complement nicely the existing methodology for nonparametric multivariate curve estimation.     

Using difference-based methods for inference in nonparametric regression with time series errors

Summary. We show that difference-based methods can be used to construct simple and explicit estimators of error covariance and autoregressive parameters in nonparametric regression with time series errors. When the error process is Gaussian our estimators are efficient, but they are available well beyond the Gaussian case. As an illustration of their usefulness we show that difference-based estimators can be used to produce a simplified version of time series cross-validation. This new approach produces a bandwidth selector that is equivalent, to both first and second orders, to that given by the full time series cross-validation algorithm. Other applications of difference-based methods are to variance estimation and construction of confidence bands in nonparametric regression.     

On multivariate variable-kernel density estimates for time series

Let X₁ be a strictly stationary multiple time series with values in R^d and with a common density f. Let X₁,.,.,X_n, be n consecutive observations of X₁. Let k = k_n, be a sequence of positive integers, and let H_ni be the distance from X_i to its kth nearest neighbour among X_j, j i. The multivariate variable-kernel estimate f_n, of f is defined by where K is a given density. The complete convergence of f_n, to f on compact sets is established for time series satisfying a dependence condition (referred to as the strong mixing condition in the locally transitive sense) weaker than the strong mixing condition. Appropriate choices of k are explicitly given. The results apply to autoregressive processes and bilinear time-series models.     

Local orthogonal polynomial expansion for density estimation

A local orthogonal polynomial expansion (LOrPE) of the empirical density function is proposed as a novel method to estimate the underlying density. The estimate is constructed by matching localised expectation values of orthogonal polynomials to the values observed in the sample. LOrPE is related to several existing methods, and generalises straightforwardly to multivariate settings. By manner of construction, it is similar to local likelihood density estimation (LLDE). In the limit of small bandwidths, LOrPE functions as kernel density estimation (KDE) with high-order (effective) kernels inherently free of boundary bias, a natural consequence of kernel reshaping to accommodate endpoints. Consistency and faster asymptotic convergence rates follow. In the limit of large bandwidths LOrPE is equivalent to orthogonal series density estimation (OSDE) with Legendre polynomials, thereby inheriting its consistency. We compare the performance of LOrPE to KDE, LLDE, and OSDE, in a number of simulation studies. In terms of mean integrated squared error, the results suggest that with a proper balance of the two tuning parameters, bandwidth and degree, LOrPE generally outperforms these competitors when estimating densities with sharply truncated supports.     

Testing for changes in autocovariances of nonparametric time series models

In the literature on change-point analysis, much attention has been paid to detecting changes in certain marginal characteristics, such as mean, variance, and marginal distribution. For time series data with nonparametric time trend, we study the change-point problem for the autocovariance structure of the unobservable error process. To derive the asymptotic distribution of the cumulative sum test statistic, we develop substantial theory for uniform convergence of weighted partial sums and weighted quadratic forms. Our asymptotic results improve upon existing works in several important aspects. The performance of the test statistic is examined through simulations and an application to interest rates data.     

Improved maximum-likelihood estimation in a regression model with general parametrization

We analyse the finite-sample behaviour of two second-order bias-corrected alternatives to the maximum-likelihood estimator of the parameters in a multivariate normal regression model with general parametrization proposed by Patriota and Lemonte [A.G. Patriota and A.J. Lemonte, Bias correction in a multivariate regression model with genereal parameterization, Stat. Prob. Lett. 79 (2009), pp. 1655–1662]. The two finite-sample corrections we consider are the conventional second-order bias-corrected estimator and the bootstrap bias correction. We present the numerical results comparing the performance of these estimators. Our results reveal that analytical bias correction outperforms numerical bias corrections obtained from bootstrapping schemes.     

Likelihood estimation for longitudinal zero-inflated power series regression models

In this paper, a zero-inflated power series regression model for longitudinal count data with excess zeros is presented. We demonstrate how to calculate the likelihood for such data when it is assumed that the increment in the cumulative total follows a discrete distribution with a location parameter that depends on a linear function of explanatory variables. Simulation studies indicate that this method can provide improvements in obtaining standard errors of the estimates. We also calculate the dispersion index for this model. The influence of a small perturbation of the dispersion index of the zero-inflated model on likelihood displacement is also studied. The zero-inflated negative binomial regression model is illustrated on data regarding joint damage in psoriatic arthritis.     

On variable bandwidth selection in local polynomial regression

The performances of data-driven bandwidth selection procedures in local polynomial regression are investigated by using asymptotic methods and simulation. The bandwidth selection procedures considered are based on minimizing 'prelimit' approximations to the (conditional) mean-squared error (MSE) when the MSE is considered as a function of the bandwidth h . We first consider approximations to the MSE that are based on Taylor expansions around h=0 of the bias part of the MSE. These approximations lead to estimators of the MSE that are accurate only for small bandwidths h . We also consider a bias estimator which instead of using small h approximations to bias naïvely estimates bias as the difference of two local polynomial estimators of different order and we show that this estimator performs well only for moderate to large h . We next define a hybrid bias estimator which equals the Taylor-expansion-based estimator for small h and the difference estimator for moderate to large h . We find that the MSE estimator based on this hybrid bias estimator leads to a bandwidth selection procedure with good asymptotic and, for our Monte Carlo examples, finite sample properties.     

A simple bootstrap bandwidth selector for local polynomial fitting

A new, fully data-driven bandwidth selector with a double smoothing (DS) bias term and a data-driven variance estimator is developed following the bootstrap idea. The data-driven variance estimation does not involve any additional bandwidth selection. The proposed bandwidth selector convergences faster than a plug-in one due to the DS bias estimate, whereas the data-driven variance improves its finite sample performance clearly and makes it stable. Asymptotic results of the proposals are obtained. A comparative simulation study was done to show the overall gains and the gains obtained by improving either the bias term or the variance estimate, respectively. It is shown that the use of a good variance estimator is more important when the sample size is relatively small.     

Robust non-parametric smoothing of non-stationary time series

Motivated by the need of extracting local trends and low frequency components in non-stationary time series, this paper discusses methods of robust non-parametric smoothing. Basic approach is the combination of the parametric M-estimation with kernel and local polynomial regression methods. The result is an iterative estimator that retains a linear structure, but has kernel weights also in the direction of the prediction errors. The design of smoothing coefficients is carried out with robust cross-validation criteria and rules of thumb. The method works well both to remove the influence of patches of outliers and to detect the local breaks and persistent structural change in time series.     

Fourier series methods in nonparametric estimation

Two different approaches to the design of optimal observations networks are compared. One approach is based on the traditional experimental design theory, the other essentially uses the covariance analysis methodology of observed fields, It is found that for random fields generated by regression models with random parameters both approaches lead to similar solutions     

Improved pretest nonparametric estimation in a multivariate regression model

Shrinkage pretest nonparametric estimation of the location parameter vector in a multivariate regression model is considered when nonsample information (NSI) about the regression parameters is available. By using the quadratic risk criterion, the dominance of the pretest estimators over the usual estimators has been investigated. We demonstrate analytically and computationally that the proposed improved pretest estimator establishes a wider dominance range for the parameter under consideration than that of the usual pretest estimator in which it is superior over the unrestricted estimator.     

Nonlinearity tests in time series analysis

Summary It is widely recognized that the class of ARIMA models may fail to capture fully the dynamics of real phenomena since these are often characterized by strong nonlinear components. Thus, it is important that any preliminary analysis (or evaluation of model adequacy) includes a check on the linearity of the generating process. The paper reviews recent developments in the theory of testing nonlinearity in time series analysis.     

Assessing influence in time series

This paper studies influential observations on the spectrum of a stationary stochastic process. We introduce a leave-one-out procedure in spectral density estimation to identify influential points. A simulated envelope is proposed to assess the magnitude of influence when the data follow an autoregressive integrated moving average model. Practical illustrations are discussed in two examples.