A Note on Window Length Selection in Singular Spectrum Analysis

In singular spectrum analysis (SSA) window length is a critical tuning parameter that must be assigned by the practitioner. This paper provides a theoretical analysis of signal–noise separation and time series reconstruction in SSA that can serve as a guide to optimal window choice. We establish numerical bounds on the mean squared reconstruction error and present their almost sure limits under very general regularity conditions on the underlying data generating mechanism. We also provide asymptotic bounds for the mean squared separation error. Evidence obtained using simulation experiments and real data sets indicates that the theoretical properties are reflected in observed behaviour, even in relatively small samples, and the results indicate how, in practice, an optimal assignment for the window length can be made.     

On multivariate nonlinear regression models with stationary correlated errors

In this paper we consider the statistical analysis of multivariate multiple nonlinear regression models with correlated errors, using Finite Fourier Transforms. Consistency and asymptotic normality of the weighted least squares estimates are established under various conditions on the regressor variables. These conditions involve different types of scalings, and the scaling factors are obtained explicitly for various types of nonlinear regression models including an interesting model which requires the estimation of unknown frequencies. The estimation of frequencies is a classical problem occurring in many areas like signal processing, environmental time series, astronomy and other areas of physical sciences. We illustrate our methodology using two real data sets taken from geophysics and environmental sciences. The data we consider from geophysics are polar motion (which is now widely known as “Chandlers Wobble”), where one has to estimate the drift parameters, the offset parameters and the two periodicities associated with elliptical motion. The data were first analyzed by Arato, Kolmogorov and Sinai who treat it as a bivariate time series satisfying a finite order time series model. They estimate the periodicities using the coefficients of the fitted models. Our analysis shows that the two dominant frequencies are 12 h and 410 days. The second example, we consider is the minimum/maximum monthly temperatures observed at the Antarctic Peninsula (Faraday/Vernadsky station). It is now widely believed that over the past 50 years there is a steady warming in this region, and if this is true, the warming has serious consequences on ecology, marine life, etc. as it can result in melting of ice shelves and glaciers. Our objective here is to estimate any existing temperature trend in the data, and we use the nonlinear regression methodology developed here to achieve that goal.     

Sparse additive models

Summary.  We present a new class of methods for high dimensional non-parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non-parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin. They are also closely related to the COSSO model of Lin and Zhang but decouple smoothing and sparsity, enabling the use of arbitrary non-parametric smoothers. We give an analysis of the theoretical properties of sparse additive models and present empirical results on synthetic and real data, showing that they can be effective in fitting sparse non-parametric models in high dimensional data.     

Optimal regression designs in the presence of random block effects

A- and D-optimal regression designs under random block-effects models are considered. We first identify certain situations where D- and A-optimal designs do not depend on the intra-block correlation and can be obtained easily from the optimal designs under uncorrelated models. For example, for quadratic regression on [−1,1], this covers D-optimal designs when the block size is a multiple of 3 and A-optimal designs when the block size is a multiple of 4. In general, the optimal designs depend on the intra-block correlation. For quadratic regression, we provide expressions for D-optimal designs for any block size. A-optimal designs with blocks of size 2 for quadratic regression are also obtained. In all the cases considered, robust designs which do not depend on the intrablock correlation can be constructed.     

Empirical likelihood confidence intervals for nonparametric functional data analysis

We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilk's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression models involving functional data. Our numerical results demonstrate improved performance of the empirical likelihood methods over normal approximation-based methods.     

Bayesian covariance estimation and inference in latent Gaussian process models

This paper describes inference methods for functional data under the assumption that the functional data of interest are smooth latent functions, characterized by a Gaussian process, which have been observed with noise over a finite set of time points. The methods we propose are completely specified in a Bayesian environment that allows for all inferences to be performed through a simple Gibbs sampler. Our main focus is in estimating and describing uncertainty in the covariance function. However, these models also encompass functional data estimation, functional regression where the predictors are latent functions, and an automatic approach to smoothing parameter selection. Furthermore, these models require minimal assumptions on the data structure as the time points for observations do not need to be equally spaced, the number and placement of observations are allowed to vary among functions, and special treatment is not required when the number of functional observations is less than the dimensionality of those observations. We illustrate the effectiveness of these models in estimating latent functional data, capturing variation in the functional covariance estimate, and in selecting appropriate smoothing parameters in both a simulation study and a regression analysis of medfly fertility data.     

Nonparametric Inference for Time-Varying Coefficient Quantile Regression

The article considers nonparametric inference for quantile regression models with time-varying coefficients. The errors and covariates of the regression are assumed to belong to a general class of locally stationary processes and are allowed to be cross-dependent. Simultaneous confidence tubes (SCTs) and integrated squared difference tests (ISDTs) are proposed for simultaneous nonparametric inference of the latter models with asymptotically correct coverage probabilities and Type I error rates. Our methodologies are shown to possess certain asymptotically optimal properties. Furthermore, we propose an information criterion that performs consistent model selection for nonparametric quantile regression models of nonstationary time series. For implementation, a wild bootstrap procedure is proposed, which is shown to be robust to the dependent and nonstationary data structure. Our method is applied to studying the asymmetric and time-varying dynamic structures of the U.S. unemployment rate since the 1940s. Supplementary materials for this article are available online.     

Semiparametric model average prediction in panel data analysis

Forecasting in economic data analysis is dominated by linear prediction methods where the predicted values are calculated from a fitted linear regression model. With multiple predictor variables, multivariate nonparametric models were proposed in the literature. However, empirical studies indicate the prediction performance of multi-dimensional nonparametric models may be unsatisfactory. We propose a new semiparametric model average prediction (SMAP) approach to analyse panel data and investigate its prediction performance with numerical examples. Estimation of individual covariate effect only requires univariate smoothing and thus may be more stable than previous multivariate smoothing approaches. The estimation of optimal weight parameters incorporates the longitudinal correlation and the asymptotic properties of the estimated results are carefully studied in this paper.     

E-optimal designs for regression models with quantitative factors— a reasonable choice?

For regression models with quantitative factors it is illustrated that the E-optimal design can be extremely inefficient in the sense that it degenerates to a design which takes all observations at only one point. This phenomenon is caused by the different size of the elements in the covariance matrix of the least-squares estimator for the unknown parameters. For these reasons we propose to replace the E-criterion by a corresponding standardized version. The advantage of this approach is demonstrated for the polynomial regression on a nonnegative interval, where the classical and standardized E-optimal designs can be found explicitly. The described phenomena are not restricted to the E-criterion but appear for nearly all optimality criteria proposed in the literature. Therefore standardization is recommended for optimal experimental design in regression models with quantitative factors. The optimal designs with respect to the new standardized criteria satisfy a similar invariance property as the famous D-optimal designs, which allows an easy calculation of standardized optimal designs on many linearly transformed design spaces.     

Multivariate Real-Time Signal Extraction by a Robust Adaptive Regression Filter

We propose a new regression-based filter for extracting signals online from multivariate high frequency time series. It separates relevant signals of several variables from noise and (multivariate) outliers.

Unlike parallel univariate filters, the new procedure takes into account the local covariance structure between the single time series components. It is based on high-breakdown estimates, which makes it robust against (patches of) outliers in one or several of the components as well as against outliers with respect to the multivariate covariance structure. Moreover, the trade-off problem between bias and variance for the optimal choice of the window width is approached by choosing the size of the window adaptively, depending on the current data situation.

Furthermore, we present an advanced algorithm of our filtering procedure that includes the replacement of missing observations in real time. Thus, the new procedure can be applied in online-monitoring practice. Applications to physiological time series from intensive care show the practical effect of the proposed filtering technique.     

Out-of-Sample Forecast Tests Robust to the Choice of Window Size

This article proposes new methodologies for evaluating economic models’ out-of-sample forecasting performance that are robust to the choice of the estimation window size. The methodologies involve evaluating the predictive ability of forecasting models over a wide range of window sizes. The study shows that the tests proposed in the literature may lack the power to detect predictive ability and might be subject to data snooping across different window sizes if used repeatedly. An empirical application shows the usefulness of the methodologies for evaluating exchange rate models’ forecasting ability.     

Undercoverage of Wavelet-Based Resampling Confidence Intervals

The decorrelating property of the discrete wavelet transformation (DWT) appears valuable because one can avoid estimating the correlation structure in the original data space by bootstrap resampling of the DWT. Several authors have shown that the wavestrap approximately retains the correlation structure of observations. However, simply retaining the same correlation structure of original observations does not guarantee enough variation for regression parameter estimators. Our simulation studies show that these wavestraps yield undercoverage of parameters for a simple linear regression for time series data of the type that arise in functional MRI experiments. It is disappointing that the wavestrap does not even provide valid resamples for both white noise sequences and fractional Brownian noise sequences. Thus, the wavestrap method is not completely valid in obtaining resamples related to linear regression analysis and should be used with caution for hypothesis testing as well. The reasons for these undercoverages are also discussed. A parametric bootstrap resampling in the wavelet domain is introduced to offer insight into these previously undiscovered defects in wavestrapping.     

Robust estimators for additive models using backfitting

Additive models provide an attractive setup to estimate regression functions in a nonparametric context. They provide a flexible and interpretable model, where each regression function depends only on a single explanatory variable and can be estimated at an optimal univariate rate. Most estimation procedures for these models are highly sensitive to the presence of even a small proportion of outliers in the data. In this paper, we show that a relatively simple robust version of the backfitting algorithm (consisting of using robust local polynomial smoothers) corresponds to the solution of a well-defined optimisation problem. This formulation allows us to find mild conditions to show Fisher consistency and to study the convergence of the algorithm. Our numerical experiments show that the resulting estimators have good robustness and efficiency properties. We illustrate the use of these estimators on a real data set where the robust fit reveals the presence of influential outliers.     

Mixtures of regression models with incomplete and noisy data

The estimation of the mixtures of regression models is usually based on the normal assumption of components and maximum likelihood estimation of the normal components is sensitive to noise, outliers, or high-leverage points. Missing values are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this article, we propose the mixtures of regression models for contaminated incomplete heterogeneous data. The proposed models provide robust estimates of regression coefficients varying across latent subgroups even under the presence of missing values. The methodology is illustrated through simulation studies and a real data analysis.     

State transition modeling of complex monitored health data

This article considers the analysis of complex monitored health data, where often one or several signals are reflecting the current health status that can be represented by a finite number of states, in addition to a set of covariates. In particular, we consider a novel application of a non-parametric state intensity regression method in order to study time-dependent effects of covariates on the state transition intensities. The method can handle baseline, time varying as well as dynamic covariates. Because of the non-parametric nature, the method can handle different data types and challenges under minimal assumptions. If the signal that is reflecting the current health status is of continuous nature, we propose the application of a weighted median and a hysteresis filter as data pre-processing steps in order to facilitate robust analysis. In intensity regression, covariates can be aggregated by a suitable functional form over a time history window. We propose to study the estimated cumulative regression parameters for different choices of the time history window in order to investigate short- and long-term effects of the given covariates. The proposed framework is discussed and applied to resuscitation data of newborns collected in Tanzania.     

Nonparametric Methods in Continuous Time Model Specification

Some popular parametric diffusion processes have been assumed as such underlying diffusion processes. This paper considers an important case where both the drift and volatility functions of the underlying diffusion process are unknown functions of the underlying process, and then proposes using two novel testing procedures for the parametric specification of both the drift and diffusion functions. The finite-sample properties of the proposed tests are assessed through using data generated from four popular parametric models. In our implementation, we suggest using a simulated critical value for each case in addition to the use of an asymptotic critical value. Our detailed studies show that there is little size distortion when using a simulated critical value while the proposed tests have some size distortions when using an asymptotic critical value in each case.     

Nonparametric Methods in Continuous Time Model Specification

Some popular parametric diffusion processes have been assumed as such underlying diffusion processes. This paper considers an important case where both the drift and volatility functions of the underlying diffusion process are unknown functions of the underlying process, and then proposes using two novel testing procedures for the parametric specification of both the drift and diffusion functions. The finite-sample properties of the proposed tests are assessed through using data generated from four popular parametric models. In our implementation, we suggest using a simulated critical value for each case in addition to the use of an asymptotic critical value. Our detailed studies show that there is little size distortion when using a simulated critical value while the proposed tests have some size distortions when using an asymptotic critical value in each case.     

Recurrent first hitting times in Wiener diffusion under several observation schemes

Recurrent events are commonly encountered in the natural sciences, engineering, and medicine. The theory of renewal and regenerative processes provides an elegant mathematical foundation for idealized recurrent event processes. In real-world applications, however, the contexts tend to be complicated by a variety of practical intricacies, including observation schemes with different phase and data structures. This paper formulates a recurrent event process as a succession of independent and identically distributed first hitting times for a Wiener sample path as it passes through successive equally-spaced levels. We develop exact mathematical results for statistical inferences based on several observation schemes that include observation initiated at a renewal point, observation of a stationary process over a finite window, and other variants. We also consider inferences drawn from different data structures, including gap times between renewal points (or fragments thereof) and counts of renewal events occurring within an observation window. We explore the precision of estimates using simulated scenarios and develop empirical regression functions for planning the sample size of a recurrent event study. We demonstrate our results using data from a clinical trial for chronic obstructive pulmonary disease in which the recurrent events are successive exacerbations of the condition. The case study demonstrates how covariates can be incorporated into the analysis using threshold regression.     

Estimation for the censored partially linear quantile regression models

In this article, we develop estimation procedures for partially linear quantile regression models, where some of the responses are censored by another random variable. The nonparametric function is estimated by basis function approximations. The estimation procedure is easy to implement through existing weighted quantile regression, and it requires no specification of the error distributions. We show the large-sample properties of the resulting estimates, the proposed estimator of the regression parameter is root-n consistent and asymptotically normal and the estimator of the functional component achieves the optimal convergence rate of the nonparametric function. The proposed method is studied via simulations and illustrated with the analysis of a primary biliary cirrhosis (BPC) data.