Modelling and Prediction of Financial Time Series

We consider statistical aspects of the modelling and prediction theory of time series in one and many dimensions. We discuss Lévy-based and general models, and the stationary and non-stationary cases. Our starting point is the recent pair of surveys, Szeg'ó's theorem and its probabilistic descendants and Multivariate prediction and matrix Szeg'ó theory, by this author.     

A Bayesian kriged Kalman model for short-term forecasting of air pollution levels

Summary.  Short-term forecasts of air pollution levels in big cities are now reported in news-papers and other media outlets. Studies indicate that even short-term exposure to high levels of an air pollutant called atmospheric particulate matter can lead to long-term health effects. Data are typically observed at fixed monitoring stations throughout a study region of interest at different time points. Statistical spatiotemporal models are appropriate for modelling these data. We consider short-term forecasting of these spatiotemporal processes by using a Bayesian kriged Kalman filtering model. The spatial prediction surface of the model is built by using the well-known method of kriging for optimum spatial prediction and the temporal effects are analysed by using the models underlying the Kalman filtering method. The full Bayesian model is implemented by using Markov chain Monte Carlo techniques which enable us to obtain the optimal Bayesian forecasts in time and space. A new cross-validation method based on the Mahalanobis distance between the forecasts and observed data is also developed to assess the forecasting performance of the model implemented.     

Conditional Functional Principal Components Analysis

Abstract.  This work proposes an extension of the functional principal components analysis (FPCA) or Karhunen–Loève expansion, which can take into account non-parametrically the effects of an additional covariate. Such models can also be interpreted as non-parametric mixed effect models for functional data. We propose estimators based on kernel smoothers and a data-driven selection procedure of the smoothing parameters based on a two-step cross-validation criterion. The conditional FPCA is illustrated with the analysis of a data set consisting of egg laying curves for female fruit flies. Convergence rates are given for estimators of the conditional mean function and the conditional covariance operator when the entire curves are collected. Almost sure convergence is also proven when one observes discretized noisy sample paths only. A simulation study allows us to check the good behaviour of the estimators.     

Estimation in conditional first order autoregression with discrete support

We consider estimation in the class of first order conditional linear autoregressive models with discrete support that are routinely used to model time series of counts. Various groups of estimators proposed in the literature are discussed: moment-based estimators; regression-based estimators; and likelihood-based estimators. Some of these have been used previously and others not. In particular, we address the performance of new types of generalized method of moments estimators and propose an exact maximum likelihood procedure valid for a Poisson marginal model using backcasting. The small sample properties of all estimators are comprehensively analyzed using simulation. Three situations are considered using data generated with: a fixed autoregressive parameter and equidispersed Poisson innovations; negative binomial innovations; and, additionally, a random autoregressive coefficient. The first set of experiments indicates that bias correction methods, not hitherto used in this context to our knowledge, are some-times needed and that likelihood-based estimators, as might be expected, perform well. The second two scenarios are representative of overdispersion. Methods designed specifically for the Poisson context now perform uniformly badly, but simple, bias-corrected, Yule-Walker and least squares estimators perform well in all cases.     

Recurrent events analysis in the presence of time-dependent covariates and dependent censoring

Summary.  Recurrent events models have had considerable attention recently. The majority of approaches show the consistency of parameter estimates under the assumption that censoring is independent of the recurrent events process of interest conditional on the covariates that are included in the model. We provide an overview of available recurrent events analysis methods and present an inverse probability of censoring weighted estimator for the regression parameters in the Andersen–Gill model that is commonly used for recurrent event analysis. This estimator remains consistent under informative censoring if the censoring mechanism is estimated consistently, and it generally improves on the naïve estimator for the Andersen–Gill model in the case of independent censoring. We illustrate the bias of ad hoc estimators in the presence of informative censoring with a simulation study and provide a data analysis of recurrent lung exacerbations in cystic fibrosis patients when some patients are lost to follow-up.     

Optimal designs for minimising covariances among parameter estimators in a linear model

We construct approximate optimal designs for minimising absolute covariances between least‐squares estimators of the parameters (or linear functions of the parameters) of a linear model, thereby rendering relevant parameter estimators approximately uncorrelated with each other. In particular, we consider first the case of the covariance between two linear combinations. We also consider the case of two such covariances. For this we first set up a compound optimisation problem which we transform to one of maximising two functions of the design weights simultaneously. The approaches are formulated for a general regression model and are explored through some examples including one practical problem arising in chemistry.     

Amplitude modulated model for analyzing non-stationary speech signals

Recently amplitude modulated (AM) model in presence of additive white noise was used to analyze certain non-stationary speech data. It is observed that the assumption of white noise may not be proper in many cases. In this article, we consider the AM signal model in presence of stationary noise. We consider the least squares estimators and the estimators obtained by maximizing the Periodogram function. The two estimators are asymptotically equivalent. We study the theoretical properties of both estimators and observe their performances through numerical simulations. One speech data is analyzed and it is observed that the performance of the proposed estimators is quite satisfactory.     

Multichannel electroencephalographic analyses via dynamic regression models with time-varying lag–lead structure

Multiple time series of scalp electrical potential activity are generated routinely in electroencephalographic (EEG) studies. Such recordings provide important non-invasive data about brain function in human neuropsychiatric disorders. Analyses of EEG traces aim to isolate characteristics of their spatiotemporal dynamics that may be useful in diagnosis, or may improve the understanding of the underlying neurophysiology or may improve treatment through identifying predictors and indicators of clinical outcomes. We discuss the development and application of non-stationary time series models for multiple EEG series generated from individual subjects in a clinical neuropsychiatric setting. The subjects are depressed patients experiencing generalized tonic–clonic seizures elicited by electroconvulsive therapy (ECT) as antidepressant treatment. Two varieties of models—dynamic latent factor models and dynamic regression models—are introduced and studied. We discuss model motivation and form, and aspects of statistical analysis including parameter identifiability, posterior inference and implementation of these models via Markov chain Monte Carlo techniques. In an application to the analysis of a typical set of 19 EEG series recorded during an ECT seizure at different locations over a patient's scalp, these models reveal time-varying features across the series that are strongly related to the placement of the electrodes. We illustrate various model outputs, the exploration of such time-varying spatial structure and its relevance in the ECT study, and in basic EEG research in general.     

Fast and robust bootstrap

In this paper we review recent developments on a bootstrap method for robust estimators which is computationally faster and more resistant to outliers than the classical bootstrap. This fast and robust bootstrap method is, under reasonable regularity conditions, asymptotically consistent. We describe the method in general and then consider its application to perform inference based on robust estimators for the linear regression and multivariate location-scatter models. In particular, we study confidence and prediction intervals and tests of hypotheses for linear regression models, inference for location-scatter parameters and principal components, and classification error estimation for discriminant analysis.     

Exact rational expectations,cointegration, and reduced rank regression

We interpret the linear relations from exact rational expectations models as restrictions on the parameters of the statistical model called the cointegrated vector autoregressive model for non-stationary variables. We then show how reduced rank regression [Anderson, T.W., 1951. Estimating linear restrictions on regression coefficients for multivariate normal distributions. Ann. Math. Statist. 22, 327–351] plays an important role in the calculation of maximum likelihood estimators of the restricted parameters.     

Prediction Error Estimation Under Bregman Divergence for Non-Parametric Regression and Classification

Abstract.  Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross-validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data-driven optimal bandwidth selector for local-likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean-prediction-error-criterion. We also propose a lower-bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying-coefficient models and additive models.     

Least squares estimation of regression parameters in mixed effects models with unmeasured covariates

We consider mixed effects models for longitudinal, repeated measures or clustered data. Unmeasured or omitted covariates in such models may be correlated with the included covanates, and create model violations when not taken into account. Previous research and experience with longitudinal data sets suggest a general form of model which should be considered when omitted covariates are likely, such as in observational studies. We derive the marginal model between the response variable and included covariates, and consider model fitting using the ordinary and weighted least squares methods, which require simple non-iterative computation and no assumptions on the distribution of random covariates or error terms, Asymptotic properties of the least squares estimators are also discussed. The results shed light on the structure of least squares estimators in mixed effects models, and provide large sample procedures for statistical inference and prediction based on the marginal model. We present an example of the relationship between fluid intake and output in very low birth weight infants, where the model is found to have the assumed structure.     

Unconditional pseudo-maximum likelihood and adaptive estimation in the presence of conditional heterogeneity of unknown form

We consider parametric non-linear regression models with additive innovations which are serially uncorrelated but not necessarily independent, and consider the consequences of maximum likelihood and related one-step iterative estimation when the innovations are treated as being iid from their unconditional density. We find that the estimators' asymptotic covariance matrices will generally differ from those that would obtain if the errors actually were iid, except for the special case of strictly exogenous regressors. One important application of these results is to analysis of the properties of adaptive estimators, which employ nonparametric kernel estimates of the unconditional density of the disturbances in the construction of one-step iterative estimators. In the presence of strictly exogenous regressors, adaptive estimators are found to be asymptotically equivalent to the one-step iterative estimators that use the correct unconditional density. We illustrate our results through a brief Monte Carlo study.     

Variance Reduction in Smoothing Splines

Abstract.  We develop a variance reduction method for smoothing splines. For a given point of estimation, we define a variance-reduced spline estimate as a linear combination of classical spline estimates at three nearby points. We first develop a variance reduction method for spline estimators in univariate regression models. We then develop an analogous variance reduction method for spline estimators in clustered/longitudinal models. Simulation studies are performed which demonstrate the efficacy of our variance reduction methods in finite sample settings. Finally, a real data analysis with the motorcycle data set is performed. Here we consider variance estimation and generate 95% pointwise confidence intervals for the unknown regression function.     

Accent on Teaching Materials

Estimation of covariance components in the multivariate random-effect model with nested covariance structure is discussed. There are two covariance matrices to be estimated, namely, the between-group and the within-group covariance matrices. These two covariance matrices are most often estimated by forming a multivariate analysis of variance and equating mean square matrices to their expectations. Such a procedure involves taking the difference between the between-group mean square and the within-group mean square matrices, and often produces an estimated between-group covariance matrix that is not nonnegative definite. We present estimators of the two covariance matrices that are always proper covariance matrices. The estimators are the restricted maximum likelihood estimators if the random effects are normally distributed. The estimation procedure is extended to more complicated models, including the twofold nested and the mixed-effect models. A numerical example is presented to illustrate the use of the estimation procedure.     

Separated hypotheses testing for autoregressive models with non-negative residuals

Normal residual is one of the usual assumptions in autoregressive model but sometimes in practice we are faced with non-negative residuals. In this paper, we have derived modified maximum likelihood estimators of parameters of the residuals and autoregressive coefficient. Also asymptotic distribution of modified maximum likelihood estimators in both stationary and non-stationary models are computed. So that, we can derive asymptotic distribution of unit root, Vuong's and Cox's tests statistics in stationary situation. Using simulation, it shows that Akaike information criterion and Vuong's test work to select the optimal autoregressive model with non-negative residuals. Sometimes Vuong's test select two competing models as equivalent models. These models may be suitable or unsuitable equivalent models. So we consider Cox's test to make inference after model selection. Kolmogorov–Smirnov test confirms our results. Also we have computed tracking interval for competing models to choosing between two close competing models when Vuong's test and Cox's test cannot detect the differences.     

The Covariance Inflation Criterion for Adaptive Model Selection

We propose a new criterion for model selection in prediction problems. The covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses, when the prediction rule is applied to permuted versions of the data set. This criterion can be applied to general prediction problems (e.g. regression or classification) and to general prediction rules (e.g. stepwise regression, tree-based models and neural nets). As a by-product we obtain a measure of the effective number of parameters used by an adaptive procedure. We relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems. We also revisit the conditional bootstrap approach to model selection.     

Estimation of regression parameters in the presence of outliers in the response

In this paper, we consider a regression model with non-spherical covariance structure and outliers in the response. The generalized least squares estimator obtained from the full data set is generally not used in the presence of outliers and an estimator based on only the non-outlying observations is preferred. Here we propose as an estimator a convex combination of the full set and the deleted set estimators and compare its performance with the other two.     

On robust forecasting in dynamic vector time series models

In this article, robust estimation and prediction in multivariate autoregressive models with exogenous variables (VARX) are considered. The conditional least squares (CLS) estimators are known to be non-robust when outliers occur. To obtain robust estimators, the method introduced in Duchesne [2005. Robust and powerful serial correlation tests with new robust estimates in ARX models. J. Time Ser. Anal. 26, 49–81] and Bou Hamad and Duchesne [2005. On robust diagnostics at individual lags using RA-ARX estimators. In: Duchesne, P., Rémillard, B. (Eds.), Statistical Modeling and Analysis for Complex Data Problems. Springer, New York] is generalized for VARX models. The asymptotic distribution of the new estimators is studied and from this is obtained in particular the asymptotic covariance matrix of the robust estimators. Classical conditional prediction intervals normally rely on estimators such as the usual non-robust CLS estimators. In the presence of outliers, such as additive outliers, these classical predictions can be severely biased. More generally, the occurrence of outliers may invalidate the usual conditional prediction intervals. Consequently, the new robust methodology is used to develop robust conditional prediction intervals which take into account parameter estimation uncertainty. In a simulation study, we investigate the finite sample properties of the robust prediction intervals under several scenarios for the occurrence of the outliers, and the new intervals are compared to non-robust intervals based on classical CLS estimators.     

Difference-based estimation and model identification for panel data semiparametric models with cross-section dependence

Abstract

In this article, we consider a panel data partially linear regression model with fixed effect and non parametric time trend function. The data can be dependent cross individuals through linear regressor and error components. Unlike the methods using non parametric smoothing technique, a difference-based method is proposed to estimate linear regression coefficients of the model to avoid bandwidth selection. Here the difference technique is employed to eliminate the non parametric function effect, not the fixed effects, on linear regressor coefficient estimation totally. Therefore, a more efficient estimator for parametric part is anticipated, which is shown to be true by the simulation results. For the non parametric component, the polynomial spline technique is implemented. The asymptotic properties of estimators for parametric and non parametric parts are presented. We also show how to select informative ones from a number of covariates in the linear part by using smoothly clipped absolute deviation-penalized estimators on a difference-based least-squares objective function, and the resulting estimators perform asymptotically as well as the oracle procedure in terms of selecting the correct model.