A recursive approach to parameter estimation in regression and time series models

In this paper we discuss the recursive (or on line) estimation in (i) regression and (ii) autoregressive integrated moving average (ARIMA) time series models. The adopted approach uses Kalman filtering techniques to calculate estimates recursively. This approach is used for the estimation of constant as well as time varying parameters. In the first section of the paper we consider the linear regression model. We discuss recursive estimation both for constant and time varying parameters. For constant parameters, Kalman filtering specializes to recursive least squares. In general, we allow the parameters to vary according to an autoregressive integrated moving average process and update the parameter estimates recursively. Since the stochastic model for the parameter changes will "be rarely known, simplifying assumptions have to be made. In particular we assume a random walk model for the time varying parameters and show how to determine whether the parameters are changing over time. This is illustrated with an example.     

Inference for linear and nonlinear stable error processes via estimating functions

This paper describes an estimating function approach for parameter estimation in linear and nonlinear times series models with infinite variance stable errors. Joint estimates of location and scale parameters are derived for classes of autoregressive (AR) models and random coefficient autoregressive (RCA) models with stable errors, as well as for AR models with stable autoregressive conditionally heteroscedastic (ARCH) errors. Fast, on-line, recursive parametric estimation for the location parameter based on estimating functions is discussed using simulation studies. A real financial time series is also discussed in some detail.     

Recursive estimation of bilinear time series models

In this paper we are concerned with the recursive estimation of bilinear models. Some methods from linear time invariant systems are adapted to suit bilinear time series models. The time-varying Kalman filter and associated parameter estimation algorithm is carried on the bilinear time series models. The methods are illustrated with examples.     

On the Nonlinear Predictability of Stock Returns Using Financial and Economic Variables

In a recent article by Qi, neural networks trained by Bayesian regularization were used to predict excess returns on the S&P 500. The article concluded that the switching portfolio based on the recursive neural-network forecasts generates higher accumulated wealth with lower risks than that based on linear regression. Unfortunately, attempts to replicate the results were unsuccessful. Replicated results using the same software, approach and data detailed by Qi indicate that, in fact, the switching portfolio based on the recursive neural-network forecasts generates lower accumulated wealth with higher risks than that based on linear regression.     

Fourth Quarter Survey of the Economic Outlook

Written mainly for its pedagogical interest, this note deals with the computational formulas for the recursive updating of weighted least squares parameter estimates and the residual sum of squares in the general linear model under the assumption that the errors have a multivariate normal distribution. This approach simplifies considerably the derivations of Haslett (1985).     

Bootstrapping time series models

This paper surveys recent development in bootstrap methods and the modifications needed for their applicability in time series models. The paper discusses some guidelines for empirical researchers in econometric analysis of time series. Different sampling schemes for bootstrap data generation and different forms of bootstrap test statistics are discussed. The paper also discusses the applicability of direct bootstrapping of data in dynamic models and cointegrating regression models. It is argued that bootstrapping residuals is the preferable approach. The bootstrap procedures covered include the recursive bootstrap, the moving block bootstrap and the stationary bootstrap.     

Bootstrapping time series models

This paper surveys recent development in bootstrap methods and the modifications needed for their applicability in time series models. The paper discusses some guidelines for empirical researchers in econometric analysis of time series. Different sampling schemes for bootstrap data generation and different forms of bootstrap test statistics are discussed. The paper also discusses the applicability of direct bootstrapping of data in dynamic models and cointegrating regression models. It is argued that bootstrapping residuals is the preferable approach. The bootstrap procedures covered include the recursive bootstrap, the moving block bootstrap and the stationary bootstrap.     

RECURSIVE ESTIMATION OF THE GENERAL LINEAR MODEL WITH DEPENDENT ERRORS AND MULTIPLE ADDITIONAL OBSERVATIONS1

The techniques for recursive estimation of the general linear model with dependent errors and known second order properties, is generalised to allow for simultaneous addition of an arbitrary number of additional observations. Computational formulae for recursive updating of parameter estimates are derived, together with a sequence of univariate recursive residuals for testing the constancy of the regression relation over time.     

Identification of vector AR models with recursive structural errors using conditional independence graphs

In canonical vector time series autoregressions, which permit dependence only on past values, the errors generally show contemporaneous correlation. By contrast structural vector autoregressions allow contemporaneous series dependence and assume errors with no contemporaneous correlation. Such models having a recursive structure can be described by a directed acyclic graph. We show, with the use of a real example, how the identification of these models may be assisted by examination of the conditional independence graph of contemporaneous and lagged variables. In this example we identify the causal dependence of monthly Italian bank loan interest rates on government bond and repurchase agreement rates. When the number of series is larger, the structural modelling of the canonical errors alone is a useful initial step, and we first present such an example to demonstrate the general approach to identifying a directed graphical model.     

Recursive online EM estimation of mixture autoregressions

This paper develops a recursive expectation–maximization (REM) algorithm for estimating a mixture autoregression (MAR) with an independent and identically distributed regime transition process. The proposed method, which is useful for long time series as well as for data available in real time, follows a recursive predictor error-type scheme. Based on a slightly modified system to the expectation–maximization (EM) equations for an MAR model, the REM algorithm consists of two steps at each iteration: the expectation step, in which the current unobserved regime transition is estimated from new data using previous recursive estimates, and the minimization step, in which the MAR parameter estimates are recursively updated following a minimization direction. Details of implementation of the REM algorithm are given and its finite-sample performance is shown via simulation experiments. In particular, the EM and REM provide roughly similar estimates, especially for moderate and long time series.     

Multiple hypothesis test for parameter constancy based on recursive residuals

This article presents a multiple hypothesis test procedure that combines two well known tests for structural change in the linear regression model, the CUSUM test and the recursive t test. The CUSUM test is run through the sequence of recursive residuals as usual; if the CUSUM plot does not violate the critical lines, one more step is taken to perform the t test for hypothesis of zero mean based on all recursive residuals. The asymptotic size of this multiple hypothesis test is derived; power simulation results suggest that it outperforms the traditional CUSUM test and complements other tests that are currently stressed in econometrics.     

Multiple hypothesis test for parameter constancy based on recursive residuals

This article presents a multiple hypothesis test procedure that combines two well known tests for structural change in the linear regression model, the CUSUM test and the recursive t test. The CUSUM test is run through the sequence of recursive residuals as usual; if the CUSUM plot does not violate the critical lines, one more step is taken to perform the t test for hypothesis of zero mean based on all recursive residuals. The asymptotic size of this multiple hypothesis test is derived; power simulation results suggest that it outperforms the traditional CUSUM test and complements other tests that are currently stressed in econometrics.     

Robust recursive estimation in nonlinear time series

Aase (1983) has dealt with recursive estimation in nonlinear time series of autoregressive type including its asymptotic properties. This contribution modifies the results for the case of nonlinear time series with outliers using the principle of M-estimation from robust statistics. Strong consistency of the robust recursive estimates is preserved under corresponding assumptions. Several types of such estimates are compared by means of a numerical simulation.     

Recursive techniques for the construction of experimental designs

The construction of experimental designs by recursive techniques is studied in this paper. Formulae for the recursive addition or deletion of data from a design are derived for a typical sub-hypothesis situation of a general experimental design. These results are used to consider the recursive construction of experimental designs with respect to different optimality criteria. This approach to the construction of designs is quite different to that of the well-established theory of optimal design.     

STAR模型中的递归退势单位根检验研究

在STAR模型框架下,考虑时间序列具有线性确定性趋势成分,本文建立了一个递归退势单位根检验统计量,推导了其渐近分布;并在考虑初始条件情形下,对递归退势、OLS和GLS退势单位根检验统计量的有限样本性质进行了细致的比较研究。若忽略初始条件的影响,GLS退势和递归退势单位根检验统计量的检验势都显著高于OLS退势。随着初始条件的增大,GLS退势单位根检验统计量的检验势下降得比较厉害,递归退势单位根检验统计量的检验势较为稳定,且在样本量较大情形下更具优势。     

Recursive integration methodologies with statistical applications

This paper shows how recursive integration methodologies can be used to evaluate high-dimensional integral expressions. This has applications to many areas of statistical inference where probability calculations and critical point evaluations often require such high-dimensional integral evaluations. Recursive integration can allow an integral expression of a given dimension to be evaluated by a series of calculations of a smaller dimension. This significantly reduces the computation time. The application of the recursive integration methodology is illustrated with several examples.     

Recursive Estimation of GARCH Models

This article develops three recursive on-line algorithms, based on a two-stage least squares scheme for estimating generalized autoregressive conditionally heteroskedastic (GARCH) models. The first one, denoted by 2S-RLS, is an adaptation of the recursive least squares method for estimating autoregressive conditionally heteroskedastic (ARCH) models. The second and the third ones (denoted, respectively, by 2S-PLR and 2S-RML) are adapted versions of the pseudolinear regression (PLR) and the recursive maximum likelihood (RML) methods to the GARCH case. We show that the proposed algorithms give consistent estimators and that the 2S-RLS and the 2S-RML estimators are asymptotically Gaussian. These methods seem very adequate for modeling the sequential feature of financial time series, which are observed on a high-frequency basis. The performance of these algorithms is shown via a simulation study.     

Conditional inference in linear versus nonlinear models for binary time series

The modelling of discrete such as binary time series, unlike the continuous time series, is not easy. This is due to the fact that there is no unique way to model the correlation structure of the repeated binary data. Some models may also provide a complicated correlation structure with narrow ranges for the correlations. In this paper, we consider a nonlinear dynamic binary time series model that provides a correlation structure which is easy to interpret and the correlations under this model satisfy the full?1 to 1 range. For the estimation of the parameters of this nonlinear model, we use a conditional generalized quasilikelihood (CGQL) approach which provides the same estimates as those of the well-known maximum likelihood approach. Furthermore, we consider a competitive linear dynamic binary time series model and examine the performance of the CGQL approach through a simulation study in estimating the parameters of this linear model. The model mis-specification effects on estimation as well as forecasting are also examined through simulations.     

Recursive improvement of estimates in a Gauss-Markov model with linear restrictions

The minimum-dispersion linear unbiased estimator of a set of estimable functions in a general Gauss-Markov model with double linear restrictions is considered. The attention is focused on developing a recursive formula in which an initial estimator, obtained from the unrestricted model, is corrected with respect to the restrictions successively incorporated into the model. The established formula generalizes known results developed for the simple Gauss-Markov model.     

Subgroup identification by recursive segmentation

A new modeling approach called ‘recursive segmentation’ is proposed to support the supervised exploration and identification of subgroups or clusters. It is based on the frameworks of recursive partitioning and the Patient Rule Induction Method (PRIM). Through combining these methods, recursive segmentation aims to exploit their respective strengths while reducing their weaknesses. Consequently, recursive segmentation can be applied in a very general way, that is in any (multivariate) regression, classification or survival (time-to-event) problem, using conditional inference, evolutionary learning or the CART algorithm, with predictor variables of any scale and with missing values. Furthermore, results of a synthetic example and a benchmark application study that comprises 26 data sets suggest that recursive segmentation achieves a competitive prediction accuracy and provides more accurate definitions of subgroups by models of less complexity as compared to recursive partitioning and PRIM. An application to the German Breast Cancer Study Group data demonstrates the improved interpretability and reliability of results produced by the new approach. The method is made publicly available through the R-package rseg (http://rseg.r-forge.r-project.org/).