Modeling nonlinear time series with local mixtures of generalized linear models

The authors consider a novel class of nonlinear time series models based on local mixtures of regressions of exponential family models, where the covariates include functions of lags of the dependent variable. They give conditions to guarantee consistency of the maximum likelihood estimator for correctly specified models, with stationary and nonstationary predictors. They show that consistency of the maximum likelihood estimator still holds under model misspecification. They also provide probabilistic results for the proposed model when the vector of predictors contains only lags of transformations of the modeled time series. They illustrate the consistency of the maximum likelihood estimator and the probabilistic properties via Monte Carlo simulations. Finally, they present an application using real data.     

Boosting techniques for nonlinear time series models

Many of the popular nonlinear time series models require a priori the choice of parametric functions which are assumed to be appropriate in specific applications. This approach is mainly used in financial applications, when sufficient knowledge is available about the nonlinear structure between the covariates and the response. One principal strategy to investigate a broader class on nonlinear time series is the Nonlinear Additive AutoRegressive (NAAR) model. The NAAR model estimates the lags of a time series as flexible functions in order to detect non-monotone relationships between current and past observations. We consider linear and additive models for identifying nonlinear relationships. A componentwise boosting algorithm is applied for simultaneous model fitting, variable selection, and model choice. Thus, with the application of boosting for fitting potentially nonlinear models we address the major issues in time series modelling: lag selection and nonlinearity. By means of simulation we compare boosting to alternative nonparametric methods. Boosting shows a strong overall performance in terms of precise estimations of highly nonlinear lag functions. The forecasting potential of boosting is examined on the German industrial production (IP); to improve the model’s forecasting quality we include additional exogenous variables. Thus we address the second major aspect in this paper which concerns the issue of high dimensionality in models. Allowing additional inputs in the model extends the NAAR model to a broader class of models, namely the NAARX model. We show that boosting can cope with large models which have many covariates compared to the number of observations.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

Abstract

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

Efficient Estimation of the Seemingly Unrelated Regression Cointegration Model and Testing for Purchasing Power Parity

This paper studies the efficient estimation of seemingly unrelated linear models with integrated regressors and stationary errors. We consider two cases. The first one has no common regressor among the equations. In this case, we show that by adding leads and lags of the first differences of the regressors and estimating this augmented dynamic regression model by generalized least squares using the long-run covariance matrix, we obtain an efficient estimator of the cointegrating vector that has a limiting mixed normal distribution. In the second case we consider, there is a common regressor to all equations, and we discuss efficient minimum distance estimation in this context. Simulation results suggests that our new estimator compares favorably with others already proposed in the literature. We apply these new estimators to the testing of the proportionality and symmetry conditions implied by purchasing power parity (PPP) among the G-7 countries. The tests based on the efficient estimates easily reject the joint hypotheses of proportionality and symmetry for all countries with either the United States or Germany as numeraire. Based on individual tests, our results suggest that Canada and Germany are the most likely countries for which the proportionality condition holds, and that Italy and Japan for the symmetry condition relative to the United States.     

Space-time short- to medium-term wind speed forecasting

Accurate wind power forecasts depend on reliable wind speed forecasts. Numerical weather predictions utilize huge amounts of computing time, but still have rather low spatial and temporal resolution. However, stochastic wind speed forecasts perform well in rather high temporal resolution settings. They consume comparably little computing resources and return reliable forecasts, if forecasting horizons are not too long. In the recent literature, spatial interdependence is increasingly taken into consideration. In this paper we propose a new and quite flexible multivariate model that accounts for neighbouring weather stations’ information and as such, exploits spatial data at a high resolution. The model is applied to forecasting horizons of up to 1 day and is capable of handling a high resolution temporal structure. We use a periodic vector autoregressive model with seasonal lags to account for the interaction of the explanatory variables. Periodicity is considered and is modelled by cubic B-splines. Due to the model’s flexibility, the number of explanatory variables becomes huge. Therefore, we utilize time-saving shrinkage methods like lasso and elastic net for estimation. Particularly, a relatively newly developed iteratively re-weighted lasso and elastic net is applied that also incorporates heteroscedasticity. We compare our model to several benchmarks. The out-of-sample forecasting results show that the exploitation of spatial information increases the forecasting accuracy tremendously, in comparison to models in use so far.     

Asymptotic variance–covariance matrix of sample autocorrelations for threshold-asymmetric GARCH processes

In the field of financial time series, threshold-asymmetric conditional variance models can be used to explain asymmetric volatilities [C.W. Li and W.K. Li, On a double-threshold autoregressive heteroscedastic time series model, J. Appl. Econometrics 11 (1996), pp. 253–274]. In this paper, we consider a broad class of threshold-asymmetric GARCH processes (TAGARCH, hereafter) including standard ARCH and GARCH models as special cases. Since sample autocorrelation function provides a useful information to identify an appropriate time-series model for the data, we derive asymptotic distributions of sample autocorrelations both for original process and for squared process. It is verified that standard errors of sample autocorrelations for TAGARCH models are significantly different from unity for lower lags and they are exponentially converging to unity for higher lags. Furthermore they are shown to be asymptotically dependent while being independent of standard GARCH models. These results will be interesting in the light of the fact that TAGARCH processes are serially uncorrelated. A simulation study is reported to illustrate our results.     

Measuring Time Series Predictability Using Support Vector Regression

Most studies involving statistical time series analysis rely on assumptions of linearity, which by its simplicity facilitates parameter interpretation and estimation. However, the linearity assumption may be too restrictive for many practical applications. The implementation of nonlinear models in time series analysis involves the estimation of a large set of parameters, frequently leading to overfitting problems. In this article, a predictability coefficient is estimated using a combination of nonlinear autoregressive models and the use of support vector regression in this model is explored. We illustrate the usefulness and interpretability of results by using electroencephalographic records of an epileptic patient.     

Multivariate elliptical models with general parameterization

In this paper we introduce a general elliptical multivariate regression model in which the mean vector and the scale matrix have parameters (or/and covariates) in common. This approach unifies several important elliptical models, such as nonlinear regressions, mixed-effects model with nonlinear fixed effects, errors-in-variables models, and so forth. We discuss maximum likelihood estimation of the model parameters and obtain the information matrix, both observed and expected. Additionally, we derived the generalized leverage as well as the normal curvatures of local influence under some perturbation schemes. An empirical application is presented for illustrative purposes.     

Testing the Granger Noncausality Hypothesis in Stationary Nonlinear Models of Unknown Functional Form

In this article, we propose a general method for testing the Granger noncausality hypothesis in stationary nonlinear models of unknown functional form. These tests are based on a Taylor expansion of the nonlinear model around a given point in the sample space. We study the performance of our tests by a Monte Carlo experiment and compare these to the most widely used linear test. Our tests appear to be well-sized and have reasonably good power properties.     

Multiple-index approach to multiple autoregressive time series model

Time series which have more than one time dependent variable require building an appropriate model in which the variables not only have relationships with each other, but also depend on previous values in time. Based on developments for a sufficient dimension reduction, we investigate a new class of multiple time series models without parametric assumptions. First, for the dependent and independent time series, we simply use a univariate time series central subspace to estimate the autoregressive lags of the series. Secondly, we extract the successive directions to estimate the time series central subspace for regressors which include past lags of dependent and independent series in a mutual information multiple-index time series. Lastly, we estimate a multiple time series model for the reduced directions. In this article, we propose a unified estimation method of minimal dimension using an Akaike information criterion, for situations in which the dimension for multiple regressors is unknown. We present an analysis using real data from the housing price index showing that our approach is an alternative for multiple time series modeling. In addition, we check the accuracy for the multiple time series central subspace method using three simulated data sets.     

A multifactor transformed diffusion model with applications to VIX and VIX futures

Abstract

Transformed diffusions (TDs) have become increasingly popular in financial modeling for their model flexibility and tractability. While existing TD models are predominately one-factor models, empirical evidence often prefers models with multiple factors. We propose a novel distribution-driven nonlinear multifactor TD model with latent components. Our model is a transformation of a underlying multivariate Ornstein–Uhlenbeck (MVOU) process, where the transformation function is endogenously specified by a flexible parametric stationary distribution of the observed variable. Computationally efficient exact likelihood inference can be implemented for our model using a modified Kalman filter algorithm and the transformed affine structure also allows us to price derivatives in semi-closed form. We compare the proposed multifactor model with existing TD models for modeling VIX and pricing VIX futures. Our results show that the proposed model outperforms all existing TD models both in the sample and out of the sample consistently across all categories and scenarios of our comparison.     

Gaussian models for degradation processes-part I: Methods for the analysis of biomarker data

We present two stochastic models that describe the relationship between biomarker process values at random time points, event times, and a vector of covariates. In both models the biomarker processes are degradation processes that represent the decay of systems over time. In the first model the biomarker process is a Wiener process whose drift is a function of the covariate vector. In the second model the biomarker process is taken to be the difference between a stationary Gaussian process and a time drift whose drift parameter is a function of the covariates. For both models we present statistical methods for estimation of the regression coefficients. The first model is useful for predicting the residual time from study entry to the time a critical boundary is reached while the second model is useful for predicting the latency time from the infection until the time the presence of the infection is detected. We present our methods principally in the context of conducting inference in a population of HIV infected individuals.     

Optimal designs for non-competitive enzyme inhibition kinetic models

In this paper, we present a new method for determining optimal designs for enzyme inhibition kinetic models, which are used to model the influence of the concentration of a substrate and an inhibition on the velocity of a reaction. The approach uses a nonlinear transformation of the vector of predictors such that the model in the new coordinates is given by an incomplete response surface model. Although there exist no explicit solutions of the optimal design problem for incomplete response surface models so far, the corresponding design problem in the new coordinates is substantially more transparent, such that explicit or numerical solutions can be determined more easily. The designs for the original problem can finally be found by an inverse transformation of the optimal designs determined for the response surface model. We illustrate the method determining explicit solutions for the D-optimal design and for the optimal design problem for estimating the individual coefficients in a non-competitive enzyme inhibition kinetic model.     

Nonlinear regression modeling via regularized radial basis function networks

The problem of constructing nonlinear regression models is investigated to analyze data with complex structure. We introduce radial basis functions with hyperparameter that adjusts the amount of overlapping basis functions and adopts the information of the input and response variables. By using the radial basis functions, we construct nonlinear regression models with help of the technique of regularization. Crucial issues in the model building process are the choices of a hyperparameter, the number of basis functions and a smoothing parameter. We present information-theoretic criteria for evaluating statistical models under model misspecification both for distributional and structural assumptions. We use real data examples and Monte Carlo simulations to investigate the properties of the proposed nonlinear regression modeling techniques. The simulation results show that our nonlinear modeling performs well in various situations, and clear improvements are obtained for the use of the hyperparameter in the basis functions.     

Pitfalls in Estimating Cointegrating Vector when Cointegration Relationship has Nonlinear Adjustment

This article investigates the properties of the estimators of the cointegrating vector when the cointegration error has a nonlinear adjustment. We investigate the properties of three estimators, namely, ordinary least squares (OLS), dynamic OLS (DOLS), and autoregressive distributed lag (ADL) models. Monte Carlo simulation results demonstrate that although all the estimators have consistency under cointegration with a nonlinear adjustment, they suffer from severe size distortions for the t-statistics of the cointegrating vector when the cointegration error has a highly persistent nonlinear adjustment and endogeneity. The results imply that the use of DOLS and ADL for cointegration with nonlinear adjustment cannot sufficiently improve the estimates and size performances.     

Threshold Vector Arma Models

In this article, we propose the threshold vector autoregressive moving average model (TVARMA). It is a multivariate nonlinear time series model characterized by two or more regimes that follow a vector ARMA structure and where the switching among them is regulated by a latent variable. The TVARMA model represents a generalization of some nonlinear models proposed in the literature and shows interesting features that are explored. The condition for the strong and weak stationarity of the TVARMA model are presented and the moments up to order two of the process are derived.     

A Statistical Method for the Determination of the Appropriate Order in a General Class of Time Series Models

Abstract

We propose a method to determine the order q of a model in a general class of time series models. For the subset of linear moving average models (MA(q)), our method is compared with that of the sample autocorrelations. Since the sample autocorrelation is meant to detect a linear structure of dependence between random variables, it turns out to be more suitable for the linear case. However, our method presents a competitive option in that case, and for nonlinear models (NLMA(q)) it is shown to work better. The main advantages of our approach are that it does not make assumptions on the existence of moments and on the distribution of the noise involved in the moving average models. We also include an example with real data corresponding to the daily returns of the exchange rate process of mexican pesos and american dollars.     

Asymmetric Forecast Densities for U.S. Macroeconomic Variables from a Gaussian Copula Model of Cross-Sectional and Serial Dependence

Most existing reduced-form macroeconomic multivariate time series models employ elliptical disturbances, so that the forecast densities produced are symmetric. In this article, we use a copula model with asymmetric margins to produce forecast densities with the scope for severe departures from symmetry. Empirical and skew t distributions are employed for the margins, and a high-dimensional Gaussian copula is used to jointly capture cross-sectional and (multivariate) serial dependence. The copula parameter matrix is given by the correlation matrix of a latent stationary and Markov vector autoregression (VAR). We show that the likelihood can be evaluated efficiently using the unique partial correlations, and estimate the copula using Bayesian methods. We examine the forecasting performance of the model for four U.S. macroeconomic variables between 1975:Q1 and 2011:Q2 using quarterly real-time data. We find that the point and density forecasts from the copula model are competitive with those from a Bayesian VAR. During the recent recession the forecast densities exhibit substantial asymmetry, avoiding some of the pitfalls of the symmetric forecast densities from the Bayesian VAR. We show that the asymmetries in the predictive distributions of GDP growth and inflation are similar to those found in the probabilistic forecasts from the Survey of Professional Forecasters. Last, we find that unlike the linear VAR model, our fitted Gaussian copula models exhibit nonlinear dependencies between some macroeconomic variables. This article has online supplementary material.     

SimSel: a new simulation method for variable selection

We propose a new simulation method, SimSel, for variable selection in linear and nonlinear modelling problems. SimSel works by disturbing the input data with pseudo-errors. We then study how this disturbance affects the quality of an approximative model fitted to the data. The main idea is that disturbing unimportant variables does not affect the quality of the model fit. The use of an approximative model has the advantage that the true underlying function does not need to be known and that the method becomes insensitive to model misspecifications. We demonstrate SimSel on simulated data from linear and nonlinear models and on two real data sets. The simulation studies suggest that SimSel works well in complicated situations, such as nonlinear errors-in-variable models.     

Local regression for vector responses

We explore a class of vector smoothers based on local polynomial regression for fitting nonparametric regression models which have a vector response. The asymptotic bias and variance for the class of estimators are derived for two different ways of representing the variance matrices within both a seemingly unrelated regression and a vector measurement error framework. We show that the asymptotic behaviour of the estimators is different in these four cases. In addition, the placement of the kernel weights in weighted least squares estimators is very important in the seeming unrelated regressions problem (to ensure that the estimator is asymptotically unbiased) but not in the vector measurement error model. It is shown that the component estimators are asymptotically uncorrelated in the seemingly unrelated regressions model but asymptotically correlated in the vector measurement error model. These new and interesting results extend our understanding of the problem of smoothing dependent data.