Wavelet packet transfer function modelling of nonstationary time series

This article shows how a non-decimated wavelet packet transform (NWPT) can be used to model a response time series, Y _t, in terms of an explanatory time series, X _t. The proposed computational technique transforms the explanatory time series into a NWPT representation and then uses standard statistical modelling methods to identify which wavelet packets are useful for modelling the response time series. We exhibit S-Plus functions from the freeware WaveThresh package that implement our methodology.The proposed modelling methodology is applied to an important problem from the wind energy industry: how to model wind speed at a target location using wind speed and direction from a reference location. Our method improves on existing target site wind speed predictions produced by widely used industry standard techniques. However, of more importance, our NWPT representation produces models to which we can attach physical and scientific interpretations and in the wind example enable us to understand more about the transfer of wind energy from site to site.     

Some applications of nonlinear and non-Gaussian state–space modelling by means of hidden Markov models

Nonlinear and non-Gaussian state–space models (SSMs) are fitted to different types of time series. The applications include homogeneous and seasonal time series, in particular earthquake counts, polio counts, rainfall occurrence data, glacial varve data and daily returns on a share. The considered SSMs comprise Poisson, Bernoulli, gamma and Student-t distributions at the observation level. Parameter estimations for the SSMs are carried out using a likelihood approximation that is obtained after discretization of the state space. The approximation can be made arbitrarily accurate, and the approximated likelihood is precisely that of a finite-state hidden Markov model (HMM). The proposed method enables us to apply standard HMM techniques. It is easy to implement and can be extended to all kinds of SSMs in a straightforward manner.     

Non-Gaussian Conditional Linear AR(1) Models

This paper gives a general formulation of a non-Gaussian conditional linear AR(1) model subsuming most of the non-Gaussian AR(1) models that have appeared in the literature. It derives some general results giving properties for the stationary process mean, variance and correlation structure, and conditions for stationarity. These results highlight similarities with and differences from the Gaussian AR(1) model, and unify many separate results appearing in the literature. Examples illustrate the wide range of properties that can appear under the conditional linear autoregressive assumption. These results are used in analysing three real datasets, illustrating general methods of estimation, model diagnostics and model selection. In particular, the theoretical results can be used to develop diagnostics for deciding if a time series can be modelled by some linear autoregressive model, and for selecting among several candidate models.     

Short-term forecasting with a computationally efficient nonparametric transfer function model

In this paper a semi-parametric approach is developed to model non-linear relationships in time series data using polynomial splines. Polynomial splines require very little assumption about the functional form of the underlying relationship, so they are very flexible and can be used to model highly non-linear relationships. Polynomial splines are also computationally very efficient. The serial correlation in the data is accounted for by modelling the noise as an autoregressive integrated moving average (ARIMA) process, by doing so, the efficiency in nonparametric estimation is improved and correct inferences can be obtained. The explicit structure of the ARIMA model allows the correlation information to be used to improve forecasting performance. An algorithm is developed to automatically select and estimate the polynomial spline model and the ARIMA model through backfitting. This method is applied on a real-life data set to forecast hourly electricity usage. The non-linear effect of temperature on hourly electricity usage is allowed to be different at different hours of the day and days of the week. The forecasting performance of the developed method is evaluated in post-sample forecasting and compared with several well-accepted models. The results show the performance of the proposed model is comparable with a long short-term memory deep learning model.     

Theory & Methods: Residual Diagnostic Plots for Checking for Model Mis‐specification in Time Series Regression

This paper considers residuals for time series regression. Despite much literature on visual diagnostics for uncorrelated data, there is little on the autocorrelated case. To examine various aspects of the fitted time series regression model, three residuals are considered. The fitted regression model can be checked using orthogonal residuals; the time series error model can be analysed using marginal residuals; and the white noise error component can be tested using conditional residuals. When used together, these residuals allow identification of outliers, model mis‐specification and mean shifts. Due to the sensitivity of conditional residuals to model mis‐specification, it is suggested that the orthogonal and marginal residuals be examined first.     

On time-irreversibility and other non-linear features in time series

Time-irreversibility, asymmetry of the distribution, and the occurrence of sudden bursts are considered, amongst others, as non-linear features in time series modeling. The implication is often made that time series showing these features must be analyzed using non-linear models. In contrast, this paper shows that time-irreversible asymmetric time series showing certain types of sudden bursts may be generated by linear models with adequate input sequences. Thus some non-linear time series features may be caused by the pattern in the input sequence rather than by non-linearity in the model. Examples are considered to illustrate the situation.     

Some Properties and Generalizations of Non-negative Bayesian Time Series Models

We study the most basic Bayesian forecasting model for exponential family time series, the power steady model (PSM) of Smith, in terms of observable properties of one-step forecast distributions and sample paths. The PSM implies a constraint between location and spread of the forecast distribution. Including a scale parameter in the models does not always give an exact solution free of this problem, but it does suggest how to define related models free of the constraint. We define such a class of models which contains the PSM. We concentrate on the case where observations are non-negative. Probability theory and simulation show that under very mild conditions almost all sample paths of these models converge to some constant, making them unsuitable for modelling in many situations. The results apply more generally to non-negative models defined in terms of exponentially weighted moving averages. We use these and related results to motivate, define and apply very simple models based on directly specifying the forecast distributions.     

A dynamic model for double‐bounded time series with chaotic‐driven conditional averages

In this work, we introduce a class of dynamic models for time series taking values on the unit interval. The proposed model follows a generalized linear model approach where the random component, conditioned on the past information, follows a beta distribution, while the conditional mean specification may include covariates and also an extra additive term given by the iteration of a map that can present chaotic behavior. The resulting model is very flexible and its systematic component can accommodate short‐ and long‐range dependence, periodic behavior, laminar phases, etc. We derive easily verifiable conditions for the stationarity of the proposed model, as well as conditions for the law of large numbers and a Birkhoff‐type theorem to hold. A Monte Carlo simulation study is performed to assess the finite sample behavior of the partial maximum likelihood approach for parameter estimation in the proposed model. Finally, an application to the proportion of stored hydroelectrical energy in Southern Brazil is presented.     

Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions

Summary. The major implementational problem for reversible jump Markov chain Monte Carlo methods is that there is commonly no natural way to choose jump proposals since there is no Euclidean structure in the parameter space to guide our choice. We consider mechanisms for guiding the choice of proposal. The first group of methods is based on an analysis of acceptance probabilities for jumps. Essentially, these methods involve a Taylor series expansion of the acceptance probability around certain canonical jumps and turn out to have close connections to Langevin algorithms. The second group of methods generalizes the reversible jump algorithm by using the so-called saturated space approach. These allow the chain to retain some degree of memory so that, when proposing to move from a smaller to a larger model, information is borrowed from the last time that the reverse move was performed. The main motivation for this paper is that, in complex problems, the probability that the Markov chain moves between such spaces may be prohibitively small, as the probability mass can be very thinly spread across the space. Therefore, finding reasonable jump proposals becomes extremely important. We illustrate the procedure by using several examples of reversible jump Markov chain Monte Carlo applications including the analysis of autoregressive time series, graphical Gaussian modelling and mixture modelling.     

Modelling concurrency of events in on-line auctions via spatiotemporal semiparametric models

Summary.  We introduce a semiparametric approach for modelling the effect of concurrent events on an outcome of interest. Concurrency manifests itself as temporal and spatial dependences. By temporal dependence we mean the effect of an event in the past. Modelling this effect is challenging since events arrive at irregularly spaced time intervals. For the spatial part we use an abstract notion of 'feature space' to conceptualize distances among a set of item features. We motivate our model in the context of on-line auctions by modelling the effect of concurrent auctions on an auction's price. Our concurrency model consists of three components: a transaction-related component that accounts for auction design and bidding competition, a spatial component that takes into account similarity between item features and a temporal component that accounts for recently closed auctions. To construct each of these we borrow ideas from spatial and mixed model methodology. The power of this model is illustrated on a large and diverse set of laptop auctions on eBay.com. We show that our model results in superior predictive performance compared with a set of competitor models. The model also allows for new insight into the factors that drive price in on-line auctions and their relationship to bidding competition, auction design, product variety and temporal learning effects.     

Functional time series approach for forecasting very short-term electricity demand

This empirical paper presents a number of functional modelling and forecasting methods for predicting very short-term (such as minute-by-minute) electricity demand. The proposed functional methods slice a seasonal univariate time series (TS) into a TS of curves; reduce the dimensionality of curves by applying functional principal component analysis before using a univariate TS forecasting method and regression techniques. As data points in the daily electricity demand are sequentially observed, a forecast updating method can greatly improve the accuracy of point forecasts. Moreover, we present a non-parametric bootstrap approach to construct and update prediction intervals, and compare the point and interval forecast accuracy with some naive benchmark methods. The proposed methods are illustrated by the half-hourly electricity demand from Monday to Sunday in South Australia.     

ASSOCIATIONS BETWEEN AIR POLLUTION AND HOSPITAL VISITS FOR CARDIOVASCULAR DISEASES IN THE ELDERLY IN SYDNEY USING BAYESIAN STATISTICAL METHODS

Using generalized linear models (GLMs), Jalaludin  et al. (2006;  J. Exposure Analysis and Epidemiology   16 , 225–237) studied the association between the daily number of visits to emergency departments for cardiovascular disease by the elderly (65+) and five measures of ambient air pollution. Bayesian methods provide an alternative approach to classical time series modelling and are starting to be more widely used. This paper considers Bayesian methods using the dataset used by Jalaludin  et al.  (2006) , and compares the results from Bayesian methods with those obtained by Jalaludin  et al.  (2006) using GLM methods.     

A copula-based Markov chain model for the analysis of binary longitudinal data

A fully parametric first-order autoregressive (AR(1)) model is proposed to analyse binary longitudinal data. By using a discretized version of a copula, the modelling approach allows one to construct separate models for the marginal response and for the dependence between adjacent responses. In particular, the transition model that is focused on discretizes the Gaussian copula in such a way that the marginal is a Bernoulli distribution. A probit link is used to take into account concomitant information in the behaviour of the underlying marginal distribution. Fixed and time-varying covariates can be included in the model. The method is simple and is a natural extension of the AR(1) model for Gaussian series. Since the approach put forward is likelihood-based, it allows interpretations and inferences to be made that are not possible with semi-parametric approaches such as those based on generalized estimating equations. Data from a study designed to reduce the exposure of children to the sun are used to illustrate the methods.     

Variable selection for first‐order Poisson integer‐valued autoregressive model with covariables

In recent years, modelling count data has become one of the most important and popular topics in time‐series analysis. At the same time, variable selection methods have become widely used in many fields as an effective statistical modelling tool. In this paper, we consider using a variable selection method to solve a modelling problem regarding the first‐order Poisson integer‐valued autoregressive (PINAR(1)) model with covariables. The PINAR(1) model with covariables is widely used in many areas because of its practicality. When using this model to deal with practical problems, multiple covariables are added to the model because it is impossible to know in advance which covariables will affect the results. But the inclusion of some insignificant covariables is almost impossible to avoid. Unfortunately, the usual estimation method is not adequate for the task of deleting the insignificant covariables that cause statistical inferences to become biased. To overcome this defect, we propose a penalised conditional least squares (PCLS) method, which can consistently select the true model. The PCLS estimator is also provided and its asymptotic properties are established. Simulation studies demonstrate that the PCLS method is effective for estimation and variable selection. One practical example is also presented to illustrate the practicability of the PCLS method.     

Fitting Time Series Models by Minimizing Multistep-ahead Errors: a Frequency Domain Approach

This paper brings together two topics in the estimation of time series forecasting models: the use of the multistep-ahead error sum of squares as a criterion to be minimized and frequency domain methods for carrying out this minimization. The methods are developed for the wide class of time series models having a spectrum which is linear in unknown coefficients. This includes the IMA(1, 1) model for which the common exponentially weigh-ted moving average predictor is optimal, besides more general structural models for series exhibiting trends and seasonality. The method is extended to include the Box–Jenkins `air line' model. The value of the multistep criterion is that it provides protection against using an incorrectly specified model. The value of frequency domain estimation is that the iteratively reweighted least squares scheme for fitting generalized linear models is readily extended to construct the parameter estimates and their standard errors. It also yields insight into the loss of efficiency when the model is correct and the robustness of the criterion against an incorrect model. A simple example is used to illustrate the method, and a real example demonstrates the extension to seasonal models. The discussion considers a diagnostic test statistic for indicating an incorrect model.     

A Bayesian hierarchical spatio-temporal rainfall model

A Bayesian hierarchical spatio-temporal rainfall model is presented and analysed. The model has the ability to deal with extensive missing or null values, uses a sophisticated variance stabilising rainfall pre-transformation, incorporates a new elevation model and can provide sub-catchment rainfall estimation and interpolation using a sequential kriging scheme. The model uses a vector autoregressive stochastic process to represent the time dependence of the rainfall field and an exponential covariogram to model the spatial correlation of the rainfall field. The model can be readily generalised to other types of stochastic processes. In this paper, some results of applying the model to a particular rainfall catchment are presented.     

Statistical methods for regular monitoring data

Summary.  Meteorological and environmental data that are collected at regular time intervals on a fixed monitoring network can be usefully studied combining ideas from multiple time series and spatial statistics, particularly when there are little or no missing data. This work investigates methods for modelling such data and ways of approximating the associated likelihood functions. Models for processes on the sphere crossed with time are emphasized, especially models that are not fully symmetric in space–time. Two approaches to obtaining such models are described. The first is to consider a rotated version of fully symmetric models for which we have explicit expressions for the covariance function. The second is based on a representation of space–time covariance functions that is spectral in just the time domain and is shown to lead to natural partially nonparametric asymmetric models on the sphere crossed with time. Various models are applied to a data set of daily winds at 11 sites in Ireland over 18 years. Spectral and space–time domain diagnostic procedures are used to assess the quality of the fits. The spectral-in-time modelling approach is shown to yield a good fit to many properties of the data and can be applied in a routine fashion relative to finding elaborate parametric models that describe the space–time dependences of the data about as well.     

Employee turnover forecasting for human resource management based on time series analysis

In some organizations, the hiring lead time is often long due to responding to human resource requirements associated with technical and security constrains. Thus, the human resource departments in these organizations are pretty interested in forecasting employee turnover since a good prediction of employee turnover could help the organizations to minimize the costs and impacts from the turnover on the operational capabilities and the budget. This study aims to enhance the ability to forecast employee turnover with or without considering the impact of economic indicators. Various time series modelling techniques were used to identify optimal models for effective employee turnover prediction. More than 11-years of monthly turnover data were used to build and validate the proposed models. Compared with other models, a dynamic regression model with additive trend, seasonality, interventions, and a very important economic indicator effectively predicted the turnover with training R²?=?0.77 and holdout R²?=?0.59. The forecasting performance of optimal models confirms that time series modelling approach has the ability to predict employee turnover for the specific scenario observed in our analysis.     

A second order model for cylindrical data

Modeling cylindrical data, comprised of a linear component and a directional component, can be done using Fourier series expansions if we consider the conditional distribution of the linear component given the angular component. This paper presents the second order model which is a natural extension of the Mardia and Sutton (1978) first order model. This model can be parameterized either in polar or Cartesian coordinates, and allows for parameter estimation using standard multiple linear regression. Characteristic of the new model, how to compare the adequacy of the fit for first and second order models, and an example involving wind direction and temperature are presented.     

Estimated Generalized Least Squares for Random Coefficient Regression Models

ABSTRACT. This paper considers a general class of random coefficient regression (RCR) models to represent pooled cross-sectional and time series data. A new method is given to estimate the covariance matrix of the error component in these RCR models. Also, the asymptotic and small sample properties of the estimated generalized least squares estimator of the regression coefficient vector are established. Procedures for testing a linear restriction on the mean vector of the random coefficients are derived. Finally, a test for non-randomness in the RCR model is devised, and the asymptotic distribution of the test statistic is obtained.