Analytical uses of Kalman filtering in econometrics — A survey

This paper surveys the different uses of Kalman filtering in the estimation of statistical (econometric) models. The Kalman filter will be portrayed as (i) a natural generalization of exponential smoothing with a time-dependent smoothing factor, (ii) a recursive estimation technique for a variety of econometric models amenable to a state space formulation in particular for econometric models with time varying coefficients (iii) an instrument for the recursive calculation of the likelihood of the (constant) state space coefficients (iv) a means of helping to implement the scoring⁻ and EM-method for iteratively maximizing this likelihood (v) an analytical tool in asymptotic estimation theory. The concluding section points to the importance of Kalman filtering for alternatives to maximum⁻ likelihood estimation of state space parameters.     

Bandwidth selection in pre-smoothed particle filters

For the purpose of maximum likelihood estimation of static parameters, we apply a kernel smoother to the particles in the standard SIR filter for non-linear state space models with additive Gaussian observation noise. This reduces the Monte Carlo error in the estimates of both the posterior density of the states and the marginal density of the observation at each time point. We correct for variance inflation in the smoother, which together with the use of Gaussian kernels, results in a Gaussian (Kalman) update when the amount of smoothing turns to infinity. We propose and study of a criterion for choosing the optimal bandwidth h in the kernel smoother. Finally, we illustrate our approach using examples from econometrics. Our filter is shown to be highly suited for dynamic models with high signal-to-noise ratio, for which the SIR filter has problems.     

Nonparametric smoothing using state space techniques

The authors examine the equivalence between penalized least squares and state space smoothing using random vectors with infinite variance. They show that despite infinite variance, many time series techniques for estimation, significance testing, and diagnostics can be used. The Kalman filter can be used to fit penalized least squares models, computing the smoothed quantities and related values. Infinite variance is equivalent to differencing to stationarity, and to adding explanatory variables. The authors examine constructs called “smoothations” which they show to be fundamental in smoothing. Applications illustrate concepts and methods.     

Reml and best linear unbiased prediction in state space models

This article takes a hierarchical model approach to the estimation of state space models with diffuse initial conditions. An initial state is said to be diffuse when it cannot be assigned a proper prior distribution. In state space models this occurs either when fixed effects are present or when modelling nonstationarity in the state transition equation. Whereas much of the literature views diffuse states as an initialization problem, we follow the approach of Sallas and Harville (1981,1988) and incorporate diffuse initial conditions via noninformative prior distributions into hierarchical linear models. We apply existing results to derive the restricted loglike-lihood and appropriate modifications to the standard Kalman filter and smoother. Our approach results in a better understanding of De Jong's (1991) contributions. This article also shows how to adjust the standard Kalman filter, the fixed inter- val smoother and the state space model forecasting recursions, together with their mean square errors, for he presence of diffuse components. Using a hierarchical model approach it is shown that the estimates obtained are Best Linear Unbiased Predictors (BLUP).     

An alternative derivation of the Kalman filter using the quasi-likelihood method

The Kalman filter gives a recursive procedure for estimating state vectors. The recursive procedure is determined by a matrix, so-called gain matrix, where the gain matrix is varied based on the system to which the Kalman filter is applied. Traditionally the gain matrix is derived through the maximum likelihood approach when the probability structure of underlying system is known. As an alternative approach, the quasi-likelihood method is considered in this paper. This method is used to derive the gain matrix without the full knowledge of the probability structure of the underlying system. Two models are considered in this paper, the simple state space model and the model with correlated between measurement and transition equation disturbances. The purposes of this paper are (i) to show a simple way to derive the gain matrix; (ii) to give an alternative approach for obtaining optimal estimation of state vector when underlying system is relatively complex.     

Deterministic approximate inference techniques for conditionally Gaussian state space models

We describe a novel deterministic approximate inference technique for conditionally Gaussian state space models, i.e. state space models where the latent state consists of both multinomial and Gaussian distributed variables. The method can be interpreted as a smoothing pass and iteration scheme symmetric to an assumed density filter. It improves upon previously proposed smoothing passes by not making more approximations than implied by the projection onto the chosen parametric form, the assumed density. Experimental results show that the novel scheme outperforms these alternative deterministic smoothing passes. Comparisons with sampling methods suggest that the performance does not degrade with longer sequences.     

Conditional Likelihood Estimators for Hidden Markov Models and Stochastic Volatility Models

ABSTRACT.  This paper develops a new contrast process for parametric inference of general hidden Markov models, when the hidden chain has a non-compact state space. This contrast is based on the conditional likelihood approach, often used for ARCH-type models. We prove the strong consistency of the conditional likelihood estimators under appropriate conditions. The method is applied to the Kalman filter (for which this contrast and the exact likelihood lead to asymptotically equivalent estimators) and to the discretely observed stochastic volatility models.     

Using recursive algorithms for the efficient identification of smoothing spline ANOVA models

In this paper we present a unified discussion of different approaches to the identification of smoothing spline analysis of variance (ANOVA) models: (i) the “classical” approach (in the line of Wahba in Spline Models for Observational Data, 1990; Gu in Smoothing Spline ANOVA Models, 2002; Storlie et al. in Stat. Sin., 2011) and (ii) the State-Dependent Regression (SDR) approach of Young in Nonlinear Dynamics and Statistics (2001). The latter is a nonparametric approach which is very similar to smoothing splines and kernel regression methods, but based on recursive filtering and smoothing estimation (the Kalman filter combined with fixed interval smoothing). We will show that SDR can be effectively combined with the “classical” approach to obtain a more accurate and efficient estimation of smoothing spline ANOVA models to be applied for emulation purposes. We will also show that such an approach can compare favorably with kriging.     

Robust estimation of linear state space models

The model parameters of linear state space models are typically estimated with maximum likelihood estimation, where the likelihood is computed analytically with the Kalman filter. Outliers can deteriorate the estimation. Therefore we propose an alternative estimation method. The Kalman filter is replaced by a robust version and the maximum likelihood estimator is robustified as well. The performance of the robust estimator is investigated in a simulation study. Robust estimation of time varying parameter regression models is considered as a special case. Finally, the methodology is applied to real data.     

The reverse kalman filter

Equations for the reverse Kalman filter are provided. This enables calculation of the distribution of the state given the future data in the state space representation of a time series model. This complements the conventional Kalman filter recursions for the distribution of the state given the past data. The usefulness of these two recursions is demonstrated by showing how they can be combined to efficiently calculate leave-k-out diagnostics or the likelihood under a model incorporating a patch of anomalies in a time series.     

Stationary state space models for longitudinal data

The authors consider a class of state space models for the analysis of non‐normal longitudinal data whose latent process follows a stationary AR(1) model with exponential dispersion model margins. They propose to estimate parameters through an estimating equation approach based on the Kalman smoother. This allows them to carry out a straightforward analysis of a wide range of non‐normal data. They illustrate their approach via a simulation study and through analyses of Brazilian precipitation and US polio infection data.     

A UNIFIED FRAMEWORK FOR MODELLING WILDLIFE POPULATION DYNAMICS†

This paper proposes a unified framework for defining and fitting stochastic, discrete‐time, discrete‐stage population dynamics models. The biological system is described by a state‐space model, where the true but unknown state of the population is modelled by a state process, and this is linked to survey data by an observation process. All sources of uncertainty in the inputs, including uncertainty about model specification, are readily incorporated. The paper shows how the state process can be represented as a generalization of the standard Leslie or Lefkovitch matrix. By dividing the state process into subprocesses, complex models can be constructed from manageable building blocks. The paper illustrates the approach with a model of the British grey seal metapopulation, using sequential importance sampling with kernel smoothing to fit the model.     

Inequality Constrained State-Space Models

ABSTRACT

The standard Kalman filter cannot handle inequality constraints imposed on the state variables, as state truncation induces a nonlinear and non-Gaussian model. We propose a Rao-Blackwellized particle filter with the optimal importance function for forward filtering and the likelihood function evaluation. The particle filter effectively enforces the state constraints when the Kalman filter violates them. Monte Carlo experiments demonstrate excellent performance of the proposed particle filter with Rao-Blackwellization, in which the Gaussian linear sub-structure is exploited at both the cross-sectional and temporal levels.     

EXPONENTIAL SMOOTHING AND NON‐NEGATIVE DATA

The most common forecasting methods in business are based on exponential smoothing, and the most common time series in business are inherently non‐negative. Therefore it is of interest to consider the properties of the potential stochastic models underlying exponential smoothing when applied to non‐negative data. We explore exponential smoothing state space models for non‐negative data under various assumptions about the innovations, or error, process. We first demonstrate that prediction distributions from some commonly used state space models may have an infinite variance beyond a certain forecasting horizon. For multiplicative error models that do not have this flaw, we show that sample paths will converge almost surely to zero even when the error distribution is non‐Gaussian. We propose a new model with similar properties to exponential smoothing, but which does not have these problems, and we develop some distributional properties for our new model. We then explore the implications of our results for inference, and compare the short‐term forecasting performance of the various models using data on the weekly sales of over 300 items of costume jewelry. The main findings of the research are that the Gaussian approximation is adequate for estimation and one‐step‐ahead forecasting. However, as the forecasting horizon increases, the approximate prediction intervals become increasingly problematic. When the model is to be used for simulation purposes, a suitably specified scheme must be employed.     

The ensemble Kalman filter is an ABC algorithm

The ensemble Kalman filter is the method of choice for many difficult high-dimensional filtering problems in meteorology, oceanography, hydrology and other fields. In this note we show that a common variant of the ensemble Kalman filter is an approximate Bayesian computation (ABC) algorithm. This is of interest for a number of reasons. First, the ensemble Kalman filter is an example of an ABC algorithm that predates the development of ABC algorithms. Second, the ensemble Kalman filter is used for very high-dimensional problems, whereas ABC methods are normally applied only in very low-dimensional problems. Third, recent state of the art extensions of the ensemble Kalman filter can also be understood within the ABC framework.     

A family of models for uniform and serial dependence in repeated measurements studies

Data arising from a randomized double-masked clinical trial for multiple sclerosis have provided particularly variable longitudinal repeated measurements responses. Specific models for such data, other than those based on the multivariate normal distribution, would be a valuable addition to the applied statistician's toolbox. A useful family of multivariate distributions can be generated by substituting the integrated intensity of one distribution into a second (outer) distribution. The parameters in the second distribution are then used to create a dependence structure among observations on a unit. These may either be a form of serial dependence for longitudinal data or of uniform dependence within clusters. These are respectively analogous to the Kalman filter of state space models and to copulas, but they have the major advantage that they do not require any explicit integration. One useful outer distribution for constructing such multivariate distributions is the Pareto distribution. Certain special models based on it have previously been used in event history analysis, but those considered here have much wider application.     

Nonlinear functional models for functional responses in reproducing kernel hilbert spaces

The author proposes an extension of reproducing kernel Hilbert space theory which provides a new framework for analyzing functional responses with regression models. The approach only presumes a general nonlinear regression structure, as opposed to existing linear regression models. The author proposes generalized cross‐validation for automatic smoothing parameter estimation. He illustrates the use of the new estimator both on real and simulated data.     

Dynamic changepoint detection in count time series: a particle filter approach

We study Bayesian dynamic models for detecting changepoints in count time series that present structural breaks. As the inferential approach, we develop a parameter learning version of the algorithm proposed by Chopin [Chopin N. Dynamic detection of changepoints in long time series. Annals of the Institute of Statistical Mathematics 2007;59:349–366.], called the Chopin filter with parameter learning, which allows us to estimate the static parameters in the model. In this extension, the static parameters are addressed by using the kernel smoothing approximations proposed by Liu and West [Liu J, West M. Combined parameters and state estimation in simulation-based filtering. In: Doucet A, de Freitas N, Gordon N, editors. Sequential Monte Carlo methods in practice. New York: Springer-Verlag; 2001]. The proposed methodology is then applied to both simulated and real data sets and the time series models include distributions that allow for overdispersion and/or zero inflation. Since our procedure is general, robust and naturally adaptive because the particle filter approach does not require restrictive specifications to ensure its validity and effectiveness, we believe it is a valuable alternative for dealing with the problem of detecting changepoints in count time series. The proposed methodology is also suitable for count time series with no changepoints and for independent count data.     

A Class of Non-Gaussian State Space Models With Exact Likelihood Inference

The likelihood function of a general nonlinear, non-Gaussian state space model is a high-dimensional integral with no closed-form solution. In this article, I show how to calculate the likelihood function exactly for a large class of non-Gaussian state space models that include stochastic intensity, stochastic volatility, and stochastic duration models among others. The state variables in this class follow a nonnegative stochastic process that is popular in econometrics for modeling volatility and intensities. In addition to calculating the likelihood, I also show how to perform filtering and smoothing to estimate the latent variables in the model. The procedures in this article can be used for either Bayesian or frequentist estimation of the model’s unknown parameters as well as the latent state variables. Supplementary materials for this article are available online.     

LOCAL LINEAR FORECASTS USING CUBIC SMOOTHING SPLINES

This paper shows how cubic smoothing splines fitted to univariate time series data can be used to obtain local linear forecasts. The approach is based on a stochastic state‐space model which allows the use of likelihoods for estimating the smoothing parameter, and which enables easy construction of prediction intervals. The paper shows that the model is a special case of an ARIMA(0, 2, 2) model; it provides a simple upper bound for the smoothing parameter to ensure an invertible model; and it demonstrates that the spline model is not a special case of Holt's local linear trend method. The paper compares the spline forecasts with Holt's forecasts and those obtained from the full ARIMA(0, 2, 2) model, showing that the restricted parameter space does not impair forecast performance. The advantage of this approach over a full ARIMA(0, 2, 2) model is that it gives a smooth trend estimate as well as a linear forecast function.