Bandwidth selection in pre-smoothed particle filters

For the purpose of maximum likelihood estimation of static parameters, we apply a kernel smoother to the particles in the standard SIR filter for non-linear state space models with additive Gaussian observation noise. This reduces the Monte Carlo error in the estimates of both the posterior density of the states and the marginal density of the observation at each time point. We correct for variance inflation in the smoother, which together with the use of Gaussian kernels, results in a Gaussian (Kalman) update when the amount of smoothing turns to infinity. We propose and study of a criterion for choosing the optimal bandwidth h in the kernel smoother. Finally, we illustrate our approach using examples from econometrics. Our filter is shown to be highly suited for dynamic models with high signal-to-noise ratio, for which the SIR filter has problems.     

Monte Carlo Kalman filter and smoothing for multivariate discrete state space models

The author studies state space models for multivariate binomial time series, focussing on the development of the Kalman filter and smoothing for state variables. He proposes a Monte Carlo approach employing the latent variable representation which transplants the classical Kalman filter and smoothing developed for Gaussian state space models to discrete models and leads to a conceptually simple and computationally convenient approach. The method is illustrated through simulations and concrete examples.     

An alternative derivation of the Kalman filter using the quasi-likelihood method

The Kalman filter gives a recursive procedure for estimating state vectors. The recursive procedure is determined by a matrix, so-called gain matrix, where the gain matrix is varied based on the system to which the Kalman filter is applied. Traditionally the gain matrix is derived through the maximum likelihood approach when the probability structure of underlying system is known. As an alternative approach, the quasi-likelihood method is considered in this paper. This method is used to derive the gain matrix without the full knowledge of the probability structure of the underlying system. Two models are considered in this paper, the simple state space model and the model with correlated between measurement and transition equation disturbances. The purposes of this paper are (i) to show a simple way to derive the gain matrix; (ii) to give an alternative approach for obtaining optimal estimation of state vector when underlying system is relatively complex.     

Stationary state space models for longitudinal data

The authors consider a class of state space models for the analysis of non‐normal longitudinal data whose latent process follows a stationary AR(1) model with exponential dispersion model margins. They propose to estimate parameters through an estimating equation approach based on the Kalman smoother. This allows them to carry out a straightforward analysis of a wide range of non‐normal data. They illustrate their approach via a simulation study and through analyses of Brazilian precipitation and US polio infection data.     

Conditional Likelihood Estimators for Hidden Markov Models and Stochastic Volatility Models

ABSTRACT.  This paper develops a new contrast process for parametric inference of general hidden Markov models, when the hidden chain has a non-compact state space. This contrast is based on the conditional likelihood approach, often used for ARCH-type models. We prove the strong consistency of the conditional likelihood estimators under appropriate conditions. The method is applied to the Kalman filter (for which this contrast and the exact likelihood lead to asymptotically equivalent estimators) and to the discretely observed stochastic volatility models.     

Choosing Prior Hyperparameters: With Applications to Time-Varying Parameter Models

Time-varying parameter models with stochastic volatility are widely used to study macroeconomic and financial data. These models are almost exclusively estimated using Bayesian methods. A common practice is to focus on prior distributions that themselves depend on relatively few hyperparameters such as the scaling factor for the prior covariance matrix of the residuals governing time variation in the parameters. The choice of these hyperparameters is crucial because their influence is sizeable for standard sample sizes. In this article, we treat the hyperparameters as part of a hierarchical model and propose a fast, tractable, easy-to-implement, and fully Bayesian approach to estimate those hyperparameters jointly with all other parameters in the model. We show via Monte Carlo simulations that, in this class of models, our approach can drastically improve on using fixed hyperparameters previously proposed in the literature. Supplementary materials for this article are available online.     

Inference for a Class of Stochastic Volatility Models Using Option and Spot Prices: Application of a Bivariate Kalman Filter

In this paper Bayesian methods are applied to a stochastic volatility model using both the prices of the asset and the prices of options written on the asset. Posterior densities for all model parameters, latent volatilities and the market price of volatility risk are produced via a Markov Chain Monte Carlo (MCMC) sampling algorithm. Candidate draws for the unobserved volatilities are obtained in blocks by applying the Kalman filter and simulation smoother to a linearization of a nonlinear state space representation of the model. Crucially, information from both the spot and option prices affects the draws via the specification of a bivariate measurement equation, with implied Black–Scholes volatilities used to proxy observed option prices in the candidate model. Alternative models nested within the Heston (1993) framework are ranked via posterior odds ratios, as well as via fit, predictive and hedging performance. The method is illustrated using Australian News Corporation spot and option price data.     

Inference for a Class of Stochastic Volatility Models Using Option and Spot Prices: Application of a Bivariate Kalman Filter

In this paper Bayesian methods are applied to a stochastic volatility model using both the prices of the asset and the prices of options written on the asset. Posterior densities for all model parameters, latent volatilities and the market price of volatility risk are produced via a Markov Chain Monte Carlo (MCMC) sampling algorithm. Candidate draws for the unobserved volatilities are obtained in blocks by applying the Kalman filter and simulation smoother to a linearization of a nonlinear state space representation of the model. Crucially, information from both the spot and option prices affects the draws via the specification of a bivariate measurement equation, with implied Black-Scholes volatilities used to proxy observed option prices in the candidate model. Alternative models nested within the Heston (1993) framework are ranked via posterior odds ratios, as well as via fit, predictive and hedging performance. The method is illustrated using Australian News Corporation spot and option price data.     

Estimation and tests of hypotheses for the initial mean and covariance in the kalman filter model

Kalman filtering techniques are widely used by engineers to recursively estimate random signal parameters which are essentially coefficients in a large-scale time series regression model. These Bayesian estimators depend on the values assumed for the mean and covariance parameters associated with the initial state of the random signal. This paper considers a likelihood approach to estimation and tests of hypotheses involving the critical initial means and covariances. A computationally simple convergent iterative algorithm is used to generate estimators which depend only on standard Kalman filter outputs at each successive stage. Conditions are given under which the maximum likelihood estimators are consistent and asymptotically normal. The procedure is illustrated using a typical large-scale data set involving 10-dimensional signal vectors.     

Inequality Constrained State-Space Models

ABSTRACT

The standard Kalman filter cannot handle inequality constraints imposed on the state variables, as state truncation induces a nonlinear and non-Gaussian model. We propose a Rao-Blackwellized particle filter with the optimal importance function for forward filtering and the likelihood function evaluation. The particle filter effectively enforces the state constraints when the Kalman filter violates them. Monte Carlo experiments demonstrate excellent performance of the proposed particle filter with Rao-Blackwellization, in which the Gaussian linear sub-structure is exploited at both the cross-sectional and temporal levels.     

Robust estimation of linear state space models

The model parameters of linear state space models are typically estimated with maximum likelihood estimation, where the likelihood is computed analytically with the Kalman filter. Outliers can deteriorate the estimation. Therefore we propose an alternative estimation method. The Kalman filter is replaced by a robust version and the maximum likelihood estimator is robustified as well. The performance of the robust estimator is investigated in a simulation study. Robust estimation of time varying parameter regression models is considered as a special case. Finally, the methodology is applied to real data.     

Analytical uses of Kalman filtering in econometrics — A survey

This paper surveys the different uses of Kalman filtering in the estimation of statistical (econometric) models. The Kalman filter will be portrayed as (i) a natural generalization of exponential smoothing with a time-dependent smoothing factor, (ii) a recursive estimation technique for a variety of econometric models amenable to a state space formulation in particular for econometric models with time varying coefficients (iii) an instrument for the recursive calculation of the likelihood of the (constant) state space coefficients (iv) a means of helping to implement the scoring⁻ and EM-method for iteratively maximizing this likelihood (v) an analytical tool in asymptotic estimation theory. The concluding section points to the importance of Kalman filtering for alternatives to maximum⁻ likelihood estimation of state space parameters.     

An Introduction to Modern Matrix Methods and Statistics

In this article, using the representation that the Kalman filter recursions in state-space models can be expressed as a matrix-weighted average of prior and sample estimates, we supplement the usual filtering algorithm by an extreme bounds analysis. Specifically, as the covariance matrix of the state error is varied in the class of symmetric and positive-definite matrices, the filtering estimates are shown to be in an ellipsoid.     

The reverse kalman filter

Equations for the reverse Kalman filter are provided. This enables calculation of the distribution of the state given the future data in the state space representation of a time series model. This complements the conventional Kalman filter recursions for the distribution of the state given the past data. The usefulness of these two recursions is demonstrated by showing how they can be combined to efficiently calculate leave-k-out diagnostics or the likelihood under a model incorporating a patch of anomalies in a time series.     

The Kalman Filter from the Perspective of Goldberger—Theil Estimators

This article is designed to point out the close connection between recursive estimation procedures, such as Kalman filter theory, familiar to control engineers, and linear least squares estimators and estimators that include prior information in the form of linear restrictions, such as mixed estimators and ridge estimators, familiar to statisticians. The only difference between the two points of view seems to be a difference in terminology. To demonstrate this point, it is shown how the Kalman filter equations can be derived from an existing textbook account of linear least squares theory and the notion of combining prior information in linear models, that is, the Goldberger—Theil mixed estimators' point of view. The author advocates the inclusion of these ideas early when least squares estimation concepts are being taught.     

Combining Bayesian method and Kalman smoother for detection additive outlier patches in autoregressive time series

ABSTRACT

This article proposes a development of detecting patches of additive outliers in autoregressive time series models. The procedure improves the existing detection methods via Gibbs sampling. We combine the Bayesian method and the Kalman smoother to present some candidate models of outlier patches and the best model with the minimum Bayesian information criterion (BIC) is selected among them. We propose that this combined Bayesian and Kalman method (CBK) can reduce the masking and swamping effects about detecting patches of additive outliers. The correctness of the method is illustrated by simulated data and then by analyzing a real set of observations.     

The mode oriented stochastic search (MOSS) algorithm for log-linear models with conjugate priors

We describe a novel stochastic search algorithm for rapidly identifying regions of high posterior probability in the space of decomposable, graphical and hierarchical log-linear models. Our approach is based on the Diaconis–Ylvisaker conjugate prior for log-linear parameters. We discuss the computation of Bayes factors through Laplace approximations and the Bayesian iterative proportional fitting algorithm for sampling model parameters. We use our model determination approach in a sparse eight-way contingency table.     

On spatiotemporal prediction for on-line monitoring data

Spatiotemporal prediction is of interest in many areas of applied statistics, especially in environmental monitoring with on-line data information. At first, this article reviews the approaches for spatiotemporal modeling in the context of stochastic processes and then introduces the new class of spatiotemporal dynamic linear models. Further, the methods for linear spatial data analysis, universal kriging and trend surface prediction, are related to the method of spatial linear Bayesian analysis. The Kalman filter is the preferred method for temporal linear Bayesian inferences. By combining the Kalman filter recursions with the trend surface predictor and universal kriging predictor, the prior and posterior spatiotemporal predictors for the observational process are derived, which form the main result of this article. The problem of spatiotemporal linear prediction in the case of unknown first and second order moments is treated as well.     

Dynamic generalized linear models with application to environmental epidemiology

Summary. We propose modelling short-term pollutant exposure effects on health by using dynamic generalized linear models. The time series of count data are modelled by a Poisson distribution having mean driven by a latent Markov process; estimation is performed by the extended Kalman filter and smoother. This modelling strategy allows us to take into account possible overdispersion and time-varying effects of the covariates. These ideas are illustrated by reanalysing data on the relationship between daily non-accidental deaths and air pollution in the city of Birmingham, Alabama.     

Emulator-assisted reduced-rank ecological data assimilation for nonlinear multivariate dynamical spatio-temporal processes

As ecological data sets increase in spatial and temporal extent with the advent of new remote sensing platforms and long-term monitoring networks, there is increasing interest in forecasting ecological processes. Such forecasts require realistic initial conditions over complete spatial domains. Typically, data sources are incomplete in space, and the processes include complicated dynamical interactions across physical and biological variables. This suggests that data assimilation, whereby observations are fused with mechanistic models, is the most appropriate means of generating complete initial conditions. Often, the mechanistic models used for these procedures are very expensive computationally. We demonstrate a rank-reduced approach for ecological data assimilation whereby the mechanistic model is based on a statistical emulator. Critically, the rank-reduction and emulator construction are linked and, by utilizing a hierarchical framework, uncertainty associated with the dynamical emulator can be accounted for. This provides a so-called “weak-constraint” data assimilation procedure. This approach is demonstrated on a high-dimensional multivariate coupled biogeochemical ocean process.