Uncertainty Quantification in the Ensemble Kalman Filter

The ensemble Kalman filter (EnKF) provides an approximate, sequential Monte Carlo solution to the recursive data assimilation algorithm for hidden Markov chains. The challenging conditioning step is approximated by a linear updating, and the updating weights, termed Kalman weights, are inferred from the ensemble members. The EnKF scheme is known to provide unstable predictions and to underestimate the prediction intervals, and even sometimes to diverge. The underlying cause for these shortcomings is poorly understood. We find that the ensemble members couple in the conditioning procedure and that the coupling increase multiplicatively in the recursive conditioning steps. Under reasonable Gauss‐independence assumptions, exact expressions for this correlation are developed. Moreover, expressions for the precision of the predictions and the downward bias in the empirical variance introduced in one conditioning step are found. These results are confirmed by a Gauss‐linear simulation study. Furthermore, we quantitatively evaluate an alternative, improved EnKF scheme on the basis of transformations of ensemble members under the same Gauss‐independent assumptions. The scheme is compared with the frequently used ensemble inflation scheme.     

Parameter Estimation for First-Order Random Coefficient Autoregressive (RCA) Models Based on Kalman Filter

In this article, we propose a new estimate algorithm for the parameters of a first-order Random Coefficient Autoregressive (RCA) Model. This algorithm turns out to be very reliable in estimating the true parameter values of a given model. It combines quasi-maximum likelihood method, the Kalman filter algorithm, and the Powell's method. Simulation results demonstrate that the algorithm is viable and promising.     

On-line inference for hidden Markov models via particle filters

Summary.  We consider the on-line Bayesian analysis of data by using a hidden Markov model, where inference is tractable conditional on the history of the state of the hidden component. A new particle filter algorithm is introduced and shown to produce promising results when analysing data of this type. The algorithm is similar to the mixture Kalman filter but uses a different resampling algorithm. We prove that this resampling algorithm is computationally efficient and optimal, among unbiased resampling algorithms, in terms of minimizing a squared error loss function. In a practical example, that of estimating break points from well-log data, our new particle filter outperforms two other particle filters, one of which is the mixture Kalman filter, by between one and two orders of magnitude.     

Filtering via approximate Bayesian computation

Approximate Bayesian computation (ABC) has become a popular technique to facilitate Bayesian inference from complex models. In this article we present an ABC approximation designed to perform biased filtering for a Hidden Markov Model when the likelihood function is intractable. We use a sequential Monte Carlo (SMC) algorithm to both fit and sample from our ABC approximation of the target probability density. This approach is shown to, empirically, be more accurate w.r.t.?the original filter than competing methods. The theoretical bias of our method is investigated; it is shown that the bias goes to zero at the expense of increased computational effort. Our approach is illustrated on a constrained sequential lasso for portfolio allocation to 15 constituents of the FTSE 100 share index.     

Mixture Kalman filters

In treating dynamic systems, sequential Monte Carlo methods use discrete samples to represent a complicated probability distribution and use rejection sampling, importance sampling and weighted resampling to complete the on-line 'filtering' task. We propose a special sequential Monte Carlo method, the mixture Kalman filter, which uses a random mixture of the Gaussian distributions to approximate a target distribution. It is designed for on-line estimation and prediction of conditional and partial conditional dynamic linear models, which are themselves a class of widely used non-linear systems and also serve to approximate many others. Compared with a few available filtering methods including Monte Carlo methods, the gain in efficiency that is provided by the mixture Kalman filter can be very substantial. Another contribution of the paper is the formulation of many non-linear systems into conditional or partial conditional linear form, to which the mixture Kalman filter can be applied. Examples in target tracking and digital communications are given to demonstrate the procedures proposed.     

Efficient likelihood-free Bayesian Computation for?household epidemics

Considerable progress has been made in applying Markov chain Monte Carlo (MCMC) methods to the analysis of epidemic data. However, this likelihood based method can be inefficient due to the limited data available concerning an epidemic outbreak. This paper considers an alternative approach to studying epidemic data using Approximate Bayesian Computation (ABC) methodology. ABC is a simulation-based technique for obtaining an approximate sample from the posterior distribution of the parameters of the model and in an epidemic context is very easy to implement. A new approach to ABC is introduced which generates a set of values from the (approximate) posterior distribution of the parameters during each simulation rather than a single value. This is based upon coupling simulations with different sets of parameters and we call the resulting algorithm coupled ABC. The new methodology is used to analyse final size data for epidemics amongst communities partitioned into households. It is shown that for the epidemic data sets coupled ABC is more efficient than ABC and MCMC-ABC.     

An Introduction to Modern Matrix Methods and Statistics

In this article, using the representation that the Kalman filter recursions in state-space models can be expressed as a matrix-weighted average of prior and sample estimates, we supplement the usual filtering algorithm by an extreme bounds analysis. Specifically, as the covariance matrix of the state error is varied in the class of symmetric and positive-definite matrices, the filtering estimates are shown to be in an ellipsoid.     

Adaptive Kalman Filtering with Multivariate Generalized Laplace System Noise

An adaptive Kalman filter is proposed to estimate the states of a system where the system noise is assumed to be a multivariate generalized Laplace random vector. In the presence of outliers in the system noise, it is shown that improved state estimates can be obtained by using an adaptive factor to estimate the dispersion matrix of the system noise term. For the implementation of the filter, an algorithm which includes both single and multiple adaptive factors is proposed. A Monte-Carlo investigation is also carried out to access the performance of the proposed filters in comparison with other robust filters. The results show that, in the sense of minimum mean squared state error, the proposed filter is superior to other filters when the magnitude of a system change is moderate or large.     

Image reconstruction by adaptive bayesian classification with a locally proportional prior

This paper concerns the problem of reconstructing images from noisy data by means of Bayesian classification methods. In Klein and Press, 1992, the authors presented a method for reconstructing images called Adaptive Bayesian Classification (ABC). The ABC procedure was shown to preform very well in simulation experiments. The ABC procedure was multistaged; moreover, it involved selecting a prior at Stage n that was the posterior at Stage n - 1. In this paper the authors show that we can improve upon ABC for some problems by modifying the way we take the prior at each stage. The new proposal is to take the prior for the pixel label at each stage as proportional to the number of pixels with that label in a small neighborhood of the pixel. The ABC procedure with a locally proportional prior (ABC/LPP) tends to improve upon the ABC procedure for some problems because the prior in the iterative portion of ABC/LPP is contextual, while that in ABC in non- contextual.     

Understanding the Kalman Filter

This is an expository article. Here we show how the successfully used Kalman filter, popular with control engineers and other scientists, can be easily understood by statisticians if we use a Bayesian formulation and some well-known results in multivariate statistics. We also give a simple example illustrating the use of the Kalman filter for quality control work.     

A kalman filter in the presence of outliers

A Kalman Filtering algorithm which is robust to observational outliers is developed by assuming that the measurement error may come from either one of two Normal distributions, and that the transition between these distributions is governed by a Markov Chain. The resulting algorithm is very simple, and consists of two parallel Kalman Filters having different gains. The state estimate is obtained as a weighted average of the estimates from the two parallel filters, where the weights are the posterior probabilities that the current observation comes from either of the two distributions. The large improvements obtained by this Robust Kalman Filter in the presence of outliers is demonstrated with examples.     

Nonlinear filters based on taylor series expansions ∗

The nonlinear filters based on Taylor series approximation are broadly used for computational simplicity, even though their filtering estimates are clearly biased. In this paper, first, we analyze what is approximated when we apply the expanded nonlinear functions to the standard linear recursive Kalman filter algorithm. Next, since the state variable α_t and α_t-t are approximated as a conditional normal distribution given information up to time t - 1 (i.e., I_t-1) in approximation of the Taylor series expansion, it might be appropriate to evaluate each expectation by generating normal random numbers of α_t and α_t-1 given I_t-1 and those of the error terms θ and η_t. Thus, we propose the Monte-Carlo simulation filter using normal random draws. Finally we perform two Monte-Carlo experiments, where we obtain the result that the Monte-Carlo simulation filter has a superior performance over the nonlinear filters such as the extended Kalman filter and the second-order nonlinear filter.     

Estimation and tests of hypotheses for the initial mean and covariance in the kalman filter model

Kalman filtering techniques are widely used by engineers to recursively estimate random signal parameters which are essentially coefficients in a large-scale time series regression model. These Bayesian estimators depend on the values assumed for the mean and covariance parameters associated with the initial state of the random signal. This paper considers a likelihood approach to estimation and tests of hypotheses involving the critical initial means and covariances. A computationally simple convergent iterative algorithm is used to generate estimators which depend only on standard Kalman filter outputs at each successive stage. Conditions are given under which the maximum likelihood estimators are consistent and asymptotically normal. The procedure is illustrated using a typical large-scale data set involving 10-dimensional signal vectors.     

Recursive estimation of bilinear time series models

In this paper we are concerned with the recursive estimation of bilinear models. Some methods from linear time invariant systems are adapted to suit bilinear time series models. The time-varying Kalman filter and associated parameter estimation algorithm is carried on the bilinear time series models. The methods are illustrated with examples.     

Inequality Constrained State-Space Models

ABSTRACT

The standard Kalman filter cannot handle inequality constraints imposed on the state variables, as state truncation induces a nonlinear and non-Gaussian model. We propose a Rao-Blackwellized particle filter with the optimal importance function for forward filtering and the likelihood function evaluation. The particle filter effectively enforces the state constraints when the Kalman filter violates them. Monte Carlo experiments demonstrate excellent performance of the proposed particle filter with Rao-Blackwellization, in which the Gaussian linear sub-structure is exploited at both the cross-sectional and temporal levels.     

An adaptive sequential Monte Carlo method for approximate Bayesian computation

Approximate Bayesian computation (ABC) is a popular approach to address inference problems where the likelihood function is intractable, or expensive to calculate. To improve over Markov chain Monte Carlo (MCMC) implementations of ABC, the use of sequential Monte Carlo (SMC) methods has recently been suggested. Most effective SMC algorithms that are currently available for ABC have a computational complexity that is quadratic in the number of Monte Carlo samples (Beaumont et al., Biometrika 86:983?C990, 2009; Peters et al., Technical report, 2008; Toni et al., J.?Roy. Soc. Interface 6:187?C202, 2009) and require the careful choice of simulation parameters. In this article an adaptive SMC algorithm is proposed which admits a computational complexity that is linear in the number of samples and adaptively determines the simulation parameters. We demonstrate our algorithm on a toy example and on a birth-death-mutation model arising in epidemiology.     

Generalized Gauss–Hermite filtering

We consider a generalization of the Gauss–Hermite filter (GHF), where the filter density is represented by a Hermite expansion with leading Gaussian term (GGHF). Thus, the usual GHF is included as a special case. The moment equations for the time update are solved stepwise by Gauss–Hermite integration, and the measurement update is computed by the Bayes formula, again using numerical integration. The performance of the filter is compared numerically with the GHF, the UKF (unscented Kalman filter) and the EKF (extended Kalman filter) and leads to a lower mean squared filter error.     

New sequential Monte Carlo methods for nonlinear dynamic systems

In this paper we present several new sequential Monte Carlo (SMC) algorithms for online estimation (filtering) of nonlinear dynamic systems. SMC has been shown to be a powerful tool for dealing with complex dynamic systems. It sequentially generates Monte Carlo samples from a proposal distribution, adjusted by a set of importance weight with respect to a target distribution, to facilitate statistical inferences on the characteristic (state) of the system. The key to a successful implementation of SMC in complex problems is the design of an efficient proposal distribution from which the Monte Carlo samples are generated. We propose several such proposal distributions that are efficient yet easy to generate samples from. They are efficient because they tend to utilize both the information in the state process and the observations. They are all Gaussian distributions hence are easy to sample from. The central ideas of the conventional nonlinear filters, such as extended Kalman filter, unscented Kalman filter and the Gaussian quadrature filter, are used to construct these proposal distributions. The effectiveness of the proposed algorithms are demonstrated through two applications—real time target tracking and the multiuser parameter tracking in CDMA communication systems.This work was supported in part by the U.S. National Science Foundation (NSF) under grants CCR-9875314, CCR-9980599, DMS-9982846, DMS-0073651 and DMS-0073601.     

ABC Shadow algorithm: a tool for statistical analysis of spatial patterns

This paper presents an original ABC algorithm, ABC Shadow, that can be applied to sample posterior densities that are continuously differentiable. The proposed algorithm solves the main condition to be fulfilled by any ABC algorithm, in order to be useful in practice. This condition requires enough samples in the parameter space region, induced by the observed statistics. The algorithm is tuned on the posterior of a Gaussian model which is entirely known, and then, it is applied for the statistical analysis of several spatial patterns. These patterns are issued or assumed to be outcomes of point processes. The considered models are: Strauss, Candy and area-interaction.