Particle filtering for partially observed Gaussian state space models

Summary. Solving Bayesian estimation problems where the posterior distribution evolves over time through the accumulation of data has many applications for dynamic models. A large number of algorithms based on particle filtering methods, also known as sequential Monte Carlo algorithms, have recently been proposed to solve these problems. We propose a special particle filtering method which uses random mixtures of normal distributions to represent the posterior distributions of partially observed Gaussian state space models. This algorithm is based on a marginalization idea for improving efficiency and can lead to substantial gains over standard algorithms. It differs from previous algorithms which were only applicable to conditionally linear Gaussian state space models. Computer simulations are carried out to evaluate the performance of the proposed algorithm for dynamic tobit and probit models.     

Asymptotic expansions of the non-central distribution of milks1 statistic in the complex gaussian case

In this paper, asymptotic expansions of the non-null distribution of the Wilks' statistic for Manova based on the complex multivariate Gaussian distribution is obtained for the cases (a) when the non-centrality parameter ω_～is constant and (b) when _～Is of the same order as the sample size. No direct method is available for the case (b) and thus the partial differential equation approach considered in this paper Is particularly useful     

An empirical likelihood ratio based goodness-of-fit test for Inverse Gaussian distributions

The Inverse Gaussian (IG) distribution is commonly introduced to model and examine right skewed data having positive support. When applying the IG model, it is critical to develop efficient goodness-of-fit tests. In this article, we propose a new test statistic for examining the IG goodness-of-fit based on approximating parametric likelihood ratios. The parametric likelihood ratio methodology is well-known to provide powerful likelihood ratio tests. In the nonparametric context, the classical empirical likelihood (EL) ratio method is often applied in order to efficiently approximate properties of parametric likelihoods, using an approach based on substituting empirical distribution functions for their population counterparts. The optimal parametric likelihood ratio approach is however based on density functions. We develop and analyze the EL ratio approach based on densities in order to test the IG model fit. We show that the proposed test is an improvement over the entropy-based goodness-of-fit test for IG presented by Mudholkar and Tian (2002). Theoretical support is obtained by proving consistency of the new test and an asymptotic proposition regarding the null distribution of the proposed test statistic. Monte Carlo simulations confirm the powerful properties of the proposed method. Real data examples demonstrate the applicability of the density-based EL ratio goodness-of-fit test for an IG assumption in practice.     

A copula-based Markov chain model for the analysis of binary longitudinal data

A fully parametric first-order autoregressive (AR(1)) model is proposed to analyse binary longitudinal data. By using a discretized version of a copula, the modelling approach allows one to construct separate models for the marginal response and for the dependence between adjacent responses. In particular, the transition model that is focused on discretizes the Gaussian copula in such a way that the marginal is a Bernoulli distribution. A probit link is used to take into account concomitant information in the behaviour of the underlying marginal distribution. Fixed and time-varying covariates can be included in the model. The method is simple and is a natural extension of the AR(1) model for Gaussian series. Since the approach put forward is likelihood-based, it allows interpretations and inferences to be made that are not possible with semi-parametric approaches such as those based on generalized estimating equations. Data from a study designed to reduce the exposure of children to the sun are used to illustrate the methods.     

Empirical Bayes spatial prediction using a Monte Carlo EM algorithm

This paper deals with an empirical Bayes approach for spatial prediction of a Gaussian random field. In fact, we estimate the hyperparameters of the prior distribution by using the maximum likelihood method. In order to maximize the marginal distribution of the data, the EM algorithm is used. Since this algorithm requires the evaluation of analytically intractable and high dimensionally integrals, a Monte Carlo method based on discretizing parameter space, is proposed to estimate the relevant integrals. Then, the approach is illustrated by its application to a spatial data set. Finally, we compare the predictive performance of this approach with the reference prior method.     

A volume based approach to establish B-spline based expressions for density functions and its application to progressive hybrid censoring

We propose an approach to determine the distribution of particular linear combinations of hybrid censored order statistics which is based on the calculation of volumes of polytopes. For this purpose, we establish efficient and compact volume formulas in terms of B-splines. Further, we illustrate our approach for ten different progressive hybrid censoring schemes under an exponential assumption.     

Geostatistical Modelling Using Non‐Gaussian Matérn Fields

This work provides a class of non‐Gaussian spatial Matérn fields which are useful for analysing geostatistical data. The models are constructed as solutions to stochastic partial differential equations driven by generalized hyperbolic noise and are incorporated in a standard geostatistical setting with irregularly spaced observations, measurement errors and covariates. A maximum likelihood estimation technique based on the Monte Carlo expectation‐maximization algorithm is presented, and a Monte Carlo method for spatial prediction is derived. Finally, an application to precipitation data is presented, and the performance of the non‐Gaussian models is compared with standard Gaussian and transformed Gaussian models through cross‐validation.     

An EM-type algorithm for multivariate mixture models

This paper introduces a new approach, based on dependent univariate GLMs, for fitting multivariate mixture models. This approach is a multivariate generalization of the method for univariate mixtures presented by Hinde (1982). Its accuracy and efficiency are compared with direct maximization of the log-likelihood. Using a simulation study, we also compare the efficiency of Monte Carlo and Gaussian quadrature methods for approximating the mixture distribution. The new approach with Gaussian quadrature outperforms the alternative methods considered. The work is motivated by the multivariate mixture models which have been proposed for modelling changes of employment states at an individual level. Similar formulations are of interest for modelling movement between other social and economic states and multivariate mixture models also occur in biostatistics and epidemiology.     

Parametrizing the Kepler exoplanet period-radius distribution with the bivariate normal inverse Gaussian distribution

This paper presents a simple and robust method for obtaining a comprehensive understanding of the joint period and radius distribution in Kepler exoplanets. The proposed method is based on particle swarm optimization and bivariate Normal Inverse Gaussian distribution. Furthermore, in the construction of the probability density function, this study selects planet-host stars with the GK-type. The injecting approach is also employed to solve the survey completeness of sample. The resulting occurrence rate of Earth analogs is 0.025 with a 95% bootstrap confidence interval between 0.023 and 0.032.     

Simultaneous confidence intervals for comparing several inverse Gaussian means under heteroscedasticity

Recently, Zhang [Simultaneous confidence intervals for several inverse Gaussian populations. Stat Probab Lett. 2014;92:125–131] proposed simultaneous pairwise confidence intervals (SPCIs) based on the fiducial generalized pivotal quantity concept to make inferences about the inverse Gaussian means under heteroscedasticity. In this paper, we propose three new methods for constructing SPCIs to make inferences on the means of several inverse Gaussian distributions when scale parameters and sample sizes are unequal. One of the methods results in a set of classic SPCIs (in the sense that it is not simulation-based inference) and the two others are based on a parametric bootstrap approach. The advantages of our proposed methods over Zhang’s (2014) method are: (i) the simulation results show that the coverage probability of the proposed parametric bootstrap approaches is fairly close to the nominal confidence coefficient while the coverage probability of Zhang’s method is smaller than the nominal confidence coefficient when the number of groups and the variance of groups are large and (ii) the proposed set of classic SPCIs is conservative in contrast to Zhang’s method.     

Parameter estimation of complex mixed models based on meta-model approach

Complex biological processes are usually experimented along time among a collection of individuals, longitudinal data are then available. The statistical challenge is to better understand the underlying biological mechanisms. A standard statistical approach is mixed-effects model where the regression function is highly-developed to describe precisely the biological processes (solutions of multi-dimensional ordinary differential equations or of partial differential equation). A classical estimation method relies on coupling a stochastic version of the EM algorithm with a Monte Carlo Markov Chain algorithm. This algorithm requires many evaluations of the regression function. This is clearly prohibitive when the solution is numerically approximated with a time-consuming solver. In this paper a meta-model relying on a Gaussian process emulator is proposed to approximate the regression function, that leads to what is called a mixed meta-model. The uncertainty of the meta-model approximation can be incorporated in the model. A control on the distance between the maximum likelihood estimates of the mixed meta-model and the maximum likelihood estimates of the exact mixed model is guaranteed. Eventually, numerical simulations are performed to illustrate the efficiency of this approach.     

A Two-Stage Estimation Method for Random Coefficient Differential Equation Models with Application to Longitudinal HIV Dynamic Data

We propose a two-stage estimation method for random coefficient ordinary differential equation (ODE) models. A maximum pseudo-likelihood estimator (MPLE) is derived based on a mixed-effects modeling approach and its asymptotic properties for population parameters are established. The proposed method does not require repeatedly solving ODEs, and is computationally efficient although it does pay a price with the loss of some estimation efficiency. However, the method does offer an alternative approach when the exact likelihood approach fails due to model complexity and high-dimensional parameter space, and it can also serve as a method to obtain the starting estimates for more accurate estimation methods. In addition, the proposed method does not need to specify the initial values of state variables and preserves all the advantages of the mixed-effects modeling approach. The finite sample properties of the proposed estimator are studied via Monte Carlo simulations and the methodology is also illustrated with application to an AIDS clinical data set.     

Bayesian estimation on semiparametric models with shape restricted B-splines

In this paper, we develop a Bayesian estimation procedure for semiparametric models under shape constrains. The approach uses a hierarchical Bayes framework and characterizations of shape-constrained B-splines. We employ Markov chain Monte Carlo methods for model fitting, using a truncated normal distribution as the prior for the coefficients of basis functions to ensure the desired shape constraints. The small sample properties of the function estimators are provided via simulation and compared with existing methods. A real data analysis is conducted to illustrate the application of the proposed method.     

Model-based clustering of Gaussian copulas for mixed data

Clustering of mixed data is important yet challenging due to a shortage of conventional distributions for such data. In this article, we propose a mixture model of Gaussian copulas for clustering mixed data. Indeed copulas, and Gaussian copulas in particular, are powerful tools for easily modeling the distribution of multivariate variables. This model clusters data sets with continuous, integer, and ordinal variables (all having a cumulative distribution function) by considering the intra-component dependencies in a similar way to the Gaussian mixture. Indeed, each component of the Gaussian copula mixture produces a correlation coefficient for each pair of variables and its univariate margins follow standard distributions (Gaussian, Poisson, and ordered multinomial) depending on the nature of the variable (continuous, integer, or ordinal). As an interesting by-product, this model generalizes many well-known approaches and provides tools for visualization based on its parameters. The Bayesian inference is achieved with a Metropolis-within-Gibbs sampler. The numerical experiments, on simulated and real data, illustrate the benefits of the proposed model: flexible and meaningful parameterization combined with visualization features.     

A sure independence screening procedure for ultra-high dimensional partially linear additive models

We introduce a two-step procedure, in the context of ultra-high dimensional additive models, which aims to reduce the size of covariates vector and distinguish linear and nonlinear effects among nonzero components. Our proposed screening procedure, in the first step, is constructed based on the concept of cumulative distribution function and conditional expectation of response in the framework of marginal correlation. B-splines and empirical distribution functions are used to estimate the two above measures. The sure screening property of this procedure is also established. In the second step, a double penalization based procedure is applied to identify nonzero and linear components, simultaneously. The performance of the designed method is examined by several test functions to show its capabilities against competitor methods when the distribution of errors is varied. Simulation studies imply that the proposed screening procedure can be applied to the ultra-high dimensional data and well detect the influential covariates. It also demonstrate the superiority in comparison with the existing methods. This method is also applied to identify most influential genes for overexpression of a G protein-coupled receptor in mice.     

An efficient correction to the density-based empirical likelihood ratio goodness-of-fit test for the inverse Gaussian distribution

The inverse Gaussian (IG) distribution is widely used to model positively skewed data. An important issue is to develop a powerful goodness-of-fit test for the IG distribution. We propose and examine novel test statistics for testing the IG goodness of fit based on the density-based empirical likelihood (EL) ratio concept. To construct the test statistics, we use a new approach that employs a method of the minimization of the discrimination information loss estimator to minimize Kullback–Leibler type information. The proposed tests are shown to be consistent against wide classes of alternatives. We show that the density-based EL ratio tests are more powerful than the corresponding classical goodness-of-fit tests. The practical efficiency of the tests is illustrated by using real data examples.     

Poisson-Driven Stationary Markov Models

ABSTRACT

We propose a simple yet powerful method to construct strictly stationary Markovian models with given but arbitrary invariant distributions. The idea is based on a Poisson-type transform modulating the dependence structure in the model. An appealing feature of our approach is the possibility to control the underlying transition probabilities and, therefore, incorporate them within standard estimation methods. Given the resulting representation of the transition density, a Gibbs sampler algorithm based on the slice method is proposed and implemented. In the discrete-time case, special attention is placed to the class of generalized inverse Gaussian distributions. In the continuous case, we first provide a brief treatment of the class of gamma distributions, and then extend it to cover other invariant distributions, such as the generalized extreme value class. The proposed approach and estimation algorithm are illustrated with real financial datasets. Supplementary materials for this article are available online.     

Extending Integrated Nested Laplace Approximation to a Class of Near‐Gaussian Latent Models

This work extends the integrated nested Laplace approximation (INLA) method to latent models outside the scope of latent Gaussian models, where independent components of the latent field can have a near‐Gaussian distribution. The proposed methodology is an essential component of a bigger project that aims to extend the R package INLA in order to allow the user to add flexibility and challenge the Gaussian assumptions of some of the model components in a straightforward and intuitive way. Our approach is applied to two examples, and the results are compared with that obtained by Markov chain Monte Carlo, showing similar accuracy with only a small fraction of computational time. Implementation of the proposed extension is available in the R‐INLA package.     

Adaptive multiple importance sampling for Gaussian processes

In applications of Gaussian processes (GPs) where quantification of uncertainty is a strict requirement, it is necessary to accurately characterize the posterior distribution over Gaussian process covariance parameters. This is normally done by means of standard Markov chain Monte Carlo (MCMC) algorithms, which require repeated expensive calculations involving the marginal likelihood. Motivated by the desire to avoid the inefficiencies of MCMC algorithms rejecting a considerable amount of expensive proposals, this paper develops an alternative inference framework based on adaptive multiple importance sampling (AMIS). In particular, this paper studies the application of AMIS for GPs in the case of a Gaussian likelihood, and proposes a novel pseudo-marginal-based AMIS algorithm for non-Gaussian likelihoods, where the marginal likelihood is unbiasedly estimated. The results suggest that the proposed framework outperforms MCMC-based inference of covariance parameters in a wide range of scenarios.     

A method for estimating parameters and quantiles of the three-parameter inverse Gaussian distribution based on statistics invariant to unknown location

The inverse Gaussian (IG) distribution, also known as the Wald distribution, is a long-tailed positively skewed distribution and a well-known lifetime distribution. In this paper, we propose an efficient method of estimation for the parameters and quantiles of the three-parameter IG distribution, which is based on statistics invariant to unknown location. Through a Monte Carlo simulation study, we then show that the proposed method performs well compared with other prominent methods in terms of bias and variance. Finally, we present two illustrative examples.