An Approximate Bayesian Marginal Likelihood Approach for Estimating Finite Mixtures

Estimation of finite mixture models when the mixing distribution support is unknown is an important problem. This article gives a new approach based on a marginal likelihood for the unknown support. Motivated by a Bayesian Dirichlet prior model, a computationally efficient stochastic approximation version of the marginal likelihood is proposed and large-sample theory is presented. By restricting the support to a finite grid, a simulated annealing method is employed to maximize the marginal likelihood and estimate the support. Real and simulated data examples show that this novel stochastic approximation and simulated annealing procedure compares favorably with existing methods.     

Some Remarks on The Paper by Theodorescu and Wolff: "Sequential Estimation of Expectations in the Presence of Trend"

Recently Theodoresu and Wolff have proposed a sequence of random variables for estimating a sequence of constants satisfying a certain recurrence equation. Their problem, which is similar to some arising in stochastic approximation theory, was solved by using techniques from this theory. The present paper shows that the theory of sums of independent random variables is a more natural tool for tackling this problem.     

Road trafficking description and short term travel time forecasting,with a classification method

The purpose of this work is, on the one hand, to study how to forecast road trafficking on highway networks and, on the other hand, to describe future traffic events. Here, road trafficking is measured by vehicle velocities. The authors propose two methodologies. The first is based on an empirical classification method, and the second on a probability mixture model. They use an SAEM‐type algorithm (a stochastic approximation of the EM algorithm) to select the densities of the mixture model. Then, they test the validity of their methodologies by forecasting short term travel times.     

Stochastic approximation Monte Carlo EM for change-point analysis

In the expectation–maximization (EM) algorithm for maximum likelihood estimation from incomplete data, Markov chain Monte Carlo (MCMC) methods have been used in change-point inference for a long time when the expectation step is intractable. However, the conventional MCMC algorithms tend to get trapped in local mode in simulating from the posterior distribution of change points. To overcome this problem, in this paper we propose a stochastic approximation Monte Carlo version of EM (SAMCEM), which is a combination of adaptive Markov chain Monte Carlo and EM utilizing a maximum likelihood method. SAMCEM is compared with the stochastic approximation version of EM and reversible jump Markov chain Monte Carlo version of EM on simulated and real datasets. The numerical results indicate that SAMCEM can outperform among the three methods by producing much more accurate parameter estimates and the ability to achieve change-point positions and estimates simultaneously.     

Discussing the “big n problem”

When a large amount of spatial data is available computational and modeling challenges arise and they are often labeled as “big n problem”. In this work we present a brief review of the literature. Then we focus on two approaches, respectively based on stochastic partial differential equations and integrated nested Laplace approximation, and on the tapering of the spatial covariance matrix. The fitting and predictive abilities of using the two methods in conjunction with Kriging interpolation are compared in a simulation study.     

Zur näherungsweisen ermittlung optimaler stochastischer steuerimgen durch diskretisierung

Considered are stochastic continuous-time control systems described by stochastic differential equations, which are defined by special martingals. Examples are given by the well known Ito equations with respect to a Wiener - or a Poisson process. By means of a performance index, regarding current yields as well as a terminal payment a control problem is formulated. The essential result in view of concrete evaluation is the approximation by a sequence of discrete-time finite dimensional control problems.     

Moment closure based parameter inference of stochastic kinetic models

Parameter inference for stochastic kinetic models is a topic that spans many disciplines. Although it is possible to carry out exact inference using partial observations of a stochastic process, it is often computationally impractical. In this paper we use the moment closure approximation of the underlying stochastic process as a fast approximation of the likelihood. We show that this approximation is fast and accurate, even when the population numbers are small.     

Simultaneous Equation Model based on the generalized maximum entropy for studying the effect of management factors on enterprise performance

The aim of this paper is to study the effect of management factors on enterprise performance, considering a survey that the University Consortium in Engineering for Quality and Innovation has led. The relationships between management factors and enterprise performance are formalized by a Simultaneous Equation Model based on the generalized maximum entropy (GME) estimation method. The format of this paper is as follows. In Section 2, the data collected, the questionnaire evaluation, and the management model analytical formulation are introduced. In Section 3, the GME formulation is specified, showing the main characteristics of the estimation method. In Section 4, the results and a comparison among GME, partial least squares (PLS), and maximum likelihood estimation (MLE) is shown. In Section 5, concluding remarks are discussed.     

Sparse Approximations of Fractional Matérn Fields

We consider fast lattice approximation methods for a solution of a certain stochastic non‐local pseudodifferential operator equation. This equation defines a Matérn class random field. We approximate the pseudodifferential operator with truncated Taylor expansion, spectral domain error functional minimization and rounding approximations. This allows us to construct Gaussian Markov random field approximations. We construct lattice approximations with finite‐difference methods. We show that the solutions can be constructed with overdetermined systems of stochastic matrix equations with sparse matrices, and we solve the system of equations with a sparse Cholesky decomposition. We consider convergence of the truncated Taylor approximation by studying band‐limited Matérn fields. We consider the convergence of the discrete approximations to the continuous limits. Finally, we study numerically the accuracy of different approximation methods with an interpolation problem.     

A permutation approach for testing heterogeneity in two-sample categorical variables

In many sciences researchers often meet the problem of establishing if the distribution of a categorical variable is more concentrated, or less heterogeneous, in population P ₁ than in population P ₂. An approximate nonparametric solution to this problem is discussed within the permutation context. Such a solution has similarities to that of testing for stochastic dominance, that is, of testing under order restrictions, for ordered categorical variables. Main properties of given solution and a Monte Carlo simulation in order to evaluate its degree of approximation and its power behaviour are examined. Two application examples are also discussed.     

Parallel and interacting stochastic approximation annealing algorithms for global optimisation

We present the parallel and interacting stochastic approximation annealing (PISAA) algorithm, a stochastic simulation procedure for global optimisation, that extends and improves the stochastic approximation annealing (SAA) by using population Monte Carlo ideas. The efficiency of standard SAA algorithm crucially depends on its self-adjusting mechanism which presents stability issues in high dimensional or rugged optimisation problems. The proposed algorithm involves simulating a population of SAA chains that interact each other in a manner that significantly improves the stability of the self-adjusting mechanism and the search for the global optimum in the sampling space, as well as it inherits SAA desired convergence properties when a square-root cooling schedule is used. It can be implemented in parallel computing environments in order to mitigate the computational overhead. As a result, PISAA can address complex optimisation problems that it would be difficult for SAA to satisfactory address. We demonstrate the good performance of the proposed algorithm on challenging applications including Bayesian network learning and protein folding. Our numerical comparisons suggest that PISAA outperforms the simulated annealing, stochastic approximation annealing, and annealing evolutionary stochastic approximation Monte Carlo.     

Improved bridge constructs for stochastic differential equations

We consider the task of generating discrete-time realisations of a nonlinear multivariate diffusion process satisfying an Itô stochastic differential equation conditional on an observation taken at a fixed future time-point. Such realisations are typically termed diffusion bridges. Since, in general, no closed form expression exists for the transition densities of the process of interest, a widely adopted solution works with the Euler–Maruyama approximation, by replacing the intractable transition densities with Gaussian approximations. However, the density of the conditioned discrete-time process remains intractable, necessitating the use of computationally intensive methods such as Markov chain Monte Carlo. Designing an efficient proposal mechanism which can be applied to a noisy and partially observed system that exhibits nonlinear dynamics is a challenging problem, and is the focus of this paper. By partitioning the process into two parts, one that accounts for nonlinear dynamics in a deterministic way, and another as a residual stochastic process, we develop a class of novel constructs that bridge the residual process via a linear approximation. In addition, we adapt a recently proposed construct to a partial and noisy observation regime. We compare the performance of each new construct with a number of existing approaches, using three applications.     

Analytical and numerical studies on the second-order asymptotic expansion method for European option pricing under two-factor stochastic volatilities

The celebrated Black–Scholes model made the assumption of constant volatility but empirical studies on implied volatility and asset dynamics motivated the use of stochastic volatilities. Christoffersen in 2009 showed that multi-factor stochastic volatilities models capture the asset dynamics more realistically. Fouque in 2012 used it to price European options. In 2013, Chiarella and Ziveyi considered Christoffersen’s ideas and introduced an asset dynamics where the two volatilities of the Heston type act separately and independently on the asset price, and using Fourier transform for the asset price process and double Laplace transform for the two volatilities processes, solved a pricing problem for American options. This paper considers the Chiarella and Ziveyi model and parameterizes it so that the volatilities revert to the long-run-mean with reversion rates that mimic fast (for example daily) and slow (for example seasonal) random effects. Applying asymptotic expansion method presented by Fouque in 2012, we make an extensive and detailed derivation of the approximation prices for European options. We also present numerical studies on the behavior and accuracy of our first- and second-order asymptotic expansion formulas.     

On the time to extinction in recurrent epidemics

An approximation is derived for the expected time to extinction in a stochastic model for recurrent epidemics. Numerical illustrations indicate that the approximation is crude but that it has the correct order of magnitude. The quasi-stationary distribution plays an important role in the derivation. Approximations for the critical community size and of the persistence threshold are derived. Comments are made on the classical study by Bartlell (1956–1960).     

Compromise allocation in multivariate stratified surveys with stochastic quadratic cost function

In many real life situations the linear cost function does not approximate the actual cost incurred adequately. The cost of traveling between the units selected in the sample within a stratum is significant, instead of linear cost function. In this paper, we have considered the problem of finding a compromise allocation for a multivariate stratified sample survey with a significant travel cost within strata is formulated as a problem of non-linear stochastic programming with multiple objective functions. The compromise solutions are obtained through Chebyshev approximation technique, D ₁- distance and goal programming. A numerical example is presented to illustrate the computational details of the proposed methods.     

An Empirical Comparison of Estimating Methods for the Ramberg–Osgood EIV Model

The standard tensile test is one of the most frequent tools performed for the evaluation of mechanical properties of metals. An empirical model proposed by Ramberg and Osgood fits the tensile test data using a nonlinear model for the strain in terms of the stress. It is an Error-In-Variables (EIV) model because of the uncertainty affecting both strain and stress measurement instruments. The SIMEX, a simulation-based method for the estimation of model parameters, is powerful in order to reduce bias due to the measurement error in EIV models. The plan of this article is the following. In Sec. 2, we introduce the Ramberg–Osgood model and another reparametrization according to different assumptions on the independent variable. In Sec. 3, there is a summary of SIMEX method for the case at hand. Section 4 is a comparison between SIMEX and others estimating methods in order to highlight the peculiarities of the different approaches. In the last section, there are some concluding remarks.     

Exact distribution-free tests for equality of several linear models

Exact tests for the equality of several linear models are developed using permutation techniques. Two cases of the linear model, characterized by either stochastic or nonstochastic predictors, are considered: the linear regression model (LRM) and the general linear model (GLM). A general class of test statistics using the volume of simplexes as the basic unit of analysis is proposed for this problem. The resulting class of statistics is shown to be a natural generalization of the multi-response permutation procedure (MRPP) test statistics which have been shown to comprise many of the statistics used in both parametric and nonparametric analysis of the standard g—sample problem. In the LRM case, exact moments of all orders are derived for the permutation distribution of any test statistic in the general class. Moment-based approximation of significance levels is shown to be computationally feasible in the simple LRM.     

Stochastic approximation Monte Carlo importance sampling for approximating exact conditional probabilities

Importance sampling and Markov chain Monte Carlo methods have been used in exact inference for contingency tables for a long time, however, their performances are not always very satisfactory. In this paper, we propose a stochastic approximation Monte Carlo importance sampling (SAMCIS) method for tackling this problem. SAMCIS is a combination of adaptive Markov chain Monte Carlo and importance sampling, which employs the stochastic approximation Monte Carlo algorithm (Liang et al., J. Am. Stat. Assoc., 102(477):305–320, 2007) to draw samples from an enlarged reference set with a known Markov basis. Compared to the existing importance sampling and Markov chain Monte Carlo methods, SAMCIS has a few advantages, such as fast convergence, ergodicity, and the ability to achieve a desired proportion of valid tables. The numerical results indicate that SAMCIS can outperform the existing importance sampling and Markov chain Monte Carlo methods: It can produce much more accurate estimates in much shorter CPU time than the existing methods, especially for the tables with high degrees of freedom.     

Computation of the mean value and variance functions in geometric process

Geometric process (GP) is widely used as a non-stationary stochastic model in reliability analysis. In many of applications related with GP its mean value and variance functions are needed. Since there are no analytical forms of these functions in a lot of situations their computations are of importance. In this study, a numerical approximation and Monte Carlo estimation method based on the convolutions of distribution functions have been proposed for both the mean value and variance functions.     

Serial correlation and distributions on the shere

Estimation and tests for serial correlation in recation and regression models with normal error have been derive from various points of view; for example: Anderson (1948), Durbi for Watson (1950, 1951, 1971), Theil (1965), Durbin (1970), Haq (1970), Kadiyala (1970), Abrahamse & Louter (1971), Levenbac (1972), Berenblut & Webb (1973), Phillips & Harvey (1974), a Sims (1975). In this paper we derive likelihood functions and most powerful tests for serial correclation in Locationa and regression models with arbitrary but specificed error; the methods extend to include the determination of the likelihood for the parameter of the error distribution.

In Section 2, we survey the modthods that have been used in deriving the various tests and estimates in the literature. In Section 2, we introduce the stataistical model that directly describes the error distribution and we obtain the likelihood function for error correlation and determine locally and specifically kost powerful tests for correlation. In Section 3 we consider the case with normal error derive a normal distribution on the sphere by radial projection. The likelihood function and test are then specialized to the case of normal error in Section 4. The computational procedures for the tests and related power functions are examined in Section 5. Power comparisons for the textile data of Theil and Nagar (1961), the consumption data of Kelin (1950), and the plums and the wheat data of Hildreth & Lu (1960) are presented in Section 6, while the likelihood functions for correlation in these data are given in Section 7.