Approximate and estimated saddlepoint approximations

Classical saddlepoint methods, which assume that the cumulant generating function is known, result in an approximation to the distribution that achieves an error of order O(n^?1). The authors give a general theorem to address the accuracy of saddlepoint approximations in which the cumulant generating function has been estimated or approximated. In practice, the resulting saddlepoint approximations are typically of the order O(n^?1/2). The authors give simulation results for small sample examples to compare estimated saddlepoint approximations.     

Saddlepoint approximations to tail expectations under non-Gaussian base distributions: option pricing applications

The saddlepoint approximation formulas provide versatile tools for analytic approximation of the tail expectation of a random variable by approximating the complex Laplace integral of the tail expectation expressed in terms of the cumulant generating function of the random variable. We generalize the saddlepoint approximation formulas for calculating tail expectations from the usual Gaussian base distribution to an arbitrary base distribution. Specific discussion is presented on the criteria of choosing the base distribution that fits better the underlying distribution. Numerical performance and comparison of accuracy are made among different saddlepoint approximation formulas. Improved accuracy of the saddlepoint approximations to tail expectations is revealed when proper base distributions are chosen. We also demonstrate enhanced accuracy of the generalized saddlepoint approximation formulas under non-Gaussian base distributions in pricing European options on continuous integrated variance under the Heston stochastic volatility model.     

Approximations to noncentral distributions

Two methods for approximating the distribution of a noncentral random variable by a central distribution in the same family are presented. The first consists of relating a stochastic expansion of a random variable to a corresponding asymptotic expansion for its distribution function. The second approximates the cumulant generating function and is used to provide central χ² and gamma approximations to the noncentral χ² and gamma distributions.     

Density Approximation by Summary Statistics: An Information-theoretic Approach

In the case of exponential families, it is a straightforward matter to approximate a density function by use of summary statistics; however, an appropriate approach to such approximation is far less clear when an exponential family is not assumed. In this paper, a maximin argument based on information theory is used to derive a new approach to density approximation from summary statistics which is not restricted by the assumption of validity of an underlying exponential family. Information-theoretic criteria are developed to assess loss of predictive power of summary statistics under such minimal knowledge. Under these criteria, optimal density approximations in the maximin sense are obtained and shown to be related to exponential families. Conditions for existence of optimal density approximations are developed. Applications of the proposed approach are illustrated, and methods for estimation of densities are provided in the case of simple random sampling. Large-sample theory for estimates is developed.     

Properties of doubly-truncated gamma variables

The truncated gamma distribution has been widely studied, primarily in life-testing and reliability settings. Most work has assumed an upper bound on the support of the random variable, i.e. the space of the distribution is (0,u). We consider a doubly-truncated gamma random variable restricted by both a lower (l) and upper (u) truncation point, both of which are considered known. We provide simple forms for the density, cumulative distribution function (CDF), moment generating function, cumulant generating function, characteristic function, and moments. We extend the results to describe the density, CDF, and moments of a doubly-truncated noncentral chi-square variable.     

A uniform saddlepoint expansion for the null‐distribution of the wilcoxon‐mann‐whitney statistic

The authors give the exact coefficient of 1/N in a saddlepoint approximation to the Wilcoxon‐Mann‐Whitney null‐distribution. This saddlepoint approximation is obtained from an Edgeworth approximation to the exponentially tilted distribution. Moreover, the rate of convergence of the relative error is uniformly of order O (1/N) in a large deviation interval as defined in Feller (1971). The proposed method for computing the coefficient of 1/N can be used to obtain the exact coefficients of 1/Nⁱ, for any i. The exact formulas for the cumulant generating function and the cumulants, needed for these results, are those of van Dantzig (1947‐1950).     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

On parameter orthogonality to the mean

The existence of orthogonal parameters to the mean is characterized by a partial differential equation involving the mean, the variance and the cumulant generating function. This condition allows to explain and construct orthogonal parametrizations in several cases of interest, including higher parametric ones.     

Simulation of Infinitely Divisible Random Fields

Two methods to approximate infinitely divisible random fields are presented. The methods are based on approximating the kernel function in the spectral representation of such fields, leading to numerical integration of the respective integrals. Error bounds for the approximation error are derived and the approximations are used to simulate certain classes of infinitely divisible random fields.     

Shuffle of min’s random variable approximations of bivariate copulas’ realization

The comonotonicity and countermonotonicity provide intuitive upper and lower dependence relationship between random variables. This paper constructs the shuffle of min’s random variable approximations for a given Uniform [0, 1] random vector. We find the two optimal orders under which the shuffle of min’s random variable approximations obtained are shown to be extensions of comonotonicity and countermonotonicity. We also provide the rate of convergence of these random vectors approximations and apply them to compute value-at-risk.     

Improved Measures of the Spread of Data for Some Unknown Complex Distributions Using Saddlepoint Approximations

Measures of the spread of data for random sums arise frequently in many problems and have a wide range of applications in real life, such as in the insurance field (e.g., the total claim size in a portfolio). The exact distribution of random sums is extremely difficult to determine, and normal approximation usually performs very badly for this complex distributions. A better method of approximating a random-sum distribution involves the use of saddlepoint approximations.

Saddlepoint approximations are powerful tools for providing accurate expressions for distribution functions that are not known in closed form. This method not only yields an accurate approximation near the center of the distribution but also controls the relative error in the far tail of the distribution.

In this article, we discuss approximations to the unknown complex random-sum Poisson–Erlang random variable, which has a continuous distribution, and the random-sum Poisson-negative binomial random variable, which has a discrete distribution. We show that the saddlepoint approximation method is not only quick, dependable, stable, and accurate enough for general statistical inference but is also applicable without deep knowledge of probability theory. Numerical examples of application of the saddlepoint approximation method to continuous and discrete random-sum Poisson distributions are presented.     

Development and comparative study of two near-exact approximations to the distribution of the product of an odd number of independent Beta random variables

Using the concept of near-exact approximation to a distribution we developed two different near-exact approximations to the distribution of the product of an odd number of particular independent Beta random variables (r.v.'s). One of them is a particular generalized near-integer Gamma (GNIG) distribution and the other is a mixture of two GNIG distributions. These near-exact distributions are mostly adequate to be used as a basis for approximations of distributions of several statistics used in multivariate analysis. By factoring the characteristic function (c.f.) of the logarithm of the product of the Beta r.v.'s, and then replacing a suitably chosen factor of that c.f. by an adequate asymptotic result it is possible to obtain what we call a near-exact c.f., which gives rise to the near-exact approximation to the exact distribution. Depending on the asymptotic result used to replace the chosen parts of the c.f., one may obtain different near-exact approximations. Moments from the two near-exact approximations developed are compared with the exact ones. The two approximations are also compared with each other, namely in terms of moments and quantiles.     

Bayesian inference for the variance components in general mixed linear models

The general mixed linear model, containing both the fixed and random effects, is considered. Using gamma priors for the variance components, the conditional posterior distributions of the fixed effects and the variance components, conditional on the random effects, are obtained. Using the normal approximation for the multiple t distribution, approximations are obtained for the posterior distributions of the variance components in infinite series form. The same approximation Is used to obtain closed expressions for the moments of the variance components. An example is considered to illustrate the procedure and a numerical study examines the closeness of the approximations.     

Spectral and circulant approximations to the likelihood for stationary Gaussian random fields

Consider a Gaussian random field model on , observed on a rectangular region. Suppose it is desired to estimate a set of parameters in the covariance function. Spectral and circulant approximations to the likelihood are often used to facilitate estimation of the parameters. The purpose of the paper is to give a careful treatment of the quality of these approximations. A spectral approximation for the likelihood was given by Guyon (Biometrika 69 (1982) 95–105) but without proof. The results given here generalize those of Guyon, and fill in the details of the proof. In addition some matrix results are derived which may be of independent interest. Applications are made to Fisher information and bias calculations for maximum likelihood estimates.     

A NORMAL APPROXIMATION TO THE FISHER DISTRIBUTION

A simple normal approximation is given for the joint probability density function of the polar co-ordinates (θ, ψ) of a random vector following the Fisher distribution with arbitrary mean direction (θ₀, ψ₀). The approximation leads to simple inference procedures which are particularly useful in regression models. Conditions for the adequacy of the approximation are investigated and summarized in tabular form.     

A taxonomy of mixing and outcome distributions based on conjugacy and bridging

Abstract

The generalized linear mixed model (GLMM) is commonly used for the analysis of hierarchical non Gaussian data. It combines an exponential family model formulation with normally distributed random effects. A drawback is the difficulty of deriving convenient marginal mean functions with straightforward parametric interpretations. Several solutions have been proposed, including the marginalized multilevel model (directly formulating the marginal mean, together with a hierarchical association structure) and the bridging approach (choosing the random-effects distribution such that marginal and hierarchical mean functions share functional forms). Another approach, useful in both a Bayesian and a maximum-likelihood setting, is to choose a random-effects distribution that is conjugate to the outcome distribution. In this paper, we contrast the bridging and conjugate approaches. For binary outcomes, using characteristic functions and cumulant generating functions, it is shown that the bridge distribution is unique. Self-bridging is introduced as the situation in which the outcome and random-effects distributions are the same. It is shown that only the Gaussian and degenerate distributions have well-defined cumulant generating functions for which self-bridging holds.     

Exact and Approximate Statistical Inference for Nonlinear Regression and the Estimating Equation Approach

The exact density distribution of the non‐linear least squares estimator in the one‐parameter regression model is derived in closed form and expressed through the cumulative distribution function of the standard normal variable. Several proposals to generalize this result are discussed. The exact density is extended to the estimating equation (EE) approach and the non‐linear regression with an arbitrary number of linear parameters and one intrinsically non‐linear parameter. For a very special non‐linear regression model, the derived density coincides with the distribution of the ratio of two normally distributed random variables previously obtained by Fieler almost a century ago, unlike other approximations previously suggested by other authors. Approximations to the density of the EE estimators are discussed in the multivariate case. Numerical complications associated with the non‐linear least squares are illustrated, such as non‐existence and/or multiple solutions, as major factors contributing to poor density approximation. The non‐linear Markov–Gauss theorem is formulated on the basis of the near exact EE density approximation.     

Sparse Approximations of Fractional Matérn Fields

We consider fast lattice approximation methods for a solution of a certain stochastic non‐local pseudodifferential operator equation. This equation defines a Matérn class random field. We approximate the pseudodifferential operator with truncated Taylor expansion, spectral domain error functional minimization and rounding approximations. This allows us to construct Gaussian Markov random field approximations. We construct lattice approximations with finite‐difference methods. We show that the solutions can be constructed with overdetermined systems of stochastic matrix equations with sparse matrices, and we solve the system of equations with a sparse Cholesky decomposition. We consider convergence of the truncated Taylor approximation by studying band‐limited Matérn fields. We consider the convergence of the discrete approximations to the continuous limits. Finally, we study numerically the accuracy of different approximation methods with an interpolation problem.     

Approximate computations for binary Markov random fields and their use in Bayesian models

Discrete Markov random fields form a natural class of models to represent images and spatial datasets. The use of such models is, however, hampered by a computationally intractable normalising constant. This makes parameter estimation and a fully Bayesian treatment of discrete Markov random fields difficult. We apply approximation theory for pseudo-Boolean functions to binary Markov random fields and construct approximations and upper and lower bounds for the associated computationally intractable normalising constant. As a by-product of this process we also get a partially ordered Markov model approximation of the binary Markov random field. We present numerical examples with both the pairwise interaction Ising model and with higher-order interaction models, showing the quality of our approximations and bounds. We also present simulation examples and one real data example demonstrating how the approximations and bounds can be applied for parameter estimation and to handle a fully Bayesian model computationally.     

Modified Normal-based Approximation to the Percentiles of Linear Combination of Independent Random Variables with Applications

A modified normal-based approximation for calculating the percentiles of a linear combination of independent random variables is proposed. This approximation is applicable in situations where expectations and percentiles of the individual random variables can be readily obtained. The merits of the approximation are evaluated for the chi-square and beta distributions using Monte Carlo simulation. An approximation to the percentiles of the ratio of two independent random variables is also given. Solutions based on the approximations are given for some classical problems such as interval estimation of the normal coefficient of variation, survival probability, the difference between or the ratio of two binomial proportions, and for some other problems. Furthermore, approximation to the percentiles of a doubly noncentral F distribution is also given. For all the problems considered, the approximation provides simple satisfactory solutions. Two examples are given to show applications of the approximation.