Conditionally Reducible Natural Exponential Families and Enriched Conjugate Priors

Consider a standard conjugate family of prior distributions for a vector-parameter indexing an exponential family. Two distinct model parameterizations may well lead to standard conjugate families which are not consistent, i.e. one family cannot be derived from the other by the usual change-of-variable technique. This raises the problem of finding suitable parameterizations that may lead to enriched conjugate families which are more flexible than the traditional ones. The previous remark motivates the definition of a new property for an exponential family, named conditional reducibility. Features of conditionally-reducible natural exponential families are investigated thoroughly. In particular, we relate this new property to the notion of cut, and show that conditionally-reducible families admit a reparameterization in terms of a vector having likelihood-independent components. A general methodology to obtain enriched conjugate distributions for conditionally-reducible families is described in detail, generalizing previous works and more recent contributions in the area. The theory is illustrated with reference to natural exponential families having simple quadratic variance function.     

Laplace Approximations for Natural Exponential Families with Cuts

Standard and fully exponential form Laplace approximations to marginal densities are described and conditions under which these give exact answers are investigated. A general result is obtained and is subsequently applied in the case of natural exponential families with cuts, in order to derive the marginal posterior density of the mean parameter corresponding to the cut, the canonical parameter corresponding to the complement of the cut and transformations of these. Important cases of families for which a cut exists and the approximations are exact are presented as examples     

Density Approximation by Summary Statistics: An Information-theoretic Approach

In the case of exponential families, it is a straightforward matter to approximate a density function by use of summary statistics; however, an appropriate approach to such approximation is far less clear when an exponential family is not assumed. In this paper, a maximin argument based on information theory is used to derive a new approach to density approximation from summary statistics which is not restricted by the assumption of validity of an underlying exponential family. Information-theoretic criteria are developed to assess loss of predictive power of summary statistics under such minimal knowledge. Under these criteria, optimal density approximations in the maximin sense are obtained and shown to be related to exponential families. Conditions for existence of optimal density approximations are developed. Applications of the proposed approach are illustrated, and methods for estimation of densities are provided in the case of simple random sampling. Large-sample theory for estimates is developed.     

The superharmonic condition for simultaneous estimation of means in exponential familles

The mean vector associated with several independent variates from the exponential subclass of Hudson (1978) is estimated under weighted squared error loss. In particular, the formal Bayes and “Stein-like” estimators of the mean vector are given. Conditions are also given under which these estimators dominate any of the “natural estimators”. Our conditions for dominance are motivated by a result of Stein (1981), who treated the N_p (θ, I) case with p ≥ 3. Stein showed that formal Bayes estimators dominate the usual estimator if the marginal density of the data is superharmonic. Our present exponential class generalization entails an elliptic differential inequality in some natural variables. Actually, we assume that each component of the data vector has a probability density function which satisfies a certain differential equation. While the densities of Hudson (1978) are particular solutions of this equation, other solutions are not of the exponential class if certain parameters are unknown. Our approach allows for the possibility of extending the parametric Stein-theory to useful nonexponential cases, but the problem of nuisance parameters is not treated here.     

Perfect samplers for mixtures of distributions

Summary. We consider the construction of perfect samplers for posterior distributions associated with mixtures of exponential families and conjugate priors, starting with a perfect slice sampler in the spirit of Mira and co-workers. The methods rely on a marginalization akin to Rao–Blackwellization and illustrate the duality principle of Diebolt and Robert. A first approximation embeds the finite support distribution on the latent variables within a continuous support distribution that is easier to simulate by slice sampling, but we later demonstrate that the approximation can be very poor. We conclude by showing that an alternative perfect sampler based on a single backward chain can be constructed. This alternative can handle much larger sample sizes than the slice sampler first proposed.     

市场风险的高效率加速算法研究

在计算投资组合市场风险时,采用高效率重要性抽样技术来处理大规模、高维度和稀有事件问题可以提高计算的速度和效率。在对投资组合损失进行Delta-Gamma近似的基础上,通过利用辅助分布变换函数,将求解抽样参数的最小抽样方差问题转化为一个非线性的广义最小二乘问题;在指数族抽样核的假设下,进一步将问题转化为迭代线性回归问题,从而简化了计算;通过德尔塔对冲和指数对冲投资组合的模拟算例验证了所提出方法的有效性。     

Series evaluation of Tweedie exponential dispersion model densities

Exponential dispersion models, which are linear exponential families with a dispersion parameter, are the prototype response distributions for generalized linear models. The Tweedie family comprises those exponential dispersion models with power mean-variance relationships. The normal, Poisson, gamma and inverse Gaussian distributions belong to theTweedie family. Apart from these special cases, Tweedie distributions do not have density functions which can be written in closed form. Instead, the densities can be represented as infinite summations derived from series expansions. This article describes how the series expansions can be summed in an numerically efficient fashion. The usefulness of the approach is demonstrated, but full machine accuracy is shown not to be obtainable using the series expansion method for all parameter values. Derivatives of the density with respect to the dispersion parameter are also derived to facilitate maximum likelihood estimation. The methods are demonstrated on two data examples and compared with with Box-Cox transformations and extended quasi-likelihoood.     

Instability, Sensitivity, and Degeneracy of Discrete Exponential Families

In applications to dependent data, first and foremost relational data, a number of discrete exponential family models has turned out to be near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We introduce the notion of instability with an eye to characterize, detect, and penalize discrete exponential family models that are near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We show that unstable discrete exponential family models are characterized by excessive sensitivity and near-degeneracy. In special cases, the subset of the natural parameter space corresponding to non-degenerate distributions and mean-value parameters far from the boundary of the mean-value parameter space turns out to be a lower-dimensional subspace of the natural parameter space. These characteristics of unstable discrete exponential family models tend to obstruct Markov chain Monte Carlo simulation and statistical inference. In applications to relational data, we show that discrete exponential family models with Markov dependence tend to be unstable and that the parameter space of some curved exponential families contains unstable subsets.     

Comparisons of Three Approaches for Discrete Conditional Models

This article describes three methods for computing a discrete joint density from full conditional densities. They are the Gibbs sampler, a hybrid method, and an interaction-based method. The hybrid method uses the iterative proportional fitting algorithm, and it is derived from the mixed parameterization of a contingency table. The interaction-based approach is derived from the canonical parameters, while the Gibbs sampler can be regarded as based on the mean parameters. In short, different approaches are motivated by different parameterizations. The setting of a bivariate conditionally specified distribution is used as the premise for comparing the numerical accuracy of the three methods. Detailed comparisons of marginal distributions, odds ratios and expected values are reported. We give theoretical justifications as to why the hybrid method produces better approximation than the Gibbs sampler. Generalizations to more than two variables are discussed. In practice, Gibbs sampler has certain advantages: it is conceptually easy to understand and there are many software tools available. Nevertheless, the hybrid method and the interaction-based method are accurate and simple alternatives when the Gibbs sampler results in a slowly mixing chain and requires substantial simulation efforts.     

Langevin incremental mixture importance sampling

This work proposes a novel method through which local information about the target density can be used to construct an efficient importance sampler. The backbone of the proposed method is the incremental mixture importance sampling (IMIS) algorithm of Raftery and Bao (Biometrics 66(4):1162–1173, 2010), which builds a mixture importance distribution incrementally, by positioning new mixture components where the importance density lacks mass, relative to the target. The key innovation proposed here is to construct the mean vectors and covariance matrices of the mixture components by numerically solving certain differential equations, whose solution depends on the local shape of the target log-density. The new sampler has a number of advantages: (a) it provides an extremely parsimonious parametrization of the mixture importance density, whose configuration effectively depends only on the shape of the target and on a single free parameter representing pseudo-time; (b) it scales well with the dimensionality of the target; (c) it can deal with targets that are not log-concave. The performance of the proposed approach is demonstrated on two synthetic non-Gaussian densities, one being defined on up to eighty dimensions, and on a Bayesian logistic regression model, using the Sonar dataset. The Julia code implementing the importance sampler proposed here can be found at https://github.com/mfasiolo/LIMIS.     

On the consistency of MLE in finite mixture models of exponential families

Finite mixtures of densities from an exponential family are frequently used in the statistical analysis of data. Modelling by finite mixtures of densities from different exponential families provide more flexibility in the fittings, and get better results. However, in mixture problems, the log-likelihood function very often does not have an upper bound and therefore a global maximum does not always exist. Redner and Walker (1984. Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239) provide conditions to assure the existence, consistency and asymptotic normality of the maximum likelihood estimator.     

Minimum variance unbiased estimation in natural exponential families generated by stable distributions

The class of nature exponential families generated by stable distributions has been introduced in different contexts by several authors. Tweedie (1984) and Jorgensen (1987) studied this class in the context of generalized liner models and exponential dispersion models. Bar-Lev and Enis (1986) introduced this class in the context of the property of reproducibility in natural exponential families and Hougaard (1986) found the distributions in this class to be natural candidates for applications as survival distributions in life tables for heterogeneous populations. In this note, we consider such a class in the context of minimum variance unbiased estimation. For each family in this class, we obtain an explicit expression for the uniformly minimum variance unbiased estimator for the r-th cumlant, the density function, and the reliability function.     

Generalized Additive Models for Zero‐Inflated Data with Partial Constraints

Abstract. Zero‐inflated data abound in ecological studies as well as in other scientific fields. Non‐parametric regression with zero‐inflated response may be studied via the zero‐inflated generalized additive model (ZIGAM) with a probabilistic mixture distribution of zero and a regular exponential family component. We propose the (partially) constrained ZIGAM, which assumes that some covariates affect the probability of non‐zero‐inflation and the regular exponential family distribution mean proportionally on the link scales. When the assumption obtains, the new approach provides a unified framework for modelling zero‐inflated data, which is more parsimonious and efficient than the unconstrained ZIGAM. We develop an iterative estimation algorithm, and discuss the confidence interval construction of the estimator. Some asymptotic properties are derived. We also propose a Bayesian model selection criterion for choosing between the unconstrained and constrained ZIGAMs. The new methods are illustrated with both simulated data and a real application in jellyfish abundance data analysis.     

A Bayesian approach to some overdispersion models

In fitting a generalized linear model, many authors have noticed that data sets can show greater residual variability than predicted under the exponential family. Two main approaches have been used to model this overdispersion. The first approach uses a sampling density which is a conjugate mixture of exponential family distributions. The second uses a quasilikelihood which adds a new scale parameter to the exponential likelihood. The approaches are compared by means of a Bayesian analysis using noninformative priors. In examples, it is indicated that the posterior analysis can be significantly different using the two approaches.     

Une condition nécessaire d'admissibilité et ses conséquences sur les estimateurs à rétrécisseur de la moyenne d'un vecteur normal

Considering exponential families of distributions, we estimate parameters which are not the natural parameters. We prove that the admissible estimators of these parameters are limits of Bayes estimators and can be expressed through a given functional form. An important particular case of this model pertains to the estimation of the mean of a multidimensional normal distribution when the variance is known up to a multiplicative factor. We deduce from the main result a necessry condition for the admissibility of matricial shrinkage estimators.     

指数族分布中参数的经验贝叶斯估计

指数族分布是一类应用广泛的分布类,包括了泊松分布、Gamma分布、Beta分布、二项分布等常见分布.在非寿险中,索赔额或索赔次数过程常常被假定服从指数族分布,由于风险的非齐次性,指数族分布中的参数θ也为随机变量,假定服从指数族共轭先验分布.此时风险参数的估计落入了Bayes框架,风险参数θ的Bayes估计被表达“信度”形式.然而,在实际运用中,由于先验分布与样本分布中仍然含有结构参数,根据样本的边际分布的似然函数估计结构参数,从而获得风险参数的经验Bayes估计,最后证明了该经验Bayes估计是渐近最优的.     

Asymptotic bayesian inference in some nonstandard cases: Bernstein–von Mises Results and regular bayes' estimators

Asymptotically with probability close to one, the convergence in variation (also in distribution) to the multivariate normal, of the aposteriori density function of a parameter agains an apriori density, viz. the BERNSTEIN–VON MISES results are established when observations are not necessarily indenpendent or identically distributed but satisfy weak regularity assumptions on their joint density function. Regular BAYES' estimators are defined with respect to regular loss functions and a positive apriori density and proved consistent, asymptotically efficient and asymptotically normal. Examples and applications to conjugate families of densities, to inference in MARKOV Chains and other nonstandard cases illustrate results     

A bayesian analysis of autoregressive models with random normal coefficients

In this paper we consider a Bayesian analysis for an autoregressive model with random normal coefficients (RCA). For the proposed procedure we use conjugate priors for some parameters and improper vague priors for others. The inference for the parameters is made via Gibbs sampler and the convergence is assessed with multiple chains and Gelman and Rubin criterium. Forecasts are based on the predictive density of future observations. Some remarks are also made regarding order determination and stationarity. Applications to simulated and real series are given.     

Statistical analysis of the Lomax–Logarithmic distribution

In this paper we introduce a three-parameter lifetime distribution following the Marshall and Olkin [New method for adding a parameter to a family of distributions with application to the exponential and Weibull families. Biometrika. 1997;84(3):641–652] approach. The proposed distribution is a compound of the Lomax and Logarithmic distributions (LLD). We provide a comprehensive study of the mathematical properties of the LLD. In particular, the density function, the shape of the hazard rate function, a general expansion for moments, the density of the rth order statistics, and the mean and median deviations of the LLD are derived and studied in detail. The maximum likelihood estimators of the three unknown parameters of LLD are obtained. The asymptotic confidence intervals for the parameters are also obtained based on asymptotic variance–covariance matrix. Finally, a real data set is analysed to show the potential of the new proposed distribution.     

On conjugate families and Jeffreys priors for von Mises–Fisher distributions

This paper discusses characteristics of standard conjugate priors and their induced posteriors in Bayesian inference for von Mises–Fisher distributions, using either the canonical natural exponential family or the more commonly employed polar coordinate parameterizations. We analyze when standard conjugate priors as well as posteriors are proper, and investigate the Jeffreys prior for the von Mises–Fisher family. Finally, we characterize the proper distributions in the standard conjugate family of the (matrix-valued) von Mises–Fisher distributions on Stiefel manifolds.