Density Approximation by Summary Statistics: An Information-theoretic Approach

In the case of exponential families, it is a straightforward matter to approximate a density function by use of summary statistics; however, an appropriate approach to such approximation is far less clear when an exponential family is not assumed. In this paper, a maximin argument based on information theory is used to derive a new approach to density approximation from summary statistics which is not restricted by the assumption of validity of an underlying exponential family. Information-theoretic criteria are developed to assess loss of predictive power of summary statistics under such minimal knowledge. Under these criteria, optimal density approximations in the maximin sense are obtained and shown to be related to exponential families. Conditions for existence of optimal density approximations are developed. Applications of the proposed approach are illustrated, and methods for estimation of densities are provided in the case of simple random sampling. Large-sample theory for estimates is developed.     

Exact distribution of the MLEs of the parameters and of the quantiles of two-parameter exponential distribution under hybrid censoring

Epstein [Truncated life tests in the exponential case, Ann. Math. Statist. 25 (1954), pp. 555–564] introduced a hybrid censoring scheme (called Type-I hybrid censoring) and Chen and Bhattacharyya [Exact confidence bounds for an exponential parameter under hybrid censoring, Comm. Statist. Theory Methods 17 (1988), pp. 1857–1870] derived the exact distribution of the maximum-likelihood estimator (MLE) of the mean of a scaled exponential distribution based on a Type-I hybrid censored sample. Childs et al. [Exact likelihood inference based on Type-I and Type-II hybrid censored samples from the exponential distribution, Ann. Inst. Statist. Math. 55 (2003), pp. 319–330] provided an alternate simpler expression for this distribution, and also developed analogous results for another hybrid censoring scheme (called Type-II hybrid censoring). The purpose of this paper is to derive the exact bivariate distribution of the MLE of the parameter vector of a two-parameter exponential model based on hybrid censored samples. The marginal distributions are derived and exact confidence bounds for the parameters are obtained. The results are also used to derive the exact distribution of the MLE of the pth quantile, as well as the corresponding confidence bounds. These exact confidence intervals are then compared with parametric bootstrap confidence intervals in terms of coverage probabilities. Finally, we present some numerical examples to illustrate the methods of inference developed here.     

On Unbiased Estimation of Positive Integral Powers of the Natural Parameter in Exponential Families

We explore the structure of one‐parameter exponential families admitting an unbiased estimator for a positive integral power of the natural parameter. It is seen that only exponential families dominated by Lebesgue measure can have this property. It is outlined that similar results can be obtained for other functions of the natural parameter.     

Jacobi and Laguerre polynomial approximations for the distributions of statistics useful in testing for outliers in exponential and gamma samples

Recently, Sanjel and Balakrishnan [A Laguerre Polynomial Approximation for a goodness-of-fit test for exponential distribution based on progressively censored data, J. Stat. Comput. Simul. 78 (2008), pp. 503–513] proposed the use of Laguerre orthogonal polynomials for a goodness-of-fit test for the exponential distribution based on progressively censored data. In this paper, we use Jacobi and Laguerre orthogonal polynomials in order to obtain density approximants for some test statistics useful in testing for outliers in gamma and exponential samples. We first obtain the exact moments of the statistics and then the density approximants, based on these moments, are expressed in terms of Jacobi and Laguerre polynomials. A comparative study is carried out of the critical values obtained by using the proposed methods to the corresponding results given by Barnett and Lewis [Outliers in Statistical Data, 3rd ed., John Wiley & Sons, New York, 1993]. This reveals that the proposed techniques provide very accurate approximations to the distributions. Finally, we present some numerical examples to illustrate the proposed approximations. Monte Carlo simulations suggest that the proposed approximate densities are very accurate.     

Conditionally Reducible Natural Exponential Families and Enriched Conjugate Priors

Consider a standard conjugate family of prior distributions for a vector-parameter indexing an exponential family. Two distinct model parameterizations may well lead to standard conjugate families which are not consistent, i.e. one family cannot be derived from the other by the usual change-of-variable technique. This raises the problem of finding suitable parameterizations that may lead to enriched conjugate families which are more flexible than the traditional ones. The previous remark motivates the definition of a new property for an exponential family, named conditional reducibility. Features of conditionally-reducible natural exponential families are investigated thoroughly. In particular, we relate this new property to the notion of cut, and show that conditionally-reducible families admit a reparameterization in terms of a vector having likelihood-independent components. A general methodology to obtain enriched conjugate distributions for conditionally-reducible families is described in detail, generalizing previous works and more recent contributions in the area. The theory is illustrated with reference to natural exponential families having simple quadratic variance function.     

Existence of Maximum Likelihood Estimates for Multi-dimensional Exponential Families

This paper deals with the existence of maximum likelihood estimators for multi-dimensional exponential families, including curved exponential families. It first gives an algorithm for determining the MLE from the data. Then it establishes that when the parameter set is either open or relatively closed in the natural parameter set, the MLE of the parameter exists in the sense of Hoffmann-Jorgensen.     

A NOTE ON MULTIVARIATE LOGISTIC MODELS FOR CONTINGENCY TABLES

The log-linear model is a tool widely accepted for modelling discrete data given in a contingency table. Although its parameters reflect the interaction structure in the joint distribution of all variables, it does not give information about structures appearing in the margins of the table. This is in contrast to multivariate logistic parameters, recently introduced by Glonek & McCullagh (1995), which have as parameters the highest order log odds ratios derived from the joint table and from each marginal table. Glonek & McCullagh give the link between the cell probabilities and the multivariate logistic parameters, in an algebraic fashion. The present paper focuses on this link, showing that it is derived by general parameter transformations in exponential families. In particular, the connection between the natural, the expectation and the mixed parameterization in exponential families (Barndorff-Nielsen, 1978) is used; this also yields the derivatives of the likelihood equation and shows properties of the Fisher matrix. The paper emphasises the analysis of independence hypotheses in margins of a contingency table.     

Modified Sarhan–Balakrishnan singular bivariate distribution

Recently Sarhan and Balakrishnan [2007. A new class of bivariate distribution and its mixture. Journal of Multivariate Analysis 98, 1508–1527] introduced a new bivariate distribution using generalized exponential and exponential distributions. They discussed several interesting properties of this new distribution. Unfortunately, they did not discuss any estimation procedure of the unknown parameters. In this paper using the similar idea as of Sarhan and Balakrishnan [2007. A new class of bivariate distribution and its mixture. Journal of Multivariate Analysis 98, 1508–1527], we have proposed a singular bivariate distribution, which has an extra shape parameter. It is observed that the marginal distributions of the proposed bivariate distribution are more flexible than the corresponding marginal distributions of the Marshall–Olkin bivariate exponential distribution, Sarhan–Balakrishnan's bivariate distribution or the bivariate generalized exponential distribution. Different properties of this new distribution have been discussed. We provide the maximum likelihood estimators of the unknown parameters using EM algorithm. We reported some simulation results and performed two data analysis for illustrative purposes. Finally we propose some generalizations of this bivariate model.     

Longitudinal network models and permutation-uniform Markov chains

Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the same parameter, improving data reduction. Further we show that the permutation-uniform subclass of these chains permit interpretation as an independent, identically distributed sequence on the same state space. We then apply these ideas to temporal exponential random graph models, for which permutation uniformity is well suited, and discuss mean-parameter convergence, dyadic independence, and exchangeability. Our framework facilitates our introducing a new network model; simplifies analysis of some network and autoregressive models from the literature, including by permitting closed-form expressions for maximum likelihood estimates for some models; and facilitates applying standard tools to longitudinal-network Markov chains from either asymptotics or single-observation exponential random graph models.     

Exact likelihood inference for two exponential populations based on a joint generalized Type-I hybrid censored sample

Following the work of Chen and Bhattacharyya [Exact confidence bounds for an exponential parameter under hybrid censoring. Comm Statist Theory Methods. 1988;17:1857–1870], several results have been developed regarding the exact likelihood inference of exponential parameters based on different forms of censored samples. In this paper, the conditional maximum likelihood estimators (MLEs) of two exponential mean parameters are derived under joint generalized Type-I hybrid censoring on the two samples. The moment generating functions (MGFs) and the exact densities of the conditional MLEs are obtained, using which exact confidence intervals are then developed for the model parameters. We also derive the means, variances, and mean squared errors of these estimates. An efficient computational method is developed based on the joint MGF. Finally, an example is presented to illustrate the methods of inference developed here.     

Independence Structure of Natural Conjugate Densities to Exponential Families and the Gibbs' Sampler

In this paper the independence between a block of natural parameters and the complementary block of mean value parameters holding for densities which are natural conjugate to some regular exponential families is used to design in a convenient way a Gibbs' sampler with block updates. Even when the densities of interest are obtained by conditioning to zero a block of natural parameters in a density conjugate to a larger "saturated" model, the updates require only the computation of marginal distributions under the "unconditional" density. For exponential families which are closed under marginalization, including both the zero mean Gaussian family and the cross-classified Bernoulli family such an implementation of the Gibbs' sampler can be seen as an Iterative Proportional Fitting algorithm with random inputs.     

All pairwise comparisons of exponential location parameters under type i censoring — equal scale parameter case

In this paper, procedures for all pairwise comparisons of location parameters of negative exponential populations are developed when the common scale parameter is known or unknown using large sample distributional approximations of the relevant random variables. The small sample performance of these procedures are then examined using Monte Carlo simulation.     

Higher-order Bayesian Approximations for Pseudo-posterior Distributions

The theory of higher-order asymptotics provides accurate approximations to posterior distributions for a scalar parameter of interest, and to the corresponding tail area, for practical use in Bayesian analysis. The aim of this article is to extend these approximations to pseudo-posterior distributions, e.g., posterior distributions based on a pseudo-likelihood function and a suitable prior, which are proved to be particularly useful when the full likelihood is analytically or computationally infeasible. In particular, from a theoretical point of view, we derive the Laplace approximation for a pseudo-posterior distribution, and for the corresponding tail area, for a scalar parameter of interest, also in the presence of nuisance parameters. From a computational point of view, starting from these higher-order approximations, we discuss the higher-order tail area (HOTA) algorithm useful to approximate marginal posterior distributions, and related quantities. Compared to standard Markov chain Monte Carlo methods, the main advantage of the HOTA algorithm is that it gives independent samples at a negligible computational cost. The relevant computations are illustrated by two examples.     

Marginal information for expectation parameters

The authors consider a special case of inference in the presence of nuisance parameters. They show that when the orthogonalized score function is a function of a statistic S, no Fisher information for the interest parameter is lost by using the marginal distribution of S rather than the full distribution of the observations. Therefore, no information for the interest parameter is recovered by conditioning on an ancillary statistic, and information will be lost by conditioning on an approximate ancillary statistic. This is the case for regular multivariate exponential families when the interest parameter is a subvector of the expectation parameter and the statistic is the maximum likelihood estimate of the interest parameter. Several examples are considered, including the 2 × 2 table.     

A computer algebra package for approximate conditional inference

This paper presents a set of REDUCE procedures that make a number of existing higher-order asymptotic results available for both theoretical and practical research. Attention has been restricted to the context of exact and approximate inference for a parameter of interest conditionally either on an ancillary statistic or on a statistic partially sufficient for the nuisance parameter. In particular, the procedures apply to regression-scale models and multiparameter exponential families. Most of them support algebraic computation as well as numerical calculation for a given data set. Examples illustrate the code.     

Robust Bayesian estimation of a two-parameter exponential distribution under generalized Type-I progressive hybrid censoring

A generalized Type-I progressive hybrid censoring scheme was proposed recently to overcome the limitations of the progressive hybrid censoring scheme. In this article, we provide a robust Bayesian method to estimate the unknown parameters of the two-parameter exponential distribution of a generalized Type-I progressive hybrid censored sample. For each parameter, we derive the marginal posterior density functions and the corresponding Bayesian estimators under the squared error loss function. To assess the proposed method, Monte Carlo simulations are performed using a real dataset.     

A note on the convergence rates for empirical bayes estimators of parameters in multiple-parameter exponential families

Under suitable conditions upon prior distribution, the convergence rates for empirical Bayes estimators of parameters in multi-parameter exponential families (M-PEF) are obtained. It is shown that the assumptions Tong (1996) imposed on the marginal density can be reduced. The above result can also be extended to more general forms of M-PEF. Finally, some examples which satisfy the conditions of the theorems are given.     

On an exponential family

This paper considers a generalization of the exponential type distributions in the class of exponential families. A characterization and a method of generating an exponential family from a given family are given. In particular the generalized gamma, the generalized Poisson, the inverse Gaussian distributions belonging to this family are discussed. The approximations of the cumulative sums for the generalized gamma and the generalized Poisson by the Chi-square are considered. Some of the results are extended to the bivariate case.     

指数族分布中参数的经验贝叶斯估计

指数族分布是一类应用广泛的分布类,包括了泊松分布、Gamma分布、Beta分布、二项分布等常见分布.在非寿险中,索赔额或索赔次数过程常常被假定服从指数族分布,由于风险的非齐次性,指数族分布中的参数θ也为随机变量,假定服从指数族共轭先验分布.此时风险参数的估计落入了Bayes框架,风险参数θ的Bayes估计被表达“信度”形式.然而,在实际运用中,由于先验分布与样本分布中仍然含有结构参数,根据样本的边际分布的似然函数估计结构参数,从而获得风险参数的经验Bayes估计,最后证明了该经验Bayes估计是渐近最优的.     

Exact inference for the two-parameter exponential distribution under Type-II hybrid censoring

Epstein (1954) introduced the Type-I hybrid censoring scheme as a mixture of Type-I and Type-II censoring schemes. Childs et al. (2003) introduced the Type-II hybrid censoring scheme as an alternative to Type-I hybrid censoring scheme, and provided the exact distribution of the maximum likelihood estimator of the mean of a one-parameter exponential distribution based on Type-II hybrid censored samples. The associated confidence interval also has been provided. The main aim of this paper is to consider a two-parameter exponential distribution, and to derive the exact distribution of the maximum likelihood estimators of the unknown parameters based on Type-II hybrid censored samples. The marginal distributions and the exact confidence intervals are also provided. The results can be used to derive the exact distribution of the maximum likelihood estimator of the percentile point, and to construct the associated confidence interval. Different methods are compared using extensive simulations and one data analysis has been performed for illustrative purposes.