Existence of Maximum Likelihood Estimates for Multi-dimensional Exponential Families

This paper deals with the existence of maximum likelihood estimators for multi-dimensional exponential families, including curved exponential families. It first gives an algorithm for determining the MLE from the data. Then it establishes that when the parameter set is either open or relatively closed in the natural parameter set, the MLE of the parameter exists in the sense of Hoffmann-Jorgensen.     

Laplace Approximations for Natural Exponential Families with Cuts

Standard and fully exponential form Laplace approximations to marginal densities are described and conditions under which these give exact answers are investigated. A general result is obtained and is subsequently applied in the case of natural exponential families with cuts, in order to derive the marginal posterior density of the mean parameter corresponding to the cut, the canonical parameter corresponding to the complement of the cut and transformations of these. Important cases of families for which a cut exists and the approximations are exact are presented as examples     

Conjugate Priors Represent Strong Pre-Experimental Assumptions

Abstract.  It is well known that Jeffreys' prior is asymptotically least favorable under the entropy risk, i.e. it asymptotically maximizes the mutual information between the sample and the parameter. However, in this paper we show that the prior that minimizes (subject to certain constraints) the mutual information between the sample and the parameter is natural conjugate when the model belongs to a natural exponential family. A conjugate prior can thus be regarded as maximally informative in the sense that it minimizes the weight of the observations on inferences about the parameter; in other words, the expected relative entropy between prior and posterior is minimized when a conjugate prior is used.     

Conditionally Reducible Natural Exponential Families and Enriched Conjugate Priors

Consider a standard conjugate family of prior distributions for a vector-parameter indexing an exponential family. Two distinct model parameterizations may well lead to standard conjugate families which are not consistent, i.e. one family cannot be derived from the other by the usual change-of-variable technique. This raises the problem of finding suitable parameterizations that may lead to enriched conjugate families which are more flexible than the traditional ones. The previous remark motivates the definition of a new property for an exponential family, named conditional reducibility. Features of conditionally-reducible natural exponential families are investigated thoroughly. In particular, we relate this new property to the notion of cut, and show that conditionally-reducible families admit a reparameterization in terms of a vector having likelihood-independent components. A general methodology to obtain enriched conjugate distributions for conditionally-reducible families is described in detail, generalizing previous works and more recent contributions in the area. The theory is illustrated with reference to natural exponential families having simple quadratic variance function.     

Independence Structure of Natural Conjugate Densities to Exponential Families and the Gibbs' Sampler

In this paper the independence between a block of natural parameters and the complementary block of mean value parameters holding for densities which are natural conjugate to some regular exponential families is used to design in a convenient way a Gibbs' sampler with block updates. Even when the densities of interest are obtained by conditioning to zero a block of natural parameters in a density conjugate to a larger "saturated" model, the updates require only the computation of marginal distributions under the "unconditional" density. For exponential families which are closed under marginalization, including both the zero mean Gaussian family and the cross-classified Bernoulli family such an implementation of the Gibbs' sampler can be seen as an Iterative Proportional Fitting algorithm with random inputs.     

A NOTE ON MULTIVARIATE LOGISTIC MODELS FOR CONTINGENCY TABLES

The log-linear model is a tool widely accepted for modelling discrete data given in a contingency table. Although its parameters reflect the interaction structure in the joint distribution of all variables, it does not give information about structures appearing in the margins of the table. This is in contrast to multivariate logistic parameters, recently introduced by Glonek & McCullagh (1995), which have as parameters the highest order log odds ratios derived from the joint table and from each marginal table. Glonek & McCullagh give the link between the cell probabilities and the multivariate logistic parameters, in an algebraic fashion. The present paper focuses on this link, showing that it is derived by general parameter transformations in exponential families. In particular, the connection between the natural, the expectation and the mixed parameterization in exponential families (Barndorff-Nielsen, 1978) is used; this also yields the derivatives of the likelihood equation and shows properties of the Fisher matrix. The paper emphasises the analysis of independence hypotheses in margins of a contingency table.     

A bayesian wls approach to generalized linear models

We use a Bayesian approach to fitting a linear regression model to transformations of the natural parameter for the exponential class of distributions. The usual Bayesian approach is to assume that a linear model exactly describes the relationship among the natural parameters. We assume only that a linear model is approximately in force. We approximate the theta-links by using a linear model obtained by minimizing the posterior expectation of a loss function.While some posterior results can be obtained analytically considerable generality follows from an exact Monte Carlo method for obtaining random samples of parameter values or functions of parameter values from their respective posterior distributions. The approach that is presented is justified for small samples, requires only one-dimensional numerical integrations, and allows for the use of regression matrices with less than full column rank. Two numerical examples are provided.     

Minimum variance unbiased estimation in natural exponential families generated by stable distributions

The class of nature exponential families generated by stable distributions has been introduced in different contexts by several authors. Tweedie (1984) and Jorgensen (1987) studied this class in the context of generalized liner models and exponential dispersion models. Bar-Lev and Enis (1986) introduced this class in the context of the property of reproducibility in natural exponential families and Hougaard (1986) found the distributions in this class to be natural candidates for applications as survival distributions in life tables for heterogeneous populations. In this note, we consider such a class in the context of minimum variance unbiased estimation. For each family in this class, we obtain an explicit expression for the uniformly minimum variance unbiased estimator for the r-th cumlant, the density function, and the reliability function.     

The use of the weighted likelihood in the natural exponential families with quadratic variance

The authors propose a weighted likelihood concept for the estimation of parameters in natural exponential families with quadratic variance. They apply the results to both simulated and real data.     

Multivariate stable exponential families and Tweedie scale

In this paper, we characterize the multivariate stable natural exponential families by a property of homogeneity of the cumulant function of some basis, and by a property of homogeneity of the variance function. We also extend the definition of a Tweedie scale to a finite dimensional space and we give a class of natural exponential families belonging to this scale on the space of symmetric matrices.     

Estimation in the generalized linear empirical bayes model using the extended quasi-likelihood

A generalized linear empirical Bayes model is developed for empirical Bayes analysis of several means in natural exponential families. A unified approach is presented for all natural exponential families with quadratic variance functions (the Normal, Poisson, Binomial, Gamma, and two others.) The hyperparameters are estimated using the extended quasi-likelihood of Nelder and Pregibon (1987), which is easily implemented via the GLIM package. The accuracy of these estimates is developed by asymptotic approximation of the variance. Two data examples are illustrated.     

Longitudinal network models and permutation-uniform Markov chains

Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the same parameter, improving data reduction. Further we show that the permutation-uniform subclass of these chains permit interpretation as an independent, identically distributed sequence on the same state space. We then apply these ideas to temporal exponential random graph models, for which permutation uniformity is well suited, and discuss mean-parameter convergence, dyadic independence, and exchangeability. Our framework facilitates our introducing a new network model; simplifies analysis of some network and autoregressive models from the literature, including by permitting closed-form expressions for maximum likelihood estimates for some models; and facilitates applying standard tools to longitudinal-network Markov chains from either asymptotics or single-observation exponential random graph models.     

Series evaluation of Tweedie exponential dispersion model densities

Exponential dispersion models, which are linear exponential families with a dispersion parameter, are the prototype response distributions for generalized linear models. The Tweedie family comprises those exponential dispersion models with power mean-variance relationships. The normal, Poisson, gamma and inverse Gaussian distributions belong to theTweedie family. Apart from these special cases, Tweedie distributions do not have density functions which can be written in closed form. Instead, the densities can be represented as infinite summations derived from series expansions. This article describes how the series expansions can be summed in an numerically efficient fashion. The usefulness of the approach is demonstrated, but full machine accuracy is shown not to be obtainable using the series expansion method for all parameter values. Derivatives of the density with respect to the dispersion parameter are also derived to facilitate maximum likelihood estimation. The methods are demonstrated on two data examples and compared with with Box-Cox transformations and extended quasi-likelihoood.     

Instability, Sensitivity, and Degeneracy of Discrete Exponential Families

In applications to dependent data, first and foremost relational data, a number of discrete exponential family models has turned out to be near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We introduce the notion of instability with an eye to characterize, detect, and penalize discrete exponential family models that are near-degenerate and problematic in terms of Markov chain Monte Carlo simulation and statistical inference. We show that unstable discrete exponential family models are characterized by excessive sensitivity and near-degeneracy. In special cases, the subset of the natural parameter space corresponding to non-degenerate distributions and mean-value parameters far from the boundary of the mean-value parameter space turns out to be a lower-dimensional subspace of the natural parameter space. These characteristics of unstable discrete exponential family models tend to obstruct Markov chain Monte Carlo simulation and statistical inference. In applications to relational data, we show that discrete exponential family models with Markov dependence tend to be unstable and that the parameter space of some curved exponential families contains unstable subsets.     

A note on the convergence rates for empirical bayes estimators of parameters in multiple-parameter exponential families

Under suitable conditions upon prior distribution, the convergence rates for empirical Bayes estimators of parameters in multi-parameter exponential families (M-PEF) are obtained. It is shown that the assumptions Tong (1996) imposed on the marginal density can be reduced. The above result can also be extended to more general forms of M-PEF. Finally, some examples which satisfy the conditions of the theorems are given.     

A computer algebra package for approximate conditional inference

This paper presents a set of REDUCE procedures that make a number of existing higher-order asymptotic results available for both theoretical and practical research. Attention has been restricted to the context of exact and approximate inference for a parameter of interest conditionally either on an ancillary statistic or on a statistic partially sufficient for the nuisance parameter. In particular, the procedures apply to regression-scale models and multiparameter exponential families. Most of them support algebraic computation as well as numerical calculation for a given data set. Examples illustrate the code.     

An extension of the generalized exponential distribution

The two-parameter generalized exponential distribution has been used recently quite extensively to analyze lifetime data. In this paper the two-parameter generalized exponential distribution has been embedded in a larger class of distributions obtained by introducing another shape parameter. Because of the additional shape parameter, more flexibility has been introduced in the family. It is observed that the new family is positively skewed, and has increasing, decreasing, unimodal and bathtub shaped hazard functions. It can be observed as a proportional reversed hazard family of distributions. This new family of distributions is analytically quite tractable and it can be used quite effectively to analyze censored data also. Analyses of two data sets are performed and the results are quite satisfactory.     

Nonnegative Unbiased Estimation of Scale Parameters and Associated Quantiles Based on a Ranked Set Sample

In this article, we are interested in estimating the scale parameter in location and scale families. It is well known that the best linear unbiased estimator (BLUE) of scale parameter based on a simple random sample (SRS) is nonnegative. However, the BLUE of scale parameter based on a ranked set sample (RSS) can assume negative values. We suggest various modifications of BLUE of scale parameter based on RSS so that the resulting estimators are unbiased as well as nonnegative. Their performances in terms of relative efficiencies are compared and some recommendations are made for normal, logistic, double exponential, two-parameter exponential and Weibull distributions. We also briefly discuss an application of the proposed nonnegative BLUE of scale parameter for quantile estimation for the above populations.     

Estimation op parameters in finite mixtures of exponential families from censored data

A general successive substitutions' scheme is developed to estimate parameters in a finite mixture of distributions from the exponential family, based on censored data. It is assumed that the data can be grouped in the first class and the number of observations in each of the remaining classes are known Examples from Poisson Exponential and Normal distributions are given A small simulation exercise has also been carried out for the mixture of two one parameter exponential population.     

Multiple-shrinkage estimators of means in exponential families

Shrinkage estimators are often obtained by adjusting the usual estimator towards a target subspace to which the true parameter might belong. However, meaningful reductions in risk below the usual estimator can typically be achieved in a very small part of the parameter space. In the multivariate-normal mean estimation problem, E. George, in a series of papers, showed how multiple-shrinkage estimators (data-weighted averages of several different shrinkage estimators) can attain substantial risk reductions in a large part of the parameter space. This paper extends the multiple-shrinkage results to the case of simultaneous estimation of the means of several one-parameter exponential families. Our results are developed by using an identity similar to that of Haff and Johnson (1986). A computer simulation is reported to indicate the magnitude of reductions in risk. Our results are also applied to the problem of how to choose appropriate component variables to combine before a suitable shrinkage estimator is considered.