Cross-entropy method for estimation of posterior expectation in Bayesian VAR models

In this article, an importance sampling (IS) method for the posterior expectation of a non linear function in a Bayesian vector autoregressive (VAR) model is developed. Most Bayesian inference problems involve the evaluation of the expectation of a function of interest, usually a non linear function of the model parameters, under the posterior distribution. Non linear functions in Bayesian VAR setting are difficult to estimate and usually require numerical methods for their evaluation. A weighted IS estimator is used for the evaluation of the posterior expectation. With the cross-entropy (CE) approach, the IS density is chosen from a specified family of densities such that the CE distance or the Kullback–Leibler divergence between the optimal IS density and the importance density is minimal. The performance of the proposed algorithm is assessed in an iterated multistep forecasting of US macroeconomic time series.     

Posterior consistency of logistic Gaussian process priors in density estimation

We establish weak and strong posterior consistency of Gaussian process priors studied by Lenk [1988. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 (402), 509–516] for density estimation. Weak consistency is related to the support of a Gaussian process in the sup-norm topology which is explicitly identified for many covariance kernels. In fact we show that this support is the space of all continuous functions when the usual covariance kernels are chosen and an appropriate prior is used on the smoothing parameters of the covariance kernel. We then show that a large class of Gaussian process priors achieve weak as well as strong posterior consistency (under some regularity conditions) at true densities that are either continuous or piecewise continuous.     

Consistency of Bernstein polynomial posteriors

A Bernstein prior is a probability measure on the space of all the distribution functions on [0, 1]. Under very general assumptions, it selects absolutely continuous distribution functions, whose densities are mixtures of known beta densities. The Bernstein prior is of interest in Bayesian nonparametric inference with continuous data. We study the consistency of the posterior from a Bernstein prior. We first show that, under mild assumptions, the posterior is weakly consistent for any distribution function P ₀ on [0, 1] with continuous and bounded Lebesgue density. With slightly stronger assumptions on the prior, the posterior is also Hellinger consistent. This implies that the predictive density from a Bernstein prior, which is a Bayesian density estimate, converges in the Hellinger sense to the true density (assuming that it is continuous and bounded). We also study a sieve maximum likelihood version of the density estimator and show that it is also Hellinger consistent under weak assumptions. When the order of the Bernstein polynomial, i.e. the number of components in the beta distribution mixture, is truncated, we show that under mild restrictions the posterior concentrates on the set of pseudotrue densities. Finally, we study the behaviour of the predictive density numerically and we also study a hybrid Bayes–maximum likelihood density estimator.     

Elicitation of multivariate prior distributions: A nonparametric Bayesian approach

In the context of Bayesian statistical analysis, elicitation is the process of formulating a prior density f(·)$f(·)$ about one or more uncertain quantities to represent a person's knowledge and beliefs. Several different methods of eliciting prior distributions for one unknown parameter have been proposed. However, there are relatively few methods for specifying a multivariate prior distribution and most are just applicable to specific classes of problems and/or based on restrictive conditions, such as independence of variables. Besides, many of these procedures require the elicitation of variances and correlations, and sometimes elicitation of hyperparameters which are difficult for experts to specify in practice. Garthwaite et al. (2005) discuss the different methods proposed in the literature and the difficulties of eliciting multivariate prior distributions. We describe a flexible method of eliciting multivariate prior distributions applicable to a wide class of practical problems. Our approach does not assume a parametric form for the unknown prior density f(·)$f(·)$, instead we use nonparametric Bayesian inference, modelling f(·)$f(·)$ by a Gaussian process prior distribution. The expert is then asked to specify certain summaries of his/her distribution, such as the mean, mode, marginal quantiles and a small number of joint probabilities. The analyst receives that information, treating it as a data set D   with which to update his/her prior beliefs to obtain the posterior distribution for f(·)$f(·)$. Theoretical properties of joint and marginal priors are derived and numerical illustrations to demonstrate our approach are given.     

A generalized skew two-piece skew-elliptical distribution

We present a new generalized family of skew two-piece skew-elliptical (GSTPSE) models and derive some its statistical properties. It is shown that the new family of distribution may be written as a mixture of generalized skew elliptical distributions. Also, a new representation theorem for a special case of GSTPSE-distribution is given. Next, we will focus on t kernel density and prove that it is a scale mixture of the generalized skew two-piece skew normal distributions. An explicit expression for the central moments as well as a recurrence relations for its cumulative distribution function and density are obtained. Since, this special case is a uni-/bimodal distribution, a sufficient condition for each cases is given. A real data set on heights of Australian females athletes is analysed. Finally, some concluding remarks and open problems are discussed.     

Models for Extremal Dependence Derived from Skew‐symmetric Families

Skew‐symmetric families of distributions such as the skew‐normal and skew‐t represent supersets of the normal and t distributions, and they exhibit richer classes of extremal behaviour. By defining a non‐stationary skew‐normal process, which allows the easy handling of positive definite, non‐stationary covariance functions, we derive a new family of max‐stable processes – the extremal skew‐t process. This process is a superset of non‐stationary processes that include the stationary extremal‐t processes. We provide the spectral representation and the resulting angular densities of the extremal skew‐t process and illustrate its practical implementation.     

Calculation of marginal densities for parameters of multinomial distributions

The full Bayesian analysis of multinomial data using informative and flexible prior distributions has, in the past, been restricted by the technical problems involved in performing the numerical integrations required to obtain marginal densities for parameters and other functions thereof. In this paper it is shown that Gibbs sampling is suitable for obtaining accurate approximations to marginal densities for a large and flexible family of posterior distributions—the family. The method is illustrated with a three-way contingency table. Two alternative Monte Carlo strategies are also discussed.     

Quasi Bayesian likelihood

If the form of the distribution of data is unknown, the Bayesian method fails in the parametric inference because there is no posterior distribution of the parameter. In this paper, a theoretical framework of Bayesian likelihood is introduced via the Hilbert space method, which is free of the distributions of data and the parameter. The posterior distribution and posterior score function based on given inner products are defined and, consequently, the quasi posterior distribution and quasi posterior score function are derived, respectively, as the projections of the posterior distribution and posterior score function onto the space spanned by given estimating functions. In the space spanned by data, particularly, an explicit representation for the quasi posterior score function is obtained, which can be derived as a projection of the true posterior score function onto this space. The methods of constructing conservative quasi posterior score and quasi posterior log-likelihood are proposed. Some examples are given to illustrate the theoretical results. As an application, the quasi posterior distribution functions are used to select variables for generalized linear models. It is proved that, for linear models, the variable selections via quasi posterior distribution functions are equivalent to the variable selections via the penalized residual sum of squares or regression sum of squares.     

Objective Bayesian Analysis of Skew‐t Distributions

Abstract. We study the Jeffreys prior and its properties for the shape parameter of univariate skew‐t distributions with linear and nonlinear Student's t skewing functions. In both cases, we show that the resulting priors for the shape parameter are symmetric around zero and proper. Moreover, we propose a Student's t approximation of the Jeffreys prior that makes an objective Bayesian analysis easy to perform. We carry out a Monte Carlo simulation study that demonstrates an overall better behaviour of the maximum a posteriori estimator compared with the maximum likelihood estimator. We also compare the frequentist coverage of the credible intervals based on the Jeffreys prior and its approximation and show that they are similar. We further discuss location‐scale models under scale mixtures of skew‐normal distributions and show some conditions for the existence of the posterior distribution and its moments. Finally, we present three numerical examples to illustrate the implications of our results on inference for skew‐t distributions.     

Forecasting future values of changing sequences

Sequences of independent random variables are observed and on the basis of these observations future values of the process are forecast. The Bayesian predictive density of k future observations for normal, exponential, and binomial sequences which change exactly once are analyzed for several cases. It is seen that the Bayesian predictive densities are mixtures of standard probability distributions. For example, with normal sequences the Bayesian predictive density is a mixture of either normal or t-distributions, depending on whether or not the common variance is known. The mixing probabilities are the same as those occurring in the corresponding posterior distribution of the mean(s) of the sequence. The predictive mass function of the number of future successes that will occur in a changing Bernoulli sequence is computed and point and interval predictors are illustrated.     

On Rates of Convergence for Bayesian Density Estimation

Abstract.  We consider the problem of estimating a compactly supported density taking a Bayesian nonparametric approach. We define a Dirichlet mixture prior that, while selecting piecewise constant densities, has full support on the Hellinger metric space of all commonly dominated probability measures on a known bounded interval. We derive pointwise rates of convergence for the posterior expected density by studying the speed at which the posterior mass accumulates on shrinking Hellinger neighbourhoods of the sampling density. If the data are sampled from a strictly positive, α -Hölderian density, with α  ∈ ( 0,1] , then the optimal convergence rate n^{− α / (2 α +1)} is obtained up to a logarithmic factor. Smoothing histograms by polygons, a continuous piecewise linear estimator is obtained that for twice continuously differentiable, strictly positive densities satisfying boundary conditions attains a rate comparable up to a logarithmic factor to the convergence rate n ^−4/5 for integrated mean squared error of kernel type density estimators.     

A new class of multivariate skew distributions with applications to bayesian regression models

Abstract: The authors develop a new class of distributions by introducing skewness in multivariate elliptically symmetric distributions. The class, which is obtained by using transformation and conditioning, contains many standard families including the multivariate skew‐normal and t distributions. The authors obtain analytical forms of the densities and study distributional properties. They give practical applications in Bayesian regression models and results on the existence of the posterior distributions and moments under improper priors for the regression coefficients. They illustrate their methods using practical examples.     

Optimal control designs using predicting densities for the multivariate linear model

We provide an application of a variety of predicting densities to quality control involving multivariate normal linear models. We produce optimal control designs for single muleivaiiate future observations using predicting densities employing estimative, profile likelihood, Hinkley-Lauritzen, Butler, Bayesian, and Parametric Bootstrap methodologies. The decision-theoretic optimality criterion is an intuitively appealing quadratic consumer-producer risk function. The optimal control design arising from an optimal Kullback-Leibler frequentist prediction density is shown to coincide with that arising from an optimal Kullback-Leibler Bayesian predictive density. An example involving EVOP is provided to illustrate the methodology and to raise questions concerning the relative merics of the variety of predictive approaches in the quality control context.     

Univariate Bayesian nonparametric mixture modeling with unimodal kernels

Within the context of mixture modeling, the normal distribution is typically used as the components distribution. However, if a cluster is skewed or heavy tailed, then the normal distribution will be inefficient and many may be needed to model a single cluster. In this paper, we present an attempt to solve this problem. We define a cluster, in the absence of further information, to be a group of data which can be modeled by a unimodal density function. Hence, our intention is to use a family of univariate distribution functions, to replace the normal, for which the only constraint is unimodality. With this aim, we devise a new family of nonparametric unimodal distributions, which has large support over the space of univariate unimodal distributions. The difficult aspect of the Bayesian model is to construct a suitable MCMC algorithm to sample from the correct posterior distribution. The key will be the introduction of strategic latent variables and the use of the Product Space view of Reversible Jump methodology.     

A class of weighted normal distributions and its variants useful for inequality constrained analysis

This article develops a class of the weighted normal distributions for which the probability density function has the form of a product of a normal density and a weight function. The class constitutes marginal distributions obtained from various kinds of doubly truncated bivariate normal distributions. This class of distributions strictly includes the normal, skew–normal and two-piece skew–normal and is useful for selection modelling and inequality constrained normal mean analysis. Some distributional properties and Bayesian perspectives of the class are given. Probabilistic representation of the distributions is also given. The representation is shown to be straightforward to specify distribution and to implement computation, with output readily adapted for required analysis. Necessary theories and illustrative examples are provided.     

On a Posterior Predictive Density Sample Size Criterion

Abstract.  Let Ω be a space of densities with respect to some σ -finite measure μ and let Π be a prior distribution having support Ω with respect to some suitable topology. Conditional on f , let X ⁿ  = ( X ₁ ,…, X _n ) be an independent and identically distributed sample of size n from f . This paper introduces a Bayesian non-parametric criterion for sample size determination which is based on the integrated squared distance between posterior predictive densities. An expression for the sample size is obtained when the prior is a Dirichlet mixture of normal densities.     

Gamma distribution and extensions by using pathway idea

Extended forms of gamma and inverse Gaussian densities are considered. Pathway model of Mathai (Linear Algebra Appl 396:317–328, 2005) is utilized for the extension. The Laplace transform of this extended function is evaluated, thereby the moment generating functions in this general class can be obtained. Some Bayesian results are derived for the extended gamma family of densities. Distributions of products and ratios of the extended variables and computable series representations are discussed.     

The t statistic of the skew elliptical distributions

This paper studies the properties of the generalized t statistic when the sample comes from the skew elliptical distributions. Several forms of the probability density functions are obtained. The robustness of the one-sided t test in the family of the skew normal distributions is investigated.     

Sufficient conditions for Bayesian consistency

We introduce the Hausdorff α$\alpha$-entropy to study the strong Hellinger consistency of posterior distributions. We obtain general Bayesian consistency theorems which extend the well-known results of Barron et al. [1999. The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27, 536–561] and Ghosal et al. [1999. Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27, 143–158] and Walker [2004. New approaches to Bayesian consistency. Ann. Statist. 32, 2028–2043]. As an application we strengthen previous results on Bayesian consistency of the (normal) mixture models.     

A bayesian analysis of trend determination in economic time series

In this paper we provide a comprehensive Bayesian posterior analysis of trend determination in general autoregressive models. Multiple lag autoregressive models with fitted drifts and time trends as well as models that allow for certain types of structural change in the deterministic components are considered. We utilize a modified information matrix-based prior that accommodates stochastic nonstationarity, takes into account the interactions between long-run and short-run dynamics and controls the degree of stochastic nonstationarity permitted. We derive analytic posterior densities for all of the trend determining parameters via the Laplace approximation to multivariate integrals. We also address the sampling properties of our posteriors under alternative data generating processes by simulation methods. We apply our Bayesian techniques to the Nelson-Plosser macroeconomic data and various stock price and dividend data. Contrary to DeJong and Whiteman (1989a,b,c), we do not find that the data overwhelmingly favor the existence of deterministic trends over stochastic trends. In addition, we find evidence supporting Perron's (1989) view that some of the Nelson and Plosser data are best construed as trend stationary with a change in the trend function occurring at 1929.