A Condition to Obtain the Same Decision in the Homogeneity Testing Problem from the Frequentist and Bayesian Point of View

We develop a Bayesian procedure for the homogeneity testing problem of r populations using r × s contingency tables. The posterior probability of the homogeneity null hypothesis is calculated using a mixed prior distribution. The methodology consists of choosing an appropriate value of π₀ for the mass assigned to the null and spreading the remainder, 1 ? π₀, over the alternative according to a density function. With this method, a theorem which shows when the same conclusion is reached from both frequentist and Bayesian points of view is obtained. A sufficient condition under which the p-value is less than a value α and the posterior probability is also less than 0.5 is provided.     

A Bayesian analysis for the Wilcoxon signed-rank statistic

A Bayesian analysis is provided for the Wilcoxon signed-rank statistic (T⁺). The Bayesian analysis is based on a sign-bias parameter φ on the (0, 1) interval. For the case of a uniform prior probability distribution for φ and for small sample sizes (i.e., 6 ? n ? 25), values for the statistic T⁺ are computed that enable probabilistic statements about φ. For larger sample sizes, approximations are provided for the asymptotic likelihood function P(T⁺|φ) as well as for the posterior distribution P(φ|T⁺). Power analyses are examined both for properly specified Gaussian sampling and for misspecified non Gaussian models. The new Bayesian metric has high power efficiency in the range of 0.9–1 relative to a standard t test when there is Gaussian sampling. But if the sampling is from an unknown and misspecified distribution, then the new statistic still has high power; in some cases, the power can be higher than the t test (especially for probability mixtures and heavy-tailed distributions). The new Bayesian analysis is thus a useful and robust method for applications where the usual parametric assumptions are questionable. These properties further enable a way to do a generic Bayesian analysis for many non Gaussian distributions that currently lack a formal Bayesian model.     

Bayesian Identification of Moving Average Models

This study approaches the Bayesian identification of moving average processes using an approximate likelihood function and a normal gamma prior density. The marginal posterior probability mass function of the model order is developed in a convenient form. Then one may investigate the posterior probabilities over the grid of the order and choose the order with the highest probability to solve the identification problem. A comprehensive simulation study is carried out to demonstrate the performance of the proposed procedure and check its adequacy in handling the identification problem. In addition, the proposed Bayesian procedure is compared with some non Bayesian automatic techniques and another Bayesian technique. The numerical results support the adequacy of using the proposed procedure in solving the identification problem of moving average processes.     

The Bayesian and frequentist approaches to testing a one-sided hypothesis about a multivariate mean

This paper compares the Bayesian and frequentist approaches to testing a one-sided hypothesis about a multivariate mean. First, this paper proposes a simple way to assign a Bayesian posterior probability to one-sided hypotheses about a multivariate mean. The approach is to use (almost) the exact posterior probability under the assumption that the data has multivariate normal distribution, under either a conjugate prior in large samples or under a vague Jeffreys prior. This is also approximately the Bayesian posterior probability of the hypothesis based on a suitably flat Dirichlet process prior over an unknown distribution generating the data. Then, the Bayesian approach and a frequentist approach to testing the one-sided hypothesis are compared, with results that show a major difference between Bayesian reasoning and frequentist reasoning. The Bayesian posterior probability can be substantially smaller than the frequentist p-value. A class of example is given where the Bayesian posterior probability is basically 0, while the frequentist p-value is basically 1. The Bayesian posterior probability in these examples seems to be more reasonable. Other drawbacks of the frequentist p-value as a measure of whether the one-sided hypothesis is true are also discussed.     

Posterior Predictive Comparisons for the Two-sample Problem

The two-sample problem of inferring whether two random samples have equal underlying distributions is formulated within the Bayesian framework as a comparison of two posterior predictive inferences rather than as a problem of model selection. The suggested approach is argued to be particularly advantageous in problems where the objective is to evaluate evidence in support of equality, along with being robust to the priors used and being capable of handling improper priors. Our approach is contrasted with the Bayes factor in a normal setting and finally, an additional example is considered where the observed samples are realizations of Markov chains.     

A note on Bayesian estimation of traffic intensity in single-server Markovian queues

ABSTRACT

In queuing theory, a major interest of researchers is studying the behavior and formation process and analyzing the performance characteristics of queues, particularly the traffic intensity, which is defined as the ratio between the arrival rate and the service rate. How these parameters can be estimated using some statistical inferential method is the mathematical problem treated here. This article aims to obtain better Bayesian estimates for the traffic intensity of M/M/1 queues, which, in Kendall notation, stand for Markovian single-server infinity queues. The Jeffreys prior is proposed to obtain the posterior and predictive distributions of some parameters of interest. Samples are obtained through simulation and some performance characteristics are analyzed. It is observed from the Bayes factor that Jeffreys prior is competitive, among informative and non-informative prior distributions, and presents the best performance in many of the cases tested.     

A Direct Approach to Understanding Posterior Consistency of Bayesian Regression Problems

Previous approaches to establishing posterior consistency of Bayesian regression problems have used general theorems that involve verifying sufficient conditions for posterior consistency. In this article, we consider a direct approach by computing the posterior density explicitly and evaluating its asymptotic behavior. For this purpose, we deal with a sample size dependent prior based on a truncated regression function with increasing sample size, and evaluate the asymptotic properties of the resulting posterior. Based on a concept called posterior density consistency, we attempt to understand posterior consistency. As an application, we illustrate that the posterior density of an orthogonal semiparametric regression model is consistent.     

Stochastic matching pursuit for Bayesian variable selection

This article proposes a stochastic version of the matching pursuit algorithm for Bayesian variable selection in linear regression. In the Bayesian formulation, the prior distribution of each regression coefficient is assumed to be a mixture of a point mass at 0 and a normal distribution with zero mean and a large variance. The proposed stochastic matching pursuit algorithm is designed for sampling from the posterior distribution of the coefficients for the purpose of variable selection. The proposed algorithm can be considered a modification of the componentwise Gibbs sampler. In the componentwise Gibbs sampler, the variables are visited by a random or a systematic scan. In the stochastic matching pursuit algorithm, the variables that better align with the current residual vector are given higher probabilities of being visited. The proposed algorithm combines the efficiency of the matching pursuit algorithm and the Bayesian formulation with well defined prior distributions on coefficients. Several simulated examples of small n and large p are used to illustrate the algorithm. These examples show that the algorithm is efficient for screening and selecting variables.     

A MCMC algorithm to fit a general exchangeable model

Consider the exchangeable Bayesian hierarchical model where observations y_i are independently distributed from sampling densities with unknown means, the means µi, are a random sample from a distribution g, and the parameters of g are assigned a known distribution h. A simple algorithm is presented for summarizing the posterior distribution based on Gibbs sampling and the Metropolis algorithm. The software program Matlab is used to implement the algorithm and provide a graphical output analysis. An binomial example is used to illustrate the flexibility of modeling possible using this algorithm. Methods of model checking and extensions to hierarchical regression modeling are discussed.     

Bayesian analysis of mixture modelling using the multivariate t distribution

A finite mixture model using the multivariate t distribution has been shown as a robust extension of normal mixtures. In this paper, we present a Bayesian approach for inference about parameters of t-mixture models. The specifications of prior distributions are weakly informative to avoid causing nonintegrable posterior distributions. We present two efficient EM-type algorithms for computing the joint posterior mode with the observed data and an incomplete future vector as the sample. Markov chain Monte Carlo sampling schemes are also developed to obtain the target posterior distribution of parameters. The advantages of Bayesian approach over the maximum likelihood method are demonstrated via a set of real data.     

Bayesian Identification of Seasonal Autoregressive Models

Abstract

A very important and essential phase of time series analysis is identifying the model orders. This article develops an approximate Bayesian procedure to identify the orders of seasonal autoregressive processes. Using either a normal-gamma prior density or a noninformative prior, which is combined with an approximate conditional likelihood function, the foundation of the proposed technique is to derive the joint posterior mass function of the model orders in an easy form. Then one may inspect the posterior mass function and choose the orders with the largest posterior probability to be the suitable orders of the time series being analyzed. A simulation study, with different priors mass functions, is carried out to test the adequacy of the proposed technique and compare it with some non-Bayesian automatic criteria. The analysis of the numerical results supports the adequacy of the proposed technique in identifying the orders of the autoregressive processes.     

A note on fiducial model averaging as an alternative to checking Bayesian and frequentist models

Just as frequentist hypothesis tests have been developed to check model assumptions, prior predictive p-values and other Bayesian p-values check prior distributions as well as other model assumptions. These model checks not only suffer from the usual threshold dependence of p-values, but also from the suppression of model uncertainty in subsequent inference. One solution is to transform Bayesian and frequentist p-values for model assessment into a fiducial distribution across the models. Averaging the Bayesian or frequentist posterior distributions with respect to the fiducial distribution can reproduce results from Bayesian model averaging or classical fiducial inference.     

Selection of Multivariate Stochastic Volatility Models via Bayesian Stochastic Search

We propose a Bayesian stochastic search approach to selecting restrictions on multivariate regression models where the errors exhibit deterministic or stochastic conditional volatilities. We develop a Markov chain Monte Carlo (MCMC) algorithm that generates posterior restrictions on the regression coefficients and Cholesky decompositions of the covariance matrix of the errors. Numerical simulations with artificially generated data show that the proposed method is effective in selecting the data-generating model restrictions and improving the forecasting performance of the model. Applying the method to daily foreign exchange rate data, we conduct stochastic search on a VAR model with stochastic conditional volatilities.     

An Objective Bayesian Criterion to Determine Model Prior Probabilities

We discuss the problem of selecting among alternative parametric models within the Bayesian framework. For model selection problems, which involve non‐nested models, the common objective choice of a prior on the model space is the uniform distribution. The same applies to situations where the models are nested. It is our contention that assigning equal prior probability to each model is over simplistic. Consequently, we introduce a novel approach to objectively determine model prior probabilities, conditionally, on the choice of priors for the parameters of the models. The idea is based on the notion of the worth of having each model within the selection process. At the heart of the procedure is the measure of this worth using the Kullback–Leibler divergence between densities from different models.     

A random weighting approach for posterior distributions

ABSTRACT

In Bayesian theory, calculating a posterior probability distribution is highly important but typically difficult. Therefore, some methods have been proposed to deal with such problem, among which, the most popular one is the asymptotic expansions of posterior distributions. In this paper, we propose an alternative approach, named a random weighting method, for scaled posterior distributions, and give an ideal convergence rate, o(n^{( ? 1/2)}), which serves as the theoretical guarantee for methods of numerical simulations.     

Alternatives to post‐processing posterior predictive p values

The posterior predictive p value (ppp) was invented as a Bayesian counterpart to classical p values. The methodology can be applied to discrepancy measures involving both data and parameters and can, hence, be targeted to check for various modeling assumptions. The interpretation can, however, be difficult since the distribution of the ppp value under modeling assumptions varies substantially between cases. A calibration procedure has been suggested, treating the ppp value as a test statistic in a prior predictive test. In this paper, we suggest that a prior predictive test may instead be based on the expected posterior discrepancy, which is somewhat simpler, both conceptually and computationally. Since both these methods require the simulation of a large posterior parameter sample for each of an equally large prior predictive data sample, we furthermore suggest to look for ways to match the given discrepancy by a computation‐saving conflict measure. This approach is also based on simulations but only requires sampling from two different distributions representing two contrasting information sources about a model parameter. The conflict measure methodology is also more flexible in that it handles non‐informative priors without difficulty. We compare the different approaches theoretically in some simple models and in a more complex applied example.     

Bayesian analysis of a linear model involving structural changes in either regression parameters or disturbances precision

ABSTRACT

The present paper considers the Bayesian analysis of a linear regression model involving structural change, which may occur either due to shift in disturbances precision or due to shift in regression parameters. The posterior density for the regression parameter has been derived and posterior odds ratio for testing the hypothesis that structural change is due to shift in disturbances precision against the alternative that the change is due to shift in regression parameters has been obtained. The findings of a numerical simulation have been presented. The proposed model has been applied to RBI data set on corporate sector.     

A Bayesian analysis for the multivariate point null testing problem

A Bayesian test for the point null testing problem in the multivariate case is developed. A procedure to get the mixed distribution using the prior density is suggested. For comparisons between the Bayesian and classical approaches, lower bounds on posterior probabilities of the null hypothesis, over some reasonable classes of prior distributions, are computed and compared with the p-value of the classical test. With our procedure, a better approximation is obtained because the p-value is in the range of the Bayesian measures of evidence.     

A non-iterative Bayesian sampling algorithm for censored Student-t linear regression models

ABSTRACT

In this paper, we consider an effective Bayesian inference for censored Student-t linear regression model, which is a robust alternative to the usual censored Normal linear regression model. Based on the mixture representation of the Student-t distribution, we propose a non-iterative Bayesian sampling procedure to obtain independently and identically distributed samples approximately from the observed posterior distributions, which is different from the iterative Markov Chain Monte Carlo algorithm. We conduct model selection and influential analysis using the posterior samples to choose the best fitted model and to detect latent outliers. We illustrate the performance of the procedure through simulation studies, and finally, we apply the procedure to two real data sets, one is the insulation life data with right censoring and the other is the wage rates data with left censoring, and we get some interesting results.     

Adaptive smoothing in associated kernel discrete functions estimation using Bayesian approach

This paper demonstrates that cross-validation (CV) and Bayesian adaptive bandwidth selection can be applied in the estimation of associated kernel discrete functions. This idea is originally proposed by Brewer [A Bayesian model for local smoothing in kernel density estimation, Stat. Comput. 10 (2000), pp. 299–309] to derive variable bandwidths in adaptive kernel density estimation. Our approach considers the adaptive binomial kernel estimator and treats the variable bandwidths as parameters with beta prior distribution. The best variable bandwidth selector is estimated by the posterior mean in the Bayesian sense under squared error loss. Monte Carlo simulations are conducted to examine the performance of the proposed Bayesian adaptive approach in comparison with the performance of the Asymptotic mean integrated squared error estimator and CV technique for selecting a global (fixed) bandwidth proposed in Kokonendji and Senga Kiessé [Discrete associated kernels method and extensions, Stat. Methodol. 8 (2011), pp. 497–516]. The Bayesian adaptive bandwidth estimator performs better than the global bandwidth, in particular for small and moderate sample sizes.