Stochastic approximation Monte Carlo Gibbs sampling for structural change inference in a Bayesian heteroscedastic time series model

We consider a Bayesian deterministically trending dynamic time series model with heteroscedastic error variance, in which there exist multiple structural changes in level, trend and error variance, but the number of change-points and the timings are unknown. For a Bayesian analysis, a truncated Poisson prior and conjugate priors are used for the number of change-points and the distributional parameters, respectively. To identify the best model and estimate the model parameters simultaneously, we propose a new method by sequentially making use of the Gibbs sampler in conjunction with stochastic approximation Monte Carlo simulations, as an adaptive Monte Carlo algorithm. The numerical results are in favor of our method in terms of the quality of estimates.     

A Comparison of Bayesian Models of Heteroscedasticity in Nested Normal Data

We consider the fitting of a Bayesian model to grouped data in which observations are assumed normally distributed around group means that are themselves normally distributed, and consider several alternatives for accommodating the possibility of heteroscedasticity within the data. We consider the case where the underlying distribution of the variances is unknown, and investigate several candidate prior distributions for those variances. In each case, the parameters of the candidate priors (the hyperparameters) are themselves given uninformative priors (hyperpriors). The most mathematically convenient model for the group variances is to assign them inverse gamma distributed priors, the inverse gamma distribution being the conjugate prior distribution for the unknown variance of a normal population. We demonstrate that for a wide class of underlying distributions of the group variances, a model that assigns the variances an inverse gamma-distributed prior displays favorable goodness-of-fit properties relative to other candidate priors, and hence may be used as standard for modeling such data. This allows us to take advantage of the elegant mathematical property of prior conjugacy in a wide variety of contexts without compromising model fitness. We test our findings on nine real world publicly available datasets from different domains, and on a wide range of artificially generated datasets.     

Intrinsic Priors for Testing Two Normal Means with Intrinsic Bayes Factors

ABSTRACT

In this article we consider the problem of comparing two normal means with unknown common variance using a Bayesian approach. Conventional Bayes factors with improper non informative priors are not well defined. The intrinsic Bayes factors are used to overcome such a difficulty. We derive intrinsic priors whose Bayes factors are asymptotically equivalent to the corresponding intrinsic Bayes factors. We illustrate our results with numerical examples.     

Natural (Non‐)Informative Priors for Skew‐symmetric Distributions

In this paper, we present an innovative method for constructing proper priors for the skewness (shape) parameter in the skew‐symmetric family of distributions. The proposed method is based on assigning a prior distribution on the perturbation effect of the shape parameter, which is quantified in terms of the total variation distance. We discuss strategies to translate prior beliefs about the asymmetry of the data into an informative prior distribution of this class. We show via a Monte Carlo simulation study that our non‐informative priors induce posterior distributions with good frequentist properties, similar to those of the Jeffreys prior. Our informative priors yield better results than their competitors from the literature. We also propose a scale‐invariant and location‐invariant prior structure for models with unknown location and scale parameters and provide sufficient conditions for the propriety of the corresponding posterior distribution. Illustrative examples are presented using simulated and real data.     

Bayesian Analysis of a Generalized Poisson Distribution

A generalized form of the Poisson Distribution with two parameters will be estimated by the Bayesian technique. When one of the parameters is known, several important parametric functions will be estimated and a numerical comparison with estimates obtained by the methods of maximum likelihood and unbiased minimum variance will be drawn. The simplicity of the posterior distribution of the unknown parameter enables us to construct exact probability intervals, and to devise a statistic to test the homogeneity of several populations. When the two parameters are unknown, dependent priors are being considered. Although the posterior distributions are sensitive to the choice of the prior, the posterior estimates are very stable and we use the Pearson system of curves to construct approximate posterior confidence limits for the parameters.     

Consistency of Posterior Distributions for Heteroscedastic Nonparametric Regression Models

In this article, we consider Bayesian inferences for the heteroscedastic nonparametric regression models, when both the mean function and variance function are unknown. We demonstrated consistency of posterior distributions for this model using priors induced by B-splines expansion, treating both random and deterministic covariates in a uniform manner.     

Bayesian estimation of the generalized lognormal distribution using objective priors

The generalized lognormal distribution plays an important role in analysing data from different life testing experiments. In this paper, we consider Bayesian analysis of this distribution using various objective priors for the model parameters. Specifically, we derive expressions for the Jeffreys-type priors, the reference priors with different group orderings of the parameters, and the first-order matching priors. We also study the properties of the posterior distributions of the parameters under these improper priors. It is shown that only two of them result in proper posterior distributions. Numerical simulation studies are conducted to compare the performances of the Bayesian estimators under the considered priors and the maximum likelihood estimates. Finally, a real-data application is also provided for illustrative purposes.     

An objective Bayesian analysis of the two-parameter half-logistic distribution based on progressively type-II censored samples

This paper develops an objective Bayesian analysis method for estimating unknown parameters of the half-logistic distribution when a sample is available from the progressively Type-II censoring scheme. Noninformative priors such as Jeffreys and reference priors are derived. In addition, derived priors are checked to determine whether they satisfy probability-matching criteria. The Metropolis–Hasting algorithm is applied to generate Markov chain Monte Carlo samples from these posterior density functions because marginal posterior density functions of each parameter cannot be expressed in an explicit form. Monte Carlo simulations are conducted to investigate frequentist properties of estimated models under noninformative priors. For illustration purposes, a real data set is presented, and the quality of models under noninformative priors is evaluated through posterior predictive checking.     

State space mixed models for longitudinal observations with binary and binomial responses

We propose a new class of state space models for longitudinal discrete response data where the observation equation is specified in an additive form involving both deterministic and random linear predictors. These models allow us to explicitly address the effects of trend, seasonal or other time-varying covariates while preserving the power of state space models in modeling serial dependence in the data. We develop a Markov chain Monte Carlo algorithm to carry out statistical inference for models with binary and binomial responses, in which we invoke de Jong and Shephard’s (Biometrika 82(2):339–350, 1995) simulation smoother to establish an efficient sampling procedure for the state variables. To quantify and control the sensitivity of posteriors on the priors of variance parameters, we add a signal-to-noise ratio type parameter in the specification of these priors. Finally, we illustrate the applicability of the proposed state space mixed models for longitudinal binomial response data in both simulation studies and data examples.     

Bayes estimation of a langevin mean direction

A Langevin distribution with two parameters (mean direction and concentration parameter) has been extensively used for modeling and analyzing problems related to directional data. In this article, we examine the estimation problem for the mean direction. Bayes estimators are derived with respect to a conjugate as well as the Jeffreys’ priors. Further in case of unknown concentration parameter, other priors are also chosen. An extensive analysis of risk behavior of Bayes estimators is carried out with the help of simulations.     

Empirical bayes methods for a two-way multiplicative-interaction model

Empirical Bayes methods are used to estimate cell probabi-lities under a multiplicative-Interaction model for a two-way contingency table. The methods assign uniform and normal priors with unknown variances to the main effects and the separable scores. A priori the analysis assumes exchangeability of sets of parameters. The unknown variance components are estimated empirically from the data via the EM algorithm as discussed by Laird (1978)and Dempster, Laird and Rubin (1977). An example Is Included.     

On minimax designs when there are two candidate models

This work is motivated by the need to find experimental designs which are robust under different model assumptions. We measure robustness by calculating a measure of design efficiency with respect to a design optimality criterion and say that a design is robust if it is reasonably efficient under different model scenarios. We discuss two design criteria and an algorithm which can be used to obtain robust designs. The first criterion employs a Bayesian-type approach by putting a prior or weight on each candidate model and possibly priors on the corresponding model parameters. We define the first criterion as the expected value of the design efficiency over the priors. The second design criterion we study is the minimax design which minimizes the worst value of a design criterion over all candidate models. We establish conditions when these two criteria are equivalent when there are two candidate models. We apply our findings to the area of accelerated life testing and perform sensitivity analysis of designs with respect to priors and misspecification of planning values.     

Objective Bayesian inference for the ratio of the scale parameters of two Weibull distributions

The Weibull distribution is widely used due to its versatility and relative simplicity. In our paper, the non informative priors for the ratio of the scale parameters of two Weibull models are provided. The asymptotic matching of coverage probabilities of Bayesian credible intervals is considered, with the corresponding frequentist coverage probabilities. We developed the various priors for the ratio of two scale parameters using the following matching criteria: quantile matching, matching of distribution function, highest posterior density matching, and inversion of test statistics. One particular prior, which meets all the matching criteria, is found. Next, we derive the reference priors for groups of ordering. We see that all the reference priors satisfy a first-order matching criterion and that the one-at-a-time reference prior is a second-order matching prior. A simulation study is performed and an example given.     

Bayesian LASSO-Regularized quantile regression for linear regression models with autoregressive errors

Quantile regression (QR) is a natural alternative for depicting the impact of covariates on the conditional distributions of a outcome variable instead of the mean. In this paper, we investigate Bayesian regularized QR for the linear models with autoregressive errors. LASSO-penalized type priors are forced on regression coefficients and autoregressive parameters of the model. Gibbs sampler algorithm is employed to draw the full posterior distributions of unknown parameters. Finally, the proposed procedures are illustrated by some simulation studies and applied to a real data analysis of the electricity consumption.     

Bayesian expectile regression with asymmetric normal distribution

In this paper, we adopt the Bayesian approach to expectile regression employing a likelihood function that is based on an asymmetric normal distribution. We demonstrate that improper uniform priors for the unknown model parameters yield a proper joint posterior. Three simulated data sets were generated to evaluate the proposed method which show that Bayesian expectile regression performs well and has different characteristics comparing with Bayesian quantile regression. We also apply this approach into two real data analysis.     

Variance Reduction in Smoothing Splines

Abstract.  We develop a variance reduction method for smoothing splines. For a given point of estimation, we define a variance-reduced spline estimate as a linear combination of classical spline estimates at three nearby points. We first develop a variance reduction method for spline estimators in univariate regression models. We then develop an analogous variance reduction method for spline estimators in clustered/longitudinal models. Simulation studies are performed which demonstrate the efficacy of our variance reduction methods in finite sample settings. Finally, a real data analysis with the motorcycle data set is performed. Here we consider variance estimation and generate 95% pointwise confidence intervals for the unknown regression function.     

Conservative prior distributions for variance parameters in hierarchical models

Bayesian hierarchical models typically involve specifying prior distributions for one or more variance components. This is rather removed from the observed data, so specification based on expert knowledge can be difficult. While there are suggestions for “default” priors in the literature, often a conditionally conjugate inverse‐gamma specification is used, despite documented drawbacks of this choice. The authors suggest “conservative” prior distributions for variance components, which deliberately give more weight to smaller values. These are appropriate for investigators who are skeptical about the presence of variability in the second‐stage parameters (random effects) and want to particularly guard against inferring more structure than is really present. The suggested priors readily adapt to various hierarchical modelling settings, such as fitting smooth curves, modelling spatial variation and combining data from multiple sites.     

Sequential parameter learning and filtering in structured autoregressive state-space models

We present particle-based algorithms for sequential filtering and parameter learning in state-space autoregressive (AR) models with structured priors. Non-conjugate priors are specified on the AR coefficients at the system level by imposing uniform or truncated normal priors on the moduli and wavelengths of the reciprocal roots of the AR characteristic polynomial. Sequential Monte Carlo algorithms are considered and implemented for on-line filtering and parameter learning within this modeling framework. More specifically, three SMC approaches are considered and compared by applying them to data simulated from different state-space AR models. An analysis of a human electroencephalogram signal is also presented to illustrate the use of the structured state-space AR models in describing biomedical signals.     

Bayesian estimation for first-order autoregressive model with explanatory variables

In this article, we develop a Bayesian analysis in autoregressive model with explanatory variables. When σ² is known, we consider a normal prior and give the Bayesian estimator for the regression coefficients of the model. For the case σ² is unknown, another Bayesian estimator is given for all unknown parameters under a conjugate prior. Bayesian model selection problem is also being considered under the double-exponential priors. By the convergence of ρ-mixing sequence, the consistency and asymptotic normality of the Bayesian estimators of the regression coefficients are proved. Simulation results indicate that our Bayesian estimators are not strongly dependent on the priors, and are robust.     

Analysis of hybrid censored competing risks data

In this paper, we consider the analysis of hybrid censored competing risks data, based on Cox's latent failure time model assumptions. It is assumed that lifetime distributions of latent causes of failure follow Weibull distribution with the same shape parameter, but different scale parameters. Maximum likelihood estimators (MLEs) of the unknown parameters can be obtained by solving a one-dimensional optimization problem, and we propose a fixed-point type algorithm to solve this optimization problem. Approximate MLEs have been proposed based on Taylor series expansion, and they have explicit expressions. Bayesian inference of the unknown parameters are obtained based on the assumption that the shape parameter has a log-concave prior density function, and for the given shape parameter, the scale parameters have Beta–Gamma priors. We propose to use Markov Chain Monte Carlo samples to compute Bayes estimates and also to construct highest posterior density credible intervals. Monte Carlo simulations are performed to investigate the performances of the different estimators, and two data sets have been analysed for illustrative purposes.