Kullback-Leibler divergence to evaluate posterior sensitivity to different priors for autoregressive time series models

In this paper we use the Kullback-Leibler divergence to measure the distance between the posteriors of the autoregressive (AR) model coefficients, aiming to evaluate mathematically the sensitivity of the coefficients posterior to different types of priors, i.e. Jeffreys’, g, and natural conjugate priors. In addition, we evaluate the impact of the posteriors distance in Bayesian estimates of mean and variance of the model coefficients by generating a large number of Monte Carlo simulations from the posteriors. Simulation study results show that the coefficients posterior is sensitive to prior distributions, and the posteriors distance has more influence on Bayesian estimates of variance than those of mean of the model coefficients. Same results are obtained from the application to real-world time series datasets.     

Bayesian Estimation of Regression Coefficients Under Extended Balanced Loss Function

Appreciating the desirability of simultaneously using both the criteria of goodness of fitted model and clustering of estimates around true parameter values, an extended version of the balanced loss function is presented and the Bayesian estimation of regression coefficients is discussed. The thus obtained optimal estimator is then compared with the least squares estimator and posterior mean vector with respect to the criteria like posterior expected loss, Bayes risk, bias vector, mean squared error matrix and risk function.     

Exact and approximate distributions for linear contrasts of means and variances

A Bayesian analysis is presented for the K-group Behrens-Fisher problem. Both exact posterior distributions and approximations were developed for both a general linear contrast of the K means and the K variances, given either proper diffuse or informative conjugate priors. The contrast of variances is a unique feature of the heterogeneous variance model that enables investigators to test specific effects of experimental manipulations on variance. Finally, important-differences were observed between the heterogeneous variance model and the homogeneous model.     

Bayesian approach to estimation of intraclass correlation using reference prior

For the balanced variance component model when the intraclass correlation coefficient is of interest, Bayesian analysis is often appropriate. Berger and Bernardo’s (1992a) grouped ordering reference prior approach is used to analyze this model. The reference priors are developed and compared for the posterior inference with real and simulated data. We examine whether the reference priors satisfy the probability-matching criterion. Further, the reference prior is shown to be good in the sense of correct frequentist coverage probability of the posterior quantile.     

A general approach to heteroscedastic linear regression

Our article presents a general treatment of the linear regression model, in which the error distribution is modelled nonparametrically and the error variances may be heteroscedastic, thus eliminating the need to transform the dependent variable in many data sets. The mean and variance components of the model may be either parametric or nonparametric, with parsimony achieved through variable selection and model averaging. A Bayesian approach is used for inference with priors that are data-based so that estimation can be carried out automatically with minimal input by the user. A Dirichlet process mixture prior is used to model the error distribution nonparametrically; when there are no regressors in the model, the method reduces to Bayesian density estimation, and we show that in this case the estimator compares favourably with a well-regarded plug-in density estimator. We also consider a method for checking the fit of the full model. The methodology is applied to a number of simulated and real examples and is shown to work well.     

Consistency of Posterior Distributions for Heteroscedastic Nonparametric Regression Models

In this article, we consider Bayesian inferences for the heteroscedastic nonparametric regression models, when both the mean function and variance function are unknown. We demonstrated consistency of posterior distributions for this model using priors induced by B-splines expansion, treating both random and deterministic covariates in a uniform manner.     

Inference with three-level prior distributions in quantile regression problems

In this paper, we propose a three level hierarchical Bayesian model for variable selection and estimation in quantile regression problems. Specifically, at the first level we consider a zero mean normal priors for the coefficients with unknown variance parameters. At the second level, we specify two different priors for the unknown variance parameters which introduce two different models producing different levels of sparsity. Then, at the third level we suggest joint improper priors for the unknown hyperparameters assuming they are independent. Simulations and Boston Housing data are utilized to compare the performance of our models with six existing models. The results indicate that our models perform good in the simulations and Boston Housing data.     

Bayesian estimation for first-order autoregressive model with explanatory variables

In this article, we develop a Bayesian analysis in autoregressive model with explanatory variables. When σ² is known, we consider a normal prior and give the Bayesian estimator for the regression coefficients of the model. For the case σ² is unknown, another Bayesian estimator is given for all unknown parameters under a conjugate prior. Bayesian model selection problem is also being considered under the double-exponential priors. By the convergence of ρ-mixing sequence, the consistency and asymptotic normality of the Bayesian estimators of the regression coefficients are proved. Simulation results indicate that our Bayesian estimators are not strongly dependent on the priors, and are robust.     

Disaggregation and Forecasting: A Bayesian Analysis

The problem of temporal disaggregation of time series is analyzed by means of Bayesian methods. The disaggregated values are obtained through a posterior distribution derived by using a diffuse prior on the parameters. Further analysis is carried out assuming alternative conjugate priors. The means of the different posterior distributions are shown to be equivalent to some sampling theory results. Bayesian prediction intervals are obtained. Forecasts for future disaggregated values are derived assuming a conjugate prior for the future aggregated value.     

On Bayesian estimators in multistage binomial designs

A new class of Bayesian estimators for a proportion in multistage binomial designs is considered. Priors belong to the beta-J distribution family, which is derived from the Fisher information associated with the design. The transposition of the beta parameters of the Haldane and the uniform priors in fixed binomial experiments into the beta-J distribution yields bias-corrected versions of these priors in multistage designs. We show that the estimator of the posterior mean based on the corrected Haldane prior and the estimator of the posterior mode based on the corrected uniform prior have good frequentist properties. An easy-to-use approximation of the estimator of the posterior mode is provided. The new Bayesian estimators are compared to Whitehead's and the uniformly minimum variance estimators through several multistage designs. Last, the bias of the estimator of the posterior mode is derived for a particular case.     

Bayesian estimation of the efficient frontier

In this paper, we consider the estimation of the three determining parameters of the efficient frontier, the expected return, and the variance of the global minimum variance portfolio and the slope parameter, from a Bayesian perspective. Their posterior distribution is derived by assigning the diffuse and the conjugate priors to the mean vector and the covariance matrix of the asset returns and is presented in terms of a stochastic representation. Furthermore, Bayesian estimates together with the standard uncertainties for all three parameters are provided, and their asymptotic distributions are established. All obtained findings are applied to real data, consisting of the returns on assets included into the S&P 500. The empirical properties of the efficient frontier are then examined in detail.     

Bayesian confidence intervals for means and variances of lognormal and bivariate lognormal distributions

The lognormal distribution is currently used extensively to describe the distribution of positive random variables. This is especially the case with data pertaining to occupational health and other biological data. One particular application of the data is statistical inference with regards to the mean of the data. Other authors, namely Zou et al. (2009), have proposed procedures involving the so-called “method of variance estimates recovery” (MOVER), while an alternative approach based on simulation is the so-called generalized confidence interval, discussed by Krishnamoorthy and Mathew (2003). In this paper we compare the performance of the MOVER-based confidence interval estimates and the generalized confidence interval procedure to coverage of credibility intervals obtained using Bayesian methodology using a variety of different prior distributions to estimate the appropriateness of each. An extensive simulation study is conducted to evaluate the coverage accuracy and interval width of the proposed methods. For the Bayesian approach both the equal-tail and highest posterior density (HPD) credibility intervals are presented. Various prior distributions (Independence Jeffreys' prior, Jeffreys'-Rule prior, namely, the square root of the determinant of the Fisher Information matrix, reference and probability-matching priors) are evaluated and compared to determine which give the best coverage with the most efficient interval width. The simulation studies show that the constructed Bayesian confidence intervals have satisfying coverage probabilities and in some cases outperform the MOVER and generalized confidence interval results. The Bayesian inference procedures (hypothesis tests and confidence intervals) are also extended to the difference between two lognormal means as well as to the case of zero-valued observations and confidence intervals for the lognormal variance. In the last section of this paper the bivariate lognormal distribution is discussed and Bayesian confidence intervals are obtained for the difference between two correlated lognormal means as well as for the ratio of lognormal variances, using nine different priors.     

Bayesian regularisation in structured additive regression: a unifying perspective on shrinkage, smoothing and predictor selection

This paper surveys various shrinkage, smoothing and selection priors from a unifying perspective and shows how to combine them for Bayesian regularisation in the general class of structured additive regression models. As a common feature, all regularisation priors are conditionally Gaussian, given further parameters regularising model complexity. Hyperpriors for these parameters encourage shrinkage, smoothness or selection. It is shown that these regularisation (log-) priors can be interpreted as Bayesian analogues of several well-known frequentist penalty terms. Inference can be carried out with unified and computationally efficient MCMC schemes, estimating regularised regression coefficients and basis function coefficients simultaneously with complexity parameters and measuring uncertainty via corresponding marginal posteriors. For variable and function selection we discuss several variants of spike and slab priors which can also be cast into the framework of conditionally Gaussian priors. The performance of the Bayesian regularisation approaches is demonstrated in a hazard regression model and a high-dimensional geoadditive regression model.     

A bayesian analysis of the mixed linear model

The mixed model is defined. The exact posterior distribution for the fixed effect vector is obtained. The exact posterior distribution for the error variance is obtained. The exact posterior mean and variance of a Bayesian estimator for the variances of random effects is also derived. All computations are non-iterative and avoid numerical integrations.     

Bayesian spectral analysis models for quantile regression with Dirichlet process mixtures

This paper presents a Bayesian analysis of partially linear additive models for quantile regression. We develop a semiparametric Bayesian approach to quantile regression models using a spectral representation of the nonparametric regression functions and the Dirichlet process (DP) mixture for error distribution. We also consider Bayesian variable selection procedures for both parametric and nonparametric components in a partially linear additive model structure based on the Bayesian shrinkage priors via a stochastic search algorithm. Based on the proposed Bayesian semiparametric additive quantile regression model referred to as BSAQ, the Bayesian inference is considered for estimation and model selection. For the posterior computation, we design a simple and efficient Gibbs sampler based on a location-scale mixture of exponential and normal distributions for an asymmetric Laplace distribution, which facilitates the commonly used collapsed Gibbs sampling algorithms for the DP mixture models. Additionally, we discuss the asymptotic property of the sempiparametric quantile regression model in terms of consistency of posterior distribution. Simulation studies and real data application examples illustrate the proposed method and compare it with Bayesian quantile regression methods in the literature.     

Bayesian LASSO-Regularized quantile regression for linear regression models with autoregressive errors

Quantile regression (QR) is a natural alternative for depicting the impact of covariates on the conditional distributions of a outcome variable instead of the mean. In this paper, we investigate Bayesian regularized QR for the linear models with autoregressive errors. LASSO-penalized type priors are forced on regression coefficients and autoregressive parameters of the model. Gibbs sampler algorithm is employed to draw the full posterior distributions of unknown parameters. Finally, the proposed procedures are illustrated by some simulation studies and applied to a real data analysis of the electricity consumption.     

Bayesian partial likelihood approach for tied observations

The Bayesian analysis based on the partial likelihood for Cox's proportional hazards model is frequently used because of its simplicity. The Bayesian partial likelihood approach is often justified by showing that it approximates the full Bayesian posterior of the regression coefficients with a diffuse prior on the baseline hazard function. This, however, may not be appropriate when ties exist among uncensored observations. In that case, the full Bayesian and Bayesian partial likelihood posteriors can be much different. In this paper, we propose a new Bayesian partial likelihood approach for many tied observations and justify its use.     

A new class of multivariate skew distributions with applications to bayesian regression models

Abstract: The authors develop a new class of distributions by introducing skewness in multivariate elliptically symmetric distributions. The class, which is obtained by using transformation and conditioning, contains many standard families including the multivariate skew‐normal and t distributions. The authors obtain analytical forms of the densities and study distributional properties. They give practical applications in Bayesian regression models and results on the existence of the posterior distributions and moments under improper priors for the regression coefficients. They illustrate their methods using practical examples.     

On asymptotic properties of Bayesian partially linear models

In this paper, we present large sample properties of a partially linear model from the Bayesian perspective, in which responses are explained by the semiparametric regression model with the additive form of the linear component and the nonparametric component. For this purpose, we investigate asymptotic behaviors of posterior distributions in terms of consistency. Specifically, we deal with a specific Bayesian partially linear regression model with additive noises in which the nonparametric component is modeled using Gaussian process priors. Under the Bayesian partially linear model using Gaussian process priors, we focus on consistency of posterior distribution and consistency of the Bayes factor, and extend these results to generalized additive regression models and study their asymptotic properties. In addition we illustrate the asymptotic properties based on empirical analysis through simulation studies.     

Bayesian tests for the balanced two-way analysis of variance model

A Bayesian approach for the balanced two-way analysis of variance is derived. Four hypotheses are considered, namely the existence of neither block nor treatment effect, the existence of block but not treatment effect, the existence of treatment but not block effect, and the existence of both block and treatment effects. Lower bounds on the posterior probability are derived over the class of conjugate priors for the normal model. The hypotheses are further combined so that they should resemble the situation in classical testing and the resultant p-values are analysed in conjunction with the lower bounds on the posterior probabilities. The situation of decision making under quadratic loss is also considered.