Simple and flexible Bayesian inferences for standardized regression coefficients

ABSTRACT

In statistical practice, inferences on standardized regression coefficients are often required, but complicated by the fact that they are nonlinear functions of the parameters, and thus standard textbook results are simply wrong. Within the frequentist domain, asymptotic delta methods can be used to construct confidence intervals of the standardized coefficients with proper coverage probabilities. Alternatively, Bayesian methods solve similar and other inferential problems by simulating data from the posterior distribution of the coefficients. In this paper, we present Bayesian procedures that provide comprehensive solutions for inferences on the standardized coefficients. Simple computing algorithms are developed to generate posterior samples with no autocorrelation and based on both noninformative improper and informative proper prior distributions. Simulation studies show that Bayesian credible intervals constructed by our approaches have comparable and even better statistical properties than their frequentist counterparts, particularly in the presence of collinearity. In addition, our approaches solve some meaningful inferential problems that are difficult if not impossible from the frequentist standpoint, including identifying joint rankings of multiple standardized coefficients and making optimal decisions concerning their sizes and comparisons. We illustrate applications of our approaches through examples and make sample R functions available for implementing our proposed methods.     

The present status of confidence interval estimation for one-factor random models

This paper summarizes the present status of confidence interval estimation for one-factor random models. We provide both closed-form intervals and generalized confidence intervals for several parameters. Simple numerical examples are provided to illustrate computations. In order to limit the focus of this review, we consider only one-at-a-time (non-simultaneous) intervals with homoscedastic normal error terms, and do not include any Bayesian or bootstrap methods.     

A nonparametric Bayesian prediction interval for a finite population mean

ABSTRACT

Given a sample from a finite population, we provide a nonparametric Bayesian prediction interval for a finite population mean when a standard normal assumption may be tenuous. We will do so using a Dirichlet process (DP), a nonparametric Bayesian procedure which is currently receiving much attention. An asymptotic Bayesian prediction interval is well known but it does not incorporate all the features of the DP. We show how to compute the exact prediction interval under the full Bayesian DP model. However, under the DP, when the population size is much larger than the sample size, the computational task becomes expensive. Therefore, for simplicity one might still want to consider useful and accurate approximations to the prediction interval. For this purpose, we provide a Bayesian procedure which approximates the distribution using the exchangeability property (correlation) of the DP together with normality. We compare the exact interval and our approximate interval with three standard intervals, namely the design-based interval under simple random sampling, an empirical Bayes interval and a moment-based interval which uses the mean and variance under the DP. However, these latter three intervals do not fully utilize the posterior distribution of the finite population mean under the DP. Using several numerical examples and a simulation study we show that our approximate Bayesian interval is a good competitor to the exact Bayesian interval for different combinations of sample sizes and population sizes.     

Bayesian inference for stable Lévy–driven stochastic differential equations with high‐frequency data

In this paper, we consider parametric Bayesian inference for stochastic differential equations driven by a pure‐jump stable Lévy process, which is observed at high frequency. In most cases of practical interest, the likelihood function is not available; hence, we use a quasi‐likelihood and place an associated prior on the unknown parameters. It is shown under regularity conditions that there is a Bernstein–von Mises theorem associated to the posterior. We then develop a Markov chain Monte Carlo algorithm for Bayesian inference, and assisted with theoretical results, we show how to scale Metropolis–Hastings proposals when the frequency of the data grows, in order to prevent the acceptance ratio from going to zero in the large data limit. Our algorithm is presented on numerical examples that help verify our theoretical findings.     

On construction of asymptotically correct confidence intervals

In this paper we discuss constructing confidence intervals based on asymptotic generalized pivotal quantities (AGPQs). An AGPQ associates a distribution with the corresponding parameter, and then an asymptotically correct confidence interval can be derived directly from this distribution like Bayesian or fiducial interval estimates. We provide two general procedures for constructing AGPQs. We also present several examples to show that AGPQs can yield new confidence intervals with better finite-sample behaviors than traditional methods.     

A Unified Method for Checking Compatibility and Uniqueness for Finite Discrete Conditional Distributions

Checking compatibility for two given conditional distributions and identifying the corresponding unique compatible marginal distributions are important problems in mathematical statistics, especially in Bayesian inferences. In this article, we develop a unified method to check the compatibility and uniqueness for two finite discrete conditional distributions. By formulating the compatibility problem into a system of linear equations subject to constraints, it can be reduced to a quadratic optimization problem with box constraints. We also extend the proposed method from two-dimensional cases to higher-dimensional cases. Finally, we show that our method can be easily applied to checking compatibility and uniqueness for a regression function and a conditional distribution. Several numerical examples are used to illustrate the proposed method. Some comparisons with existing methods are also presented.     

Fiducial and Confidence Distributions for Real Exponential Families

We develop an easy and direct way to define and compute the fiducial distribution of a real parameter for both continuous and discrete exponential families. Furthermore, such a distribution satisfies the requirements to be considered a confidence distribution. Many examples are provided for models, which, although very simple, are widely used in applications. A characterization of the families for which the fiducial distribution coincides with a Bayesian posterior is given, and the strict connection with Jeffreys prior is shown. Asymptotic expansions of fiducial distributions are obtained without any further assumptions, and again, the relationship with the objective Bayesian analysis is pointed out. Finally, using the Edgeworth expansions, we compare the coverage of the fiducial intervals with that of other common intervals, proving the good behaviour of the former.     

A novel Bayesian regression model for counts with an application to health data

Discrete data are collected in many application areas and are often characterised by highly-skewed distributions. An example of this, which is considered in this paper, is the number of visits to a specialist, often taken as a measure of demand in healthcare. A discrete Weibull regression model was recently proposed for regression problems with a discrete response and it was shown to possess desirable properties. In this paper, we propose the first Bayesian implementation of this model. We consider a general parametrization, where both parameters of the discrete Weibull distribution can be conditioned on the predictors, and show theoretically how, under a uniform non-informative prior, the posterior distribution is proper with finite moments. In addition, we consider closely the case of Laplace priors for parameter shrinkage and variable selection. Parameter estimates and their credible intervals can be readily calculated from their full posterior distribution. A simulation study and the analysis of four real datasets of medical records show promises for the wide applicability of this approach to the analysis of count data. The method is implemented in the R package BDWreg.     

Bayesian confidence intervals of proportion with misclassified binary data

A Bayesian approach is considered for the interval estimation of a binomial proportion in doubly sampled data. The coverage probability and the expected width of the Bayesian confidence interval are compared with likelihood-related confidence intervals. It is shown that a hierarchical Bayesian approach provides relatively simple and effective confidence intervals. In addition, it is shown that Agresti–Coull type confidence interval, discussed by  Lee and Choi (2009), can be justified by the Bayesian framework.     

A note on effective sample size for constructing confidence intervals for the difference of two proportions

Proportion differences are often used to estimate and test treatment effects in clinical trials with binary outcomes. In order to adjust for other covariates or intra-subject correlation among repeated measures, logistic regression or longitudinal data analysis models such as generalized estimating equation or generalized linear mixed models may be used for the analyses. However, these analysis models are often based on the logit link which results in parameter estimates and comparisons in the log-odds ratio scale rather than in the proportion difference scale. A two-step method is proposed in the literature to approximate the calculation of confidence intervals for the proportion difference using a concept of effective sample sizes. However, the performance of this two-step method has not been investigated in their paper. On this note, we examine the properties of the two-step method and propose an adjustment to the effective sample size formula based on Bayesian information theory. Simulations are conducted to evaluate the performance and to show that the modified effective sample size improves the coverage property of the confidence intervals.     

Point and Interval Estimation for a Generalized Logistic Distribution Under Progressive Type II Censoring

The likelihood equations based on a progressively Type II censored sample from a Type I generalized logistic distribution do not provide explicit solutions for the location and scale parameters. We present a simple method of deriving explicit estimators by approximating the likelihood equations appropriately. We examine numerically the bias and variance of these estimators and show that these estimators are as efficient as the maximum likelihood estimators (MLEs). The probability coverages of the pivotal quantities (for location and scale parameters) based on asymptotic normality are shown to be unsatisfactory, especially when the effective sample size is small. Therefore we suggest using unconditional simulated percentage points of these pivotal quantities for the construction of confidence intervals. A wide range of sample sizes and progressive censoring schemes have been considered in this study. Finally, we present a numerical example to illustrate the methods of inference developed here.     

Polya后验方法在有限总体抽样估计中的模拟研究

polya后验方法作为一种无信息贝叶斯估计方法,在有限总体抽样中,通过观测的样本,构造一系列的模拟总体,然后进行统计推断。通过统计模拟研究了polya后验方法估计的一些特点,并和Bootstrap方法进行比较。模拟结果显示:polya后验方法能够很好地估计总体的均值,随着样本量的增大,估计值与真值的差距越来越小。采用polya后验方法构造的置信区间区间长度较小,能够很好地覆盖真值。     

Stochastic models for greenhouse gas emission rate estimation from hydroelectric reservoirs: a Bayesian hierarchical approach

Herein, we propose a fully Bayesian approach to the greenhouse gas emission problem. The goal of this work is to estimate the emission rate of polluting gases from the area flooded by hydroelectric reservoirs. We present models for gas concentration evolution in two ways: first, by proposing them from ordinary differential equation solutions and, second, by using stochastic differential equations with a discretization scheme. Finally, we present techniques to estimate the emission rate for the entire reservoir. In order to carry out the inference, we use the Bayesian framework with Monte Carlo via Markov Chain methods. Discretization schemes over continuous differential equations are used when necessary. These models applied to greenhouse gas emission and Bayesian inference for this purpose are completely new in statistical literature, as far as we know, and contribute to estimate the amount of polluting gases released from hydroelectric reservoirs in Brazil. The proposed models are applied in a real data set and results are presented.     

Bayesian Procrustes analysis with applications to hydrology

In this paper, we introduce Procrustes analysis in a Bayesian framework, by treating the classic Procrustes regression equation from a Bayesian perspective, while modeling shapes in two dimensions. The Bayesian approach allows us to compute point estimates and credible sets for the full Procrustes fit parameters. The methods are illustrated through an application to radar data from short-term weather forecasts (nowcasts), a very important problem in hydrology and meteorology.     

Comparison of confidence intervals for the Behrens-Fisher problem

The Behrens–Fisher problem concerns the inferences for the difference between means of two independent normal populations without the assumption of equality of variances. In this article, we compare three approximate confidence intervals and a generalized confidence interval for the Behrens–Fisher problem. We also show how to obtain simultaneous confidence intervals for the three population case (analysis of variance, ANOVA) by the Bonferroni correction factor. We conduct an extensive simulation study to evaluate these methods in respect to their type I error rate, power, expected confidence interval width, and coverage probability. Finally, the considered methods are applied to two real dataset.     

Laplace based approximate posterior inference for differential equation models

Ordinary differential equations are arguably the most popular and useful mathematical tool for describing physical and biological processes in the real world. Often, these physical and biological processes are observed with errors, in which case the most natural way to model such data is via regression where the mean function is defined by an ordinary differential equation believed to provide an understanding of the underlying process. These regression based dynamical models are called differential equation models. Parameter inference from differential equation models poses computational challenges mainly due to the fact that analytic solutions to most differential equations are not available. In this paper, we propose an approximation method for obtaining the posterior distribution of parameters in differential equation models. The approximation is done in two steps. In the first step, the solution of a differential equation is approximated by the general one-step method which is a class of numerical numerical methods for ordinary differential equations including the Euler and the Runge-Kutta procedures; in the second step, nuisance parameters are marginalized using Laplace approximation. The proposed Laplace approximated posterior gives a computationally fast alternative to the full Bayesian computational scheme (such as Makov Chain Monte Carlo) and produces more accurate and stable estimators than the popular smoothing methods (called collocation methods) based on frequentist procedures. For a theoretical support of the proposed method, we prove that the Laplace approximated posterior converges to the actual posterior under certain conditions and analyze the relation between the order of numerical error and its Laplace approximation. The proposed method is tested on simulated data sets and compared with the other existing methods.     

Confidence Intervals for the Weighted Sum of Two Independent Binomial Proportions

Confidence intervals for the difference of two binomial proportions are well known, however, confidence intervals for the weighted sum of two binomial proportions are less studied. We develop and compare seven methods for constructing confidence intervals for the weighted sum of two independent binomial proportions. The interval estimates are constructed by inverting the Wald test, the score test and the Likelihood ratio test. The weights can be negative, so our results generalize those for the difference between two independent proportions. We provide a numerical study that shows that these confidence intervals based on large‐sample approximations perform very well, even when a relatively small amount of data is available. The intervals based on the inversion of the score test showed the best performance. Finally, we show that as for the difference of two binomial proportions, adding four pseudo‐outcomes to the Wald interval for the weighted sum of two binomial proportions improves its coverage significantly, and we provide a justification for this correction.     

Central limit theorems for smoothed extreme value estimates of Poisson point processes boundaries

In this paper, we give sufficient conditions to establish central limit theorems for boundary estimates of Poisson point processes. The considered estimates are obtained by smoothing some bias-corrected extreme values of the point process. We show how the smoothing leads to Gaussian asymptotic distributions and therefore pointwise confidence intervals. Some new unidimensional and multidimensional examples are provided.     

Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study

This paper illustrates how the design and statistical analysis of the primary endpoint of a proof‐of‐concept study can be formulated within a Bayesian framework and is motivated by and illustrated with a Pfizer case study in chronic kidney disease. It is shown how decision criteria for success can be formulated, and how the study design can be assessed in relation to these, both using the traditional approach of probability of success conditional on the true treatment difference and also using Bayesian assurance and pre‐posterior probabilities. The case study illustrates how an informative prior on placebo response can have a dramatic effect in reducing sample size, saving time and resource, and we argue that in some cases, it can be considered unethical not to include relevant literature data in this way. Copyright © 2015 John Wiley & Sons, Ltd.