Bayesian inference for the variance components in general mixed linear models

The general mixed linear model, containing both the fixed and random effects, is considered. Using gamma priors for the variance components, the conditional posterior distributions of the fixed effects and the variance components, conditional on the random effects, are obtained. Using the normal approximation for the multiple t distribution, approximations are obtained for the posterior distributions of the variance components in infinite series form. The same approximation Is used to obtain closed expressions for the moments of the variance components. An example is considered to illustrate the procedure and a numerical study examines the closeness of the approximations.     

Incorporating Previous Results Into the Analysis of General Balanced Designed Experiments

When previous results are available about quantities of interest in a designed experiment they should be incorporated into the analysis. We suppose that estimates of treatment effects and variance components and their precisions are available from previous data but not necessarily the full data. A prior-posterior method is described which incorporates these previous estimates directly into the analysis of a current set of data. General but concise formula are derived for the class of generally balanced designs. Previous results are represented by a multivariate normal prior for the treatment means and independent inverse chi-squared distributions for the variance components. Joint and marginal posterior modes and a measure of dispersion are proposed as combined or updated estimates. These posterior summary statistics have highly interpretable forms and are readily computed.     

Exact Bayesian modeling for bivariate Poisson data and extensions

Bivariate count data arise in several different disciplines (epidemiology, marketing, sports statistics just to name a few) and the bivariate Poisson distribution being a generalization of the Poisson distribution plays an important role in modelling such data. In the present paper we present a Bayesian estimation approach for the parameters of the bivariate Poisson model and provide the posterior distributions in closed forms. It is shown that the joint posterior distributions are finite mixtures of conditionally independent gamma distributions for which their full form can be easily deduced by a recursively updating scheme. Thus, the need of applying computationally demanding MCMC schemes for Bayesian inference in such models will be removed, since direct sampling from the posterior will become available, even in cases where the posterior distribution of functions of the parameters is not available in closed form. In addition, we define a class of prior distributions that possess an interesting conjugacy property which extends the typical notion of conjugacy, in the sense that both prior and posteriors belong to the same family of finite mixture models but with different number of components. Extension to certain other models including multivariate models or models with other marginal distributions are discussed.     

Estimating the parameters of mixed linear models with modal estimators

This paper presents a procedure to estimate the variance components and fixed effects of mixed linear models. The mode of the joint posterior distribution of all the parameters is obtained by an iterative technique.

The proposed method is illustrated with one-way and two-fold nested random models. Two numerical examples demonstrate the iterative solution.     

Conservative prior distributions for variance parameters in hierarchical models

Bayesian hierarchical models typically involve specifying prior distributions for one or more variance components. This is rather removed from the observed data, so specification based on expert knowledge can be difficult. While there are suggestions for “default” priors in the literature, often a conditionally conjugate inverse‐gamma specification is used, despite documented drawbacks of this choice. The authors suggest “conservative” prior distributions for variance components, which deliberately give more weight to smaller values. These are appropriate for investigators who are skeptical about the presence of variability in the second‐stage parameters (random effects) and want to particularly guard against inferring more structure than is really present. The suggested priors readily adapt to various hierarchical modelling settings, such as fitting smooth curves, modelling spatial variation and combining data from multiple sites.     

The posterior distribution of the fixed and random effects in a mixed-effects linear model

The subject of this paper is Bayesian inference about the fixed and random effects of a mixed-effects linear statistical model with two variance components. It is assumed that a priori the fixed effects have a noninformative distribution and that the reciprocals of the variance components are distributed independently (of each other and of the fixed effects) as gamma random variables. It is shown that techniques similar to those employed in a ridge analysis of a response surface can be used to construct a one-dimensional curve that contains all of the stationary points of the posterior density of the random effects. The “ridge analysis” (of the posterior density) can be useful (from a computational standpoint) in finding the number and the locations of the stationary points and can be very informative about various features of the posterior density. Depending on what is revealed by the ridge analysis, a multivariate normal or multivariate-t distribution that is centered at a posterior mode may provide a satisfactory approximation to the posterior distribution of the random effects (which is of the poly-t form).     

Umvu estimators for the population mean and variance based on random effects models for lognormal data

Cross-classified data are often obtained in controlled experimental situations and in epidemiologic studies. As an example of the latter, occupational health studies sometimes require personal exposure measurements on a random sample of workers from one or more job groups, in one or more plant locations, on several different sampling dates. Because the marginal distributions of exposure data from such studies are generally right-skewed and well-approximated as lognormal, researchers in this area often consider the use of ANOVA models after a logarithmic transformation. While it is then of interest to estimate original-scale population parameters (e.g., the overall mean and variance), standard candidates such as maximum likelihood estimators (MLEs) can be unstable and highly biased. Uniformly minimum variance unbiased (UMVU) cstiniators offer a viable alternative, and are adaptable to sampling schemes that are typiral of experimental or epidemiologic studies. In this paper, we provide UMVU estimators for the mean and variance under two random effects ANOVA models for logtransformed data. We illustrate substantial mean squared error gains relative to the MLE when estimating the mean under a one-way classification. We illustrate that the results can readily be extended to encompass a useful class of purely random effects models, provided that the study data are balanced.     

Relabelling in Bayesian mixture models by pivotal units

Label switching is a well-known and fundamental problem in Bayesian estimation of finite mixture models. It arises when exploring complex posterior distributions by Markov Chain Monte Carlo (MCMC) algorithms, because the likelihood of the model is invariant to the relabelling of mixture components. If the MCMC sampler randomly switches labels, then it is unsuitable for exploring the posterior distributions for component-related parameters. In this paper, a new procedure based on the post-MCMC relabelling of the chains is proposed. The main idea of the method is to perform a clustering technique on the similarity matrix, obtained through the MCMC sample, whose elements are the probabilities that any two units in the observed sample are drawn from the same component. Although it cannot be generalized to any situation, it may be handy in many applications because of its simplicity and very low computational burden.     

Bayesian Conditional Mean Estimation in Log‐Normal Linear Regression Models with Finite Quadratic Expected Loss

Log‐normal linear regression models are popular in many fields of research. Bayesian estimation of the conditional mean of the dependent variable is problematic as many choices of the prior for the variance (on the log‐scale) lead to posterior distributions with no finite moments. We propose a generalized inverse Gaussian prior for this variance and derive the conditions on the prior parameters that yield posterior distributions of the conditional mean of the dependent variable with finite moments up to a pre‐specified order. The conditions depend on one of the three parameters of the suggested prior; the other two have an influence on inferences for small and medium sample sizes. A second goal of this paper is to discuss how to choose these parameters according to different criteria including the optimization of frequentist properties of posterior means.     

Asymptotic posterior distributions for balanced nested multi-way models with a large number of main effect levels

We derive the explicit form for the asymptotic posterior distribution of the balanced nested multi-way variance components model with the assumption that the number of the main factor levels tends to infinity while the number of any specific effect factor levels remains fixed. Under the multi-way model, we also study two different parameterizations, called the standard and the centering, and the relationship between certain quadratic forms of random effects and the variance component parameters. The asymptotic results are illustrated by a three-way model and by a simulation study under a two-way case.     

On the variance estimate of a wilcoxon-mann whitney statistic for group ordered data

Two consistent estimators for the non-null variance of Wil-coxon-Mann-Whitney’s statistic applied to grouped ordered data, are considered. The first is based on U-statistics and the sec-ond is obtained by the Delta method. Some examples are given to demonstrate the extent of error when using a null variance esti-mate for constructing confidence intervals. It appears that the two consistent estimates are very close, but may both be disting-uishably larger or smaller than the null variance estimate.     

Value of the beta prior information

Information gain or loss is studied here by considering changes in the reciprocal of the expected posterior variance. For two beta distributions, the ratio of the expected values of their posterior variances provides a convenient criterion which is consistent with several results of the bayesian set-up and also permits the determination of the least informative prior beta distribution. Relations with results obtained by other authors are also discussed.     

Non-negative estimation of variance components in heteroscedastic one-way random-effects ANOVA models

There is a considerable amount of literature dealing with inference about the parameters in a heteroscedastic one-way random-effects ANOVA model. In this paper, we primarily address the problem of improved quadratic estimation of the random-effect variance component. It turns out that such estimators with a smaller mean squared error compared with some standard unbiased quadratic estimators exist under quite general conditions. Improved estimators of the error variance components are also established.     

Multiparameter estimation of discrete exponential distributions

Let X₁, …, X_p be independent random variables, all having the same distribution up to a possibly varying unspecified parameter, where each of the p distributions belongs to the family of one parameter discrete exponential distributions. The problem is to estimate the unknown parameters simultaneously. Hudson (1978) shows that the minimum variance unbiased estimator (MVUE) of the parameters is inadmissible under squared error loss, and estimators better than the MVUE are proposed. Essentially, these estimators shrink the MVUE towards the origin. In this paper, we indicate that estimators shifting the MVUE towards a point different from the origin or a point determined by the observations can be obtained.     

The mixed model for survey regression estimation

Estimation of the population mean under the regression model with random components is considered. Conditions under which the random components regression estimator is design consistent are given. It is shown that consistency holds when incorrect values are used for the variance components. The regression estimator constructed with model parameters that differ considerably from the true parameters performed well in a Monte Carlo study. Variance estimators for the regression predictor are suggested. A variance estimator appropriate for estimators constructed with a biased estimator for the between-group variance component performed well in the Monte Carlo study.     

Empirical Bayes Confidence Intervals for Means of Natural Exponential Family-Quadratic Variance Function Distributions with Application to Small Area Estimation

Abstract.  The paper develops empirical Bayes (EB) confidence intervals for population means with distributions belonging to the natural exponential family-quadratic variance function (NEF-QVF) family when the sample size for a particular population is moderate or large. The basis for such development is to find an interval centred around the posterior mean which meets the target coverage probability asymptotically, and then show that the difference between the coverage probabilities of the Bayes and EB intervals is negligible up to a certain order. The approach taken is Edgeworth expansion so that the sample sizes from the different populations need not be significantly large. The proposed intervals meet the target coverage probabilities asymptotically, and are easy to construct. We illustrate use of these intervals in the context of small area estimation both through real and simulated data. The proposed intervals are different from the bootstrap intervals. The latter can be applied quite generally, but the order of accuracy of these intervals in meeting the desired coverage probability is unknown.     

An adaptive resampling scheme for cycle estimation

Bayesian dynamic linear models (DLMs) are useful in time series modelling, because of the flexibility that they off er for obtaining a good forecast. They are based on a decomposition of the relevant factors which explain the behaviour of the series through a series of state parameters. Nevertheless, the DLM as developed by West and Harrison depend on additional quantities, such as the variance of the system disturbances, which, in practice, are unknown. These are referred to here as 'hyper-parameters' of the model. In this paper, DLMs with autoregressive components are used to describe time series that show cyclic behaviour. The marginal posterior distribution for state parameters can be obtained by weighting the conditional distribution of state parameters by the marginal distribution of hyper-parameters. In most cases, the joint distribution of the hyperparameters can be obtained analytically but the marginal distributions of the components cannot, so requiring numerical integration. We propose to obtain samples of the hyperparameters by a variant of the sampling importance resampling method. A few applications are shown with simulated and real data sets.     

ESTIMATION, PREDICTION AND INFERENCE FOR THE LASSO RANDOM EFFECTS MODEL

The least absolute shrinkage and selection operator (LASSO) can be formulated as a random effects model with an associated variance parameter that can be estimated with other components of variance. In this paper, estimation of the variance parameters is performed by means of an approximation to the marginal likelihood of the observed outcomes. The approximation is based on an alternative but equivalent formulation of the LASSO random effects model. Predictions can be made using point summaries of the predictive distribution of the random effects given the data with the parameters set to their estimated values. The standard LASSO method uses the mode of this distribution as the predictor. It is not the only choice, and a number of other possibilities are defined and empirically assessed in this article. The predictive mode is competitive with the predictive mean (best predictor), but no single predictor performs best across in all situations. Inference for the LASSO random effects is performed using predictive probability statements, which are more appropriate under the random effects formulation than tests of hypothesis.     

Manipulating and summarizing posterior simulations using random variable objects

Practical Bayesian data analysis involves manipulating and summarizing simulations from the posterior distribution of the unknown parameters. By manipulation we mean computing posterior distributions of functions of the unknowns, and generating posterior predictive distributions. The results need to be summarized both numerically and graphically. We introduce, and implement in R, an object-oriented programming paradigm based on a random variable object type that is implicitly represented by simulations. This makes it possible to define vector and array objects that may contain both random and deterministic quantities, and syntax rules that allow to treat these objects like any numeric vectors or arrays, providing a solution to various problems encountered in Bayesian computing involving posterior simulations. We illustrate the use of this new programming environment with examples of Bayesian computing, demonstrating missing-value imputation, nonlinear summary of regression predictions, and posterior predictive checking.     

Observations on the effect of the prior distribution on the predictive distribution in Bayesian inferences

Typically, in the brief discussion of Bayesian inferential methods presented at the beginning of calculus-based undergraduate or graduate mathematical statistics courses, little attention is paid to the process of choosing the parameter value(s) for the prior distribution. Even less attention is paid to the impact of these choices on the predictive distribution of the data. Reasons for this include that the posterior can be found by ignoring the predictive distribution thereby streamlining the derivation of the posterior and/or that computer software can be used to find the posterior distribution. In this paper, the binomial, negative-binomial and Poisson distributions along with their conjugate beta and gamma priors are utilized to obtain the resulting predictive distributions. It is then demonstrated that specific choices of the parameters of the priors can lead to predictive distributions with properties that might be surprising to a non-expert user of Bayesian methods.