Analysis of Variance for Bayesian Inference

This paper develops a multiway analysis of variance for non-Gaussian multivariate distributions and provides a practical simulation algorithm to estimate the corresponding components of variance. It specifically addresses variance in Bayesian predictive distributions, showing that it may be decomposed into the sum of extrinsic variance, arising from posterior uncertainty about parameters, and intrinsic variance, which would exist even if parameters were known. Depending on the application at hand, further decomposition of extrinsic or intrinsic variance (or both) may be useful. The paper shows how to produce simulation-consistent estimates of all of these components, and the method demands little additional effort or computing time beyond that already invested in the posterior simulator. It illustrates the methods using a dynamic stochastic general equilibrium model of the US economy, both before and during the global financial crisis.     

Effect of heteroscedasticity between treatment groups on mixed‐effects models for repeated measures

Mixed‐effects models for repeated measures (MMRM) analyses using the Kenward‐Roger method for adjusting standard errors and degrees of freedom in an “unstructured” (UN) covariance structure are increasingly becoming common in primary analyses for group comparisons in longitudinal clinical trials. We evaluate the performance of an MMRM‐UN analysis using the Kenward‐Roger method when the variance of outcome between treatment groups is unequal. In addition, we provide alternative approaches for valid inferences in the MMRM analysis framework. Two simulations are conducted in cases with (1) unequal variance but equal correlation between the treatment groups and (2) unequal variance and unequal correlation between the groups. Our results in the first simulation indicate that MMRM‐UN analysis using the Kenward‐Roger method based on a common covariance matrix for the groups yields notably poor coverage probability (CP) with confidence intervals for the treatment effect when both the variance and the sample size between the groups are disparate. In addition, even when the randomization ratio is 1:1, the CP will fall seriously below the nominal confidence level if a treatment group with a large dropout proportion has a larger variance. Mixed‐effects models for repeated measures analysis with the Mancl and DeRouen covariance estimator shows relatively better performance than the traditional MMRM‐UN analysis method. In the second simulation, the traditional MMRM‐UN analysis leads to bias of the treatment effect and yields notably poor CP. Mixed‐effects models for repeated measures analysis fitting separate UN covariance structures for each group provides an unbiased estimate of the treatment effect and an acceptable CP. We do not recommend MMRM‐UN analysis using the Kenward‐Roger method based on a common covariance matrix for treatment groups, although it is frequently seen in applications, when heteroscedasticity between the groups is apparent in incomplete longitudinal data.     

Computation of the mean value and variance functions in geometric process

Geometric process (GP) is widely used as a non-stationary stochastic model in reliability analysis. In many of applications related with GP its mean value and variance functions are needed. Since there are no analytical forms of these functions in a lot of situations their computations are of importance. In this study, a numerical approximation and Monte Carlo estimation method based on the convolutions of distribution functions have been proposed for both the mean value and variance functions.     

A Weighted Least Squares Approach to Levene's Test of Homogeneity of Variance

In 1960 Levene suggested a potentially robust test of homogeneity of variance based on an ordinary least squares analysis of variance of the absolute values of mean-based residuals. Levene's test has since been shown to have inflated levels of significance when based on the F-distribution, and tests a hypothesis other than homogeneity of variance when treatments are unequally replicated, but the incorrect formulation is now standard output in several statistical packages. This paper develops a weighted least squares analysis of variance of the absolute values of both mean-based and median-based residuals. It shows how to adjust the residuals so that tests using the F -statistic focus on homogeneity of variance for both balanced and unbalanced designs. It shows how to modify the F -statistics currently produced by statistical packages so that the distribution of the resultant test statistic is closer to an F-distribution than is currently the case. The weighted least squares approach also produces component mean squares that are unbiased irrespective of which variable is used in Levene's test. To complete this aspect of the investigation the paper derives exact second-order moments of the component sums of squares used in the calculation of the mean-based test statistic. It shows that, for large samples, both ordinary and weighted least squares test statistics are equivalent; however they are over-dispersed compared to an F variable.     

Exact and approximate distributions for linear contrasts of means and variances

A Bayesian analysis is presented for the K-group Behrens-Fisher problem. Both exact posterior distributions and approximations were developed for both a general linear contrast of the K means and the K variances, given either proper diffuse or informative conjugate priors. The contrast of variances is a unique feature of the heterogeneous variance model that enables investigators to test specific effects of experimental manipulations on variance. Finally, important-differences were observed between the heterogeneous variance model and the homogeneous model.     

Comparison of models for average bioequivalence in replicated crossover designs

Average bioequivalence (ABE) has been the regulatory standard for bioequivalence (BE) since the 1990s. BE studies are commonly two-period crossovers, but may also use replicated designs. The replicated crossover will provide greater power for the ABE assessment. FDA has recommended that ABE analysis of replicated crossovers use a model which includes terms for separate within- and between-subject components for each formulation and which allows for a subject x formulation interaction component. Our simulation study compares the performance of four alternative mixed effects models: the FDA model, a three variance component model proposed by Ekbohm and Melander (EM), a random intercepts and slopes model (RIS) proposed by Patterson and Jones, and a simple model that contains only two variance components. The simple model fails (when not 'true') to provide adequate coverage and it accepts the hypothesis of equivalence too often. FDA and EM models are frequently indistinguishable and often provide the best performance with respect to coverage and probability of concluding BE. The RIS model concludes equivalence too often when both the within- and between-subject variance components differ between formulations. The FDA analysis model is recommended because it provides the most detail regarding components of variability and has a slight advantage over the EM model in confidence interval length.     

A Note on Sufficient Conditions for Valid Unmodified t Testing in Correlation Analysis with Autocorrelated and Heteroscedastic Sample Data

Traditionally, sphericity (i.e., independence and homoscedasticity for raw data) is put forward as the condition to be satisfied by the variance–covariance matrix of at least one of the two observation vectors analyzed for correlation, for the unmodified t test of significance to be valid under the Gaussian and constant population mean assumptions. In this article, the author proves that the sphericity condition is too strong and a weaker (i.e., more general) sufficient condition for valid unmodified t testing in correlation analysis is circularity (i.e., independence and homoscedasticity after linear transformation by orthonormal contrasts), to be satisfied by the variance–covariance matrix of one of the two observation vectors. Two other conditions (i.e., compound symmetry for one of the two observation vectors; absence of correlation between the components of one observation vector, combined with a particular pattern of joint heteroscedasticity in the two observation vectors) are also considered and discussed. When both observation vectors possess the same variance–covariance matrix up to a positive multiplicative constant, the circularity condition is shown to be necessary and sufficient. “Observation vectors” may designate partial realizations of temporal or spatial stochastic processes as well as profile vectors of repeated measures. From the proof, it follows that an effective sample size appropriately defined can measure the discrepancy from the more general sufficient condition for valid unmodified t testing in correlation analysis with autocorrelated and heteroscedastic sample data. The proof is complemented by a simulation study. Finally, the differences between the role of the circularity condition in the correlation analysis and its role in the repeated measures ANOVA (i.e., where it was first introduced) are scrutinized, and the link between the circular variance–covariance structure and the centering of observations with respect to the sample mean is emphasized.     

The Joint Treatment of Exact and Stochastic Restrictions

In regression analysis both exact and stochastic extraneous information may be represented via restrictions on the parameters of a linear model which then may be estimated by applying constrained generalized least squares. It is shown that this estimator can be recast as a computationally simpler estimator that is a combination of the ordinary least squares estimator and the discrepancy between the OLS estimator and both types of restrictions. The variance of the restricted parameters is explicitly shown to depend on the variance of the extraneous information.     

Comparison of ANOVA-F and ANOM tests with regard to type I error rate and test power

A Monte Carlo simulation was conducted to compare the type I error rate and test power of the analysis of means (ANOM) test to the one-way analysis of variance F-test (ANOVA-F). Simulation results showed that as long as the homogeneity of the variance assumption was satisfied, regardless of the shape of the distribution, number of group and the combination of observations, both ANOVA-F and ANOM test have displayed similar type I error rates. However, both tests have been negatively affected from the heterogeneity of the variances. This case became more obvious when the variance ratios increased. The test power values of both tests changed with respect to the effect size (Δ), variance ratio and sample size combinations. As long as the variances are homogeneous, ANOVA-F and ANOM test have similar powers except unbalanced cases. Under unbalanced conditions, the ANOVA-F was observed to be powerful than the ANOM-test. On the other hand, an increase in total number of observations caused the power values of ANOVA-F and ANOM test approach to each other. The relations between effect size (Δ) and the variance ratios affected the test power, especially when the sample sizes are not equal. As ANOVA-F has become to be superior in some of the experimental conditions being considered, ANOM is superior in the others. However, generally, when the populations with large mean have larger variances as well, ANOM test has been seen to be superior. On the other hand, when the populations with large mean have small variances, generally, ANOVA-F has observed to be superior. The situation became clearer when the number of the groups is 4 or 5.     

A clustering approach to interpretable principal components

A new method for constructing interpretable principal components is proposed. The method first clusters the variables, and then interpretable (sparse) components are constructed from the correlation matrices of the clustered variables. For the first step of the method, a new weighted-variances method for clustering variables is proposed. It reflects the nature of the problem that the interpretable components should maximize the explained variance and thus provide sparse dimension reduction. An important feature of the new clustering procedure is that the optimal number of clusters (and components) can be determined in a non-subjective manner. The new method is illustrated using well-known simulated and real data sets. It clearly outperforms many existing methods for sparse principal component analysis in terms of both explained variance and sparseness.     

OPTIMAL DESIGN OF PLANS FOR ACCEPTANCE SAMPLING BY VARIABLES WITH INVERSE GAUSSIAN DISTRIBUTION

ABSTRACT

A vast majority of the literature on the design of sampling plans by variables assumes that the distribution of the quality characteristic variable is normal, and that only its mean varies while its variance is known and remains constant. But, for many processes, the quality variable is nonnormal, and also either one or both of the mean and the variance of the variable can vary randomly. In this paper, an optimal economic approach is developed for design of plans for acceptance sampling by variables having Inverse Gaussian (IG) distributions. The advantage of developing an IG distribution based model is that it can be used for diverse quality variables ranging from highly skewed to almost symmetrical. We assume that the process has two independent assignable causes, one of which shifts the mean of the quality characteristic variable of a product and the other shifts the variance. Since a product quality variable may be affected by any one or both of the assignable causes, three different likely cases of shift (mean shift only, variance shift only, and both mean and variance shift) have been considered in the modeling process. For all of these likely scenarios, mathematical models giving the cost of using a variable acceptance sampling plan are developed. The cost models are optimized in selecting the optimal sampling plan parameters, such as the sample size, and the upper and lower acceptance limits. A large set of numerical example problems is solved for all the cases. Some of these numerical examples are also used in depicting the consequences of: 1) using the assumption that the quality variable is normally distributed when the true distribution is IG, and 2) using sampling plans from the existing standards instead of the optimal plans derived by the methodology developed in this paper. Sensitivities of some of the model input parameters are also studied using the analysis of variance technique. The information obtained on the parameter sensitivities can be used by the model users on prudently allocating resources for estimation of input parameters.     

Estimation and variable selection in nonparametric heteroscedastic regression

The article considers a Gaussian model with the mean and the variance modeled flexibly as functions of the independent variables. The estimation is carried out using a Bayesian approach that allows the identification of significant variables in the variance function, as well as averaging over all possible models in both the mean and the variance functions. The computation is carried out by a simulation method that is carefully constructed to ensure that it converges quickly and produces iterates from the posterior distribution that have low correlation. Real and simulated examples demonstrate that the proposed method works well. The method in this paper is important because (a) it produces more realistic prediction intervals than nonparametric regression estimators that assume a constant variance; (b) variable selection identifies the variables in the variance function that are important; (c) variable selection and model averaging produce more efficient prediction intervals than those obtained by regular nonparametric regression.     

Random rounded integer-valued autoregressive conditional heteroskedastic process

The statistical literature on the analysis of discrete variate time series has concentrated mainly on parametric models, that is the conditional probability mass function is assumed to belong to a parametric family. Generally, these parametric models impose strong assumptions on the relationship between the conditional mean and variance. To generalize these implausible assumptions, this paper instead considers a more realistic semiparametric model, called random rounded integer-valued autoregressive conditional heteroskedastic (RRINARCH) model, where there are essentially no assumptions on the relationship between the conditional mean and variance. The new model has several advantages: (a) it provides a coherent semiparametric framework for discrete variate time series, in which the conditional mean and variance can be modeled separately; (b) it allows negative values both for the series and its autocorrelation function; (c) its autocorrelation structure is the same as that of a standard autoregressive (AR) process; (d) standard software for its estimation is directly applicable. For the new model, conditions for stationarity, ergodicity and the existence of moments are established and the consistency and asymptotic normality of the conditional least squares estimator are proved. Simulation experiments are carried out to assess the performance of the model. The analyses of real data sets illustrate the flexibility and usefulness of the RRINARCH model for obtaining more realistic forecast means and variances.     

Increasing the sample size at interim for a two‐sample experiment without Type I error inflation

For the case of a one‐sample experiment with known variance σ²=1, it has been shown that at interim analysis the sample size (SS) may be increased by any arbitrary amount provided: (1) The conditional power (CP) at interim is ?50% and (2) there can be no decision to decrease the SS (stop the trial early). In this paper we verify this result for the case of a two‐sample experiment with proportional SS in the treatment groups and an arbitrary common variance. Numerous authors have presented the formula for the CP at interim for a two‐sample test with equal SS in the treatment groups and an arbitrary common variance, for both the one‐ and two‐sided hypothesis tests. In this paper we derive the corresponding formula for the case of unequal, but proportional SS in the treatment groups for both one‐sided superiority and two‐sided hypothesis tests. Finally, we present an SAS macro for doing this calculation and provide a worked out hypothetical example. In discussion we note that this type of trial design trades the ability to stop early (for lack of efficacy) for the elimination of the Type I error penalty. The loss of early stopping requires that such a design employs a data monitoring committee, blinding of the sponsor to the interim calculations, and pre‐planning of how much and under what conditions to increase the SS and that this all be formally written into an interim analysis plan before the start of the study. Copyright © 2009 John Wiley & Sons, Ltd.     

Modified Group Runs Control Charts to Detect Increases in Fraction Non Conforming and Shifts in the Process Mean

The nonparametric two-sample bootstrap is employed to compute uncertainties of measures in receiver operating characteristic (ROC) analysis on large datasets in areas such as biometrics, and so on. In this framework, the bootstrap variability was empirically studied without a normality assumption, exhaustively in five scenarios involving both high- and low-accuracy matching algorithms. With a tolerance 0.02 of the coefficient of variation, it was found that 2000 bootstrap replications were appropriate for ROC analysis on large datasets in order to reduce the bootstrap variance and ensure the accuracy of the computation.     

On choosing between fixed and random block effects in some no-interaction models

The purpose of this article is to strengthen the understanding of the relationship between a fixed-blocks and random-blocks analysis in models that do not include interactions between treatments and blocks. Treating the block effects as random has been recommended in the literature for balanced incomplete block designs (BIBD) because it results in smaller variances of treatment contrasts. This reduction in variance is large if the block-to-block variation relative to the total variation is small. However, this analysis is also more complicated because it results in a subjective interpretation of results if the block variance component is non-positive. The probability of a non-positive variance component is large precisely in those situations where a random-blocks analysis is useful – that is, when the block-to-block variation, relative to the total variation, is small. In contrast, the analysis in which the block effects are fixed is computationally simpler and less subjective. The loss in power for some BIBD with a fixed effects analysis is trivial. In such cases, we recommend treating the block effects as fixed. For response surface experiments designed in blocks, however, an opposite recommendation is made. When block effects are fixed, the variance of the estimated response surface is not uniquely estimated, and in practice this variance is obtained by ignoring the block effect. It is argued that a more reasonable approach is to treat the block effects to be random than to ignore it.     

Graphical technique for comparing designs for random models

Methods for comparing designs for a random (or mixed) linear model have focused primarily on criteria based on single-valued functions. In general, these functions are difficult to use, because of their complex forms, in addition to their dependence on the model's unknown variance components. In this paper, a graphical approach is presented for comparing designs for random models. The one-way model is used for illustration. The proposed approach is based on using quantiles of an estimator of a function of the variance components. The dependence of these quantiles on the true values of the variance components is depicted by plotting the so-called quantile dispersion graphs (QDGs), which provide a comprehensive picture of the quality of estimation obtained with a given design. The QDGs can therefore be used to compare several candidate designs. Two methods of estimation of variance components are considered, namely analysis of variance and maximum-likelihood estimation.     

Adjustment of the Profile Likelihood for a Class of Normal Regression Models

We apply the method of McCullagh & Tibshirani (1990) to a generalization of the model for variance components in which the parameter of interest can appear in both the mean and variance. We obtain the exact adjusted profile log-likelihood score function. For the variance components model, we obtain the adjusted profile log-likelihood and show that it equals the restricted log-likelihood of Patterson & Thompson (1971). We discuss an example due to Kempton (1982) of a regression model with autoregressive terms in which the parameter of interest appears in both the mean and variance.     

Modelling interval data with Normal and Skew-Normal distributions

A parametric modelling for interval data is proposed, assuming a multivariate Normal or Skew-Normal distribution for the midpoints and log-ranges of the interval variables. The intrinsic nature of the interval variables leads to special structures of the variance–covariance matrix, which is represented by five different possible configurations. Maximum likelihood estimation for both models under all considered configurations is studied. The proposed modelling is then considered in the context of analysis of variance and multivariate analysis of variance testing. To access the behaviour of the proposed methodology, a simulation study is performed. The results show that, for medium or large sample sizes, tests have good power and their true significance level approaches nominal levels when the constraints assumed for the model are respected; however, for small samples, sizes close to nominal levels cannot be guaranteed. Applications to Chinese meteorological data in three different regions and to credit card usage variables for different card designations, illustrate the proposed methodology.