On the Use of a Log-Rate Model for Survey-Weighted Categorical Data

For the analysis of survey-weighted categorical data, one recommended method of analysis is a log-rate model. For each cell in a contingency table, the survey weights are averaged across subjects and incorporated into an offset for a loglinear model. Supposedly, one can then proceed with the analysis of unweighted observed cell counts. We provide theoretical and simulation-based evidence to show that the log-rate analysis is not an effective statistical analysis method and should not be used in general. The root of the problem is in its failure to properly account for variability in the individual weights within cells of a contingency table. This results in goodness-of-fit tests that have higher-than-nominal error rates and confidence intervals for odds ratios that have lower-than-nominal coverage.     

Analyzing Binomial Data in a Split-Plot Design: Classical Approach or Modern Techniques?

Modeling data that are non-normally distributed with random effects is the major challenge in analyzing binomial data in split-plot designs. Seven methods for analyzing such data using mixed, generalized linear, or generalized linear mixed models are compared for the size and power of the tests. This study shows that analyzing random effects properly is more important than adjusting the analysis for non-normality. Methods based on mixed and generalized linear mixed models hold Type I error rates better than generalized linear models. Mixed model methods tend to have higher power than generalized linear mixed models when the sample size is small.     

Modeling Association Between Two or More Categorical Variables that Allow for Multiple Category Choices

Multiple-response (or pick any/c) categorical variables summarize responses to survey questions that ask “pick any” from a set of item responses. Extensions to loglinear model methodology are proposed to model associations between these variables across all their items simultaneously. Because individual item responses to a multiple-response categorical variable are likely to be correlated, the usual chi-square distributional approximations for model-comparison statistics are not appropriate. Adjusted statistics and a new bootstrap procedure are developed to facilitate distributional approximations. Odds ratio and standardized Pearson residual measures are also developed to estimate specific associations and examine deviations from a specified model.     

Bootstrapping Regression Parameters in Multivariate Survival Analysis

Bootstrap methods are proposed for estimating sampling distributions and associated statistics for regression parameters in multivariate survival data. We use an Independence Working Model (IWM) approach, fitting margins independently, to obtain consistent estimates of the parameters in the marginal models. Resampling procedures, however, are applied to an appropriate joint distribution to estimate covariance matrices, make bias corrections, and construct confidence intervals. The proposed methods allow for fixed or random explanatory variables, the latter case using extensions of existing resampling schemes (Loughin,1995), and they permit the possibility of random censoring. An application is shown for the viral positivity time data previously analyzed by Wei, Lin, and Weissfeld (1989). A simulation study of small-sample properties shows that the proposed bootstrap procedures provide substantial improvements in variance estimation over the robust variance estimator commonly used with the IWM. This revised version was published online in July 2006 with corrections to the Cover Date.     

On the Bootstrap and Monotone Likelihood in the Cox Proportional Hazards Regression Model

Recent literature has provided encouragement for using the bootstrap for inference on regression parameters in the Cox proportional hazards (PH) model. However, generating and performing the necessary partial likelihood computations on multitudinous bootstrap samples greatly increases the chances of incurring problems with monotone likelihood at some point in the analysis. The only symptom of monotone likelihood may be a failure to converge in the numerical maximization procedure, and so the problem might naively be dismissed by deleting the offending data set and replacing it with a new one. This strategy is shown to lead to potentially high selection biases in the subsequent summary statistics. This note discusses the importance of keeping track of these monotone likelihood cases and provides recommendations for their use in interpreting bootstrap findings, and for avoiding unwanted biases that may result from high rates of occurrence. In many cases, high monotone likelihood rates indicate that a more highly-specified model may be preferred. Special consideration is given to the problem of high monotone likelihood incidence in Monte Carlo studies of the bootstrap.     

On the analysis of long-term experiments

Summary.  Long-term experiments are commonly used tools in agronomy, soil science and other disciplines for comparing the effects of different treatment regimes over an extended length of time. Periodic measurements, typically annual, are taken on experimental units and are often analysed by using customary tools and models for repeated measures. These models contain nothing that accounts for the random environmental variations that typically affect all experimental units simultaneously and can alter treatment effects. This added variability can dominate that from all other sources and can adversely influence the results of a statistical analysis and interfere with its interpretation. The effect that this has on the standard repeated measures analysis is quantified by using an alternative model that allows for random variations over time. This model, however, is not useful for analysis because the random effects are confounded with fixed effects that are already in the repeated measures model. Possible solutions are reviewed and recommendations are made for improving statistical analysis and interpretation in the presence of these extra random variations.     

Modeling multiple‐response categorical data from complex surveys

Although “choose all that apply” questions are common in modern surveys, methods for analyzing associations among responses to such questions have only recently been developed. These methods are generally valid only for simple random sampling, but these types of questions often appear in surveys conducted under more complex sampling plans. The purpose of this article is to provide statistical analysis methods that can be applied to “choose all that apply” questions in complex survey sampling situations. Loglinear models are developed to incorporate the multiple responses inherent in these types of questions. Statistics to compare models and to measure association are proposed and their asymptotic distributions are derived. Monte Carlo simulations show that tests based on adjusted Pearson statistics generally hold their correct size when comparing models. These simulations also show that confidence intervals for odds ratios estimated from loglinear models have good coverage properties, while being shorter than those constructed using empirical estimates. Furthermore, the methods are shown to be applicable to more general problems of modeling associations between elements of two or more binary vectors. The proposed analysis methods are applied to data from the National Health and Nutrition Examination Survey. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

Multiple Marginal Independence Testing for Pick Any/C Variables

Many survey questions allow respondents to pick any number out of c possible categorical responses or “items”. These kinds of survey questions often use the terminology “choose all that apply” or “pick any”. Often of interest is determining if the marginal response distributions of each item differ among r different groups of respondents. Agresti and Liu (1998, 1999) call this a test for multiple marginal independence (MMI). If respondents are allowed to pick only 1 out of c responses, the hypothesis test may be performed using the Pearson chi-square test of independence. However, since respondents may pick more or less than 1 response, the test's assumptions that responses are made independently of each other is violated. Recently, a few MMI testing methods have been proposed. Loughin and Scherer (1998) propose using a bootstrap method based on a modified version of the Pearson chi-square test statistic. Agresti and Liu (1998, 1999) propose using marginal logit models, quasisymmetric loglinear models, and a few methods based on Pearson chi-square test statistics. Decady and Thomas (1999) propose using a Rao-Scott adjusted chi-squared test statistic. There has not been a full investigation of these MMI testing methods. The purpose here is to evaluate the proposed methods and propose a few new methods. Recommendations are given to guide the practitioner in choosing which MMI testing methods to use.