Joint modeling of hierarchically clustered and overdispersed non‐gaussian continuous outcomes for comet assay data

Multivariate longitudinal or clustered data are commonly encountered in clinical trials and toxicological studies. Typically, there is no single standard endpoint to assess the toxicity or efficacy of the compound of interest, but co‐primary endpoints are available to assess the toxic effects or the working of the compound. Modeling the responses jointly is thus appealing to draw overall inferences using all responses and to capture the association among the responses. Non‐Gaussian outcomes are often modeled univariately using exponential family models. To accommodate both the overdispersion and hierarchical structure in the data, Molenberghs et al. A family of generalized linear models for repeated measures with normal and conjugate random effects. Statistical Science 2010; 25:325–347 proposed using two separate sets of random effects. This papers considers a model for multivariate data with hierarchically clustered and overdispersed non‐Gaussian data. Gamma random effect for the over‐dispersion and normal random effects for the clustering in the data are being used. The two outcomes are jointly analyzed by assuming that the normal random effects for both endpoints are correlated. The association structure between the response is analytically derived. The fit of the joint model to data from a so‐called comet assay are compared with the univariate analysis of the two outcomes. Copyright © 2012 John Wiley & Sons, Ltd.