Improved estimation procedures for multilevel models with binary response: a case-study

During recent years, analysts have been relying on approximate methods of inference to estimate multilevel models for binary or count data. In an earlier study of random-intercept models for binary outcomes we used simulated data to demonstrate that one such approximation, known as marginal quasi-likelihood, leads to a substantial attenuation bias in the estimates of both fixed and random effects whenever the random effects are non-trivial. In this paper, we fit three-level random-intercept models to actual data for two binary outcomes, to assess whether refined approximation procedures, namely penalized quasi-likelihood and second-order improvements to marginal and penalized quasi-likelihood, also underestimate the underlying parameters. The extent of the bias is assessed by two standards of comparison: exact maximum likelihood estimates, based on a Gauss–Hermite numerical quadrature procedure, and a set of Bayesian estimates, obtained from Gibbs sampling with diffuse priors. We also examine the effectiveness of a parametric bootstrap procedure for reducing the bias. The results indicate that second-order penalized quasi-likelihood estimates provide a considerable improvement over the other approximations, but all the methods of approximate inference result in a substantial underestimation of the fixed and random effects when the random effects are sizable. We also find that the parametric bootstrap method can eliminate the bias but is computationally very intensive.     

On the estimation of normal copula discrete regression models using the continuous extension and simulated likelihood

The continuous extension of a discrete random variable is amongst the computational methods used for estimation of multivariate normal copula-based models with discrete margins. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the asymptotic and small-sample efficiency of two variants of the method for estimating the multivariate normal copula with univariate binary, Poisson, and negative binomial regressions, and show that they lead to biased estimates for the latent correlations, and the univariate marginal parameters that are not regression coefficients. We implement a maximum simulated likelihood method, which is based on evaluating the multidimensional integrals of the likelihood with randomized quasi-Monte Carlo methods. Asymptotic and small-sample efficiency calculations show that our method is nearly as efficient as maximum likelihood for fully specified multivariate normal copula-based models. An illustrative example is given to show the use of our simulated likelihood method.     

Marginalized transition random effect models for multivariate longitudinal binary data

Generalized linear models with random effects and/or serial dependence are commonly used to analyze longitudinal data. However, the computation and interpretation of marginal covariate effects can be difficult. This led Heagerty (1999, 2002) to propose models for longitudinal binary data in which a logistic regression is first used to explain the average marginal response. The model is then completed by introducing a conditional regression that allows for the longitudinal, within‐subject, dependence, either via random effects or regressing on previous responses. In this paper, the authors extend the work of Heagerty to handle multivariate longitudinal binary response data using a triple of regression models that directly model the marginal mean response while taking into account dependence across time and across responses. Markov Chain Monte Carlo methods are used for inference. Data from the Iowa Youth and Families Project are used to illustrate the methods.     

Regression models for analyzing clustered binary and continuous outcomes under an assumption of exchangeability

Scientific experiments commonly result in clustered discrete and continuous data. Existing methods for analyzing such data include the use of quasi-likelihood procedures and generalized estimating equations to estimate marginal mean response parameters. In applications to areas such as developmental toxicity studies, where discrete and continuous measurements are recorded on each fetus, or clinical ophthalmologic trials, where different types of observations are made on each eye, the assumption that data within cluster are exchangeable is often very reasonable. We use this assumption to formulate fully parametric regression models for clusters of bivariate data with binary and continuous components. The regression models proposed have marginal interpretations and reproducible model structures. Tractable expressions for likelihood equations are derived and iterative schemes are given for computing efficient estimates (MLEs) of the marginal mean, correlations, variances and higher moments. We demonstrate the use the ‘exchangeable’ procedure with an application to a developmental toxicity study involving fetal weight and malformation data.     

A comparison of methods for simulating correlated binary variables with specified marginal means and correlations

Simulation studies employed to study properties of estimators for parameters in population-average models for clustered or longitudinal data require suitable algorithms for data generation. Methods for generating correlated binary data that allow general specifications of the marginal mean and correlation structures are particularly useful. We compare an algorithm based on dichotomizing multi-normal variates to one based on a conditional linear family (CLF) of distributions [Qaqish BF. A family of multivariate binary distributions for simulating correlated binary variables with specified marginal means and correlations. Biometrika. 2003;90:455–463] with respect to range restrictions induced on correlations. Examples include generating longitudinal binary data and generating correlated binary data compatible with specified marginal means and covariance structures for bivariate, overdispersed binomial outcomes. Results show the CLF method gives a wider range of correlations for longitudinal data having autocorrelated within-subject associations, while the multivariate probit method gives a wider range of correlations for clustered data having exchangeable-type correlations. In the case of a decaying-product correlation structure, it is shown that the CLF method achieves the nonparametric limits on the range of correlations, which cannot be surpassed by any method.     

Performance of and software for a modified mantel-haenszel statistic for correlated data

This paper presents the results of a small sample simulation study designed to evaluate the performance of a recently proposed test statistic for the analysis of correlated binary data. The new statistic is an adjusted Mantel-Haenszel test, which may be used in testing for association between a binary exposure and a binary outcome of interest across several fourfold tables when the data have been collected under a cluster sampling design. Al- though originally developed for the analysis of periodontal data, the proposed method may be applied to clustered binary data arising in a variety of settings, including longitu- dinal studies, family studies, and school-based research. The features of the simulation are intended to mimic those of a research study of periodontal health, in which a large number of observations is made on each of a relatively small number of patients. The simulation reveals that the adjusted test statistic performs well in finite samples, having empirical type I error rates close to nominal and empirical power similar to that of more complicated marginal regression methods. Software for computing the adjusted statistic is also provided.     

Dirichlet models for square contingency tables

We consider the square contingency tables which arise when the same method of classification is applied twice. The hypothesis of marginal homogeneity is then relevant! and can be tested by various methods Models are discussed which contain marginal homogeneity as a special case. They include a class based on univariate and bivariate Dirichlet distributions. The question of ordered categories is briefly discussed. Applications are made to data on unaided distance vision.     

Concurrent generation of binary and nonnormal continuous data through fifth-order power polynomials

Data collection process in most observational and experimental studies yield different types of variables, leading to the use of joint models that are capable of handling multiple data types. Evaluation of various statistical techniques that have been developed for mixed data in simulated environments requires concurrent generation of multiple variables. In this article, I present an important augmentation to a unified framework proposed in our previously published work for simultaneously generating binary and nonnormal continuous data given the marginal characteristics and correlation structure, via fifth-order power polynomials that are known to extend the area covered in the skewness-elongation plane and to provide a better approximation to the probability density function of the continuous variables. I evaluate how well the improved methodology performs in comparison to the original one, in a simulated setting with illustrations of algorithmic steps. Although the relative gains for the associational quantities are not substantial, the augmented version appears to better capture the marginal quantities that are pertinent to the higher-order moments, as indicated by very close resemblance between the specified and empirically computed quantities on average.     

A unified approach to estimating association measures via a joint generalized linear model for paired binary data

Paired binary data arise frequently in biomedical studies with unique features of their own. For instance, in clinical studies involving pairs such as ears, eyes etc., often both the intrapair association parameter and the event probability are of interest. In addition, we may be interested in the dependence of the association parameter on certain covariates as well. Although various methods have been proposed to model paired binary data, this paper proposes a unified approach for estimating various intrapair measures under a generalized linear model with simultaneous maximum likelihood estimates of the marginal probabilities and the intrapair association. The methods are illustrated with a twin morbidity study.     

An Approximate Bayesian Marginal Likelihood Approach for Estimating Finite Mixtures

Estimation of finite mixture models when the mixing distribution support is unknown is an important problem. This article gives a new approach based on a marginal likelihood for the unknown support. Motivated by a Bayesian Dirichlet prior model, a computationally efficient stochastic approximation version of the marginal likelihood is proposed and large-sample theory is presented. By restricting the support to a finite grid, a simulated annealing method is employed to maximize the marginal likelihood and estimate the support. Real and simulated data examples show that this novel stochastic approximation and simulated annealing procedure compares favorably with existing methods.     

A comparison of analysis procedures for correlated binary data in dedicated multi‐rater imaging trials

In this paper, three analysis procedures for repeated correlated binary data with no a priori ordering of the measurements are described and subsequently investigated. Examples for correlated binary data could be the binary assessments of subjects obtained by several raters in the framework of a clinical trial. This topic is especially of relevance when success criteria have to be defined for dedicated imaging trials involving several raters conducted for regulatory purposes. First, an analytical result on the expectation of the ‘Majority rater’ is presented when only the marginal distributions of the single raters are given. The paper provides a simulation study where all three analysis procedures are compared for a particular setting. It turns out that in many cases, ‘Average rater’ is associated with a gain in power. Settings were identified where ‘Majority significant’ has favorable properties. ‘Majority rater’ is in many cases difficult to interpret. Copyright © 2014 John Wiley & Sons, Ltd.     

Non-parametric estimation of distance between groups

A numerical procedure is outlined for obtaining the distance between samples from two populations. First, the probability densities in the two populations are estimated by kernel methods, and then the distance is derived by numerical integration of a suitable function of these densities. Various such functions have been proposed in the past; they are all implemented and compared with each other and with Mahalanobis D 2 on several real and simulated data sets. The results show the method to be viable, and to perform well against the Mahalanobis D 2 standard.     

A note on generating correlated matched-pair binary data through conditional linear family

Generating correlated binary data with specified marginal probabilities and correlation structure is often needed and useful in simulation studies to investigate the finite sample performance of statistical methods. Conditional linear family provides a powerful and flexible tool to generate correlated matched-pair binary data including the physician–patients and clustered match-pair data. To ensure the validity of the data generation process, constraints for parameters of the conditional linear family are needed. For the correlated matched-pair binary data with an exchangeable-type correlation structure, we derive the explicit expressions to check these constraints and it provides an efficient and convenient computational tool in validating the data generation process. The results are applied to check the constraints for two typical correlated matched-pair binary data.     

Modelling paired categorical outcomes in clinical trials

The goal of this paper is to discuss methods for testing the homogeneity of treatment‐induced changes in trials with paired categorical responses. Widely used marginal homogeneity tests ignore the information contained in concordant pairs of observations and become highly underpowered for configurations of parameters encountered in real trials. This paper considers models for paired binary or ordinal outcomes based on both discordant and concordant pairs that provide a natural extension of marginal models. Likelihood‐ratio tests associated with these models are developed and are demonstrated to be at least as powerful as or more powerful than marginal homogeneity tests. The proposed models are easy to fit using standard statistical software. Copyright © 2003 John Wiley & Sons, Ltd.     

Visual assessment of matrix-variate normality

In recent years, the analysis of three-way data has become ever more prevalent in the literature. It is becoming increasingly common to analyse such data by means of matrix-variate distributions, the most prevalent of which is the matrix-variate normal distribution. Although many methods exist for assessing multivariate normality, there is a relative paucity of approaches for assessing matrix-variate normality. Herein, a new visual method is proposed for assessing matrix-variate normality by means of a distance–distance plot. In addition, a testing procedure is discussed to be used in tandem with the proposed visual method. The proposed approach is illustrated via simulated data as well as an application on analysing handwritten digits.     

A comparison of efficiencies between quasi-likelihood and pseudo-likelihood estimates in non-separable spatial–temporal binary data

The goal of this paper is to compare the performance of two estimation approaches, the quasi-likelihood estimating equation and the pseudo-likelihood equation, against model mis-specification for non-separable binary data. This comparison, to the authors’ knowledge, has not been done yet. In this paper, we first extend the quasi-likelihood work on spatial data to non-separable binary data. Some asymptotic properties of the quasi-likelihood estimate are also briefly discussed. We then use the techniques of a truncated Gaussian random field with a quasi-likelihood type model and a Gibbs sampler with a conditional model in the Markov random field to generate spatial–temporal binary data, respectively. For each simulated data set, both of the estimation methods are used to estimate parameters. Some discussion about the simulation results are also included.     

Principal weighted logistic regression for sufficient dimension reduction in binary classification

Sufficient dimension reduction (SDR) is a popular supervised machine learning technique that reduces the predictor dimension and facilitates subsequent data analysis in practice. In this article, we propose principal weighted logistic regression (PWLR), an efficient SDR method in binary classification where inverse-regression-based SDR methods often suffer. We first develop linear PWLR for linear SDR and study its asymptotic properties. We then extend it to nonlinear SDR and propose the kernel PWLR. Evaluations with both simulated and real data show the promising performance of the PWLR for SDR in binary classification.     

Exact dimensionality selection for Bayesian PCA

We present a Bayesian model selection approach to estimate the intrinsic dimensionality of a high-dimensional dataset. To this end, we introduce a novel formulation of the probabilisitic principal component analysis model based on a normal-gamma prior distribution. In this context, we exhibit a closed-form expression of the marginal likelihood which allows to infer an optimal number of components. We also propose a heuristic based on the expected shape of the marginal likelihood curve in order to choose the hyperparameters. In nonasymptotic frameworks, we show on simulated data that this exact dimensionality selection approach is competitive with both Bayesian and frequentist state-of-the-art methods.     

A new class of INAR(1) model for count time series

The present work proposes a new integer valued autoregressive model with Poisson marginal distribution based on the mixing Pegram and dependent Bernoulli thinning operators. Properties of the model are discussed. We consider several methods for estimating the unknown parameters of the model. Also, the classical and Bayesian approaches are used for forecasting. Simulations are performed for the performance of these estimators and forecasting methods. Finally, the analysis of two real data has been presented for illustrative purposes.     

Orthogonalized Residuals for Estimation of Marginally Specified Association Parameters in Multivariate Binary Data

Abstract. This paper focuses on marginal regression models for correlated binary responses when estimation of the association structure is of primary interest. A new estimating function approach based on orthogonalized residuals is proposed. A special case of the proposed procedure allows a new representation of the alternating logistic regressions method through marginal residuals. The connections between second‐order generalized estimating equations, alternating logistic regressions, pseudo‐likelihood and other methods are explored. Efficiency comparisons are presented, with emphasis on variable cluster size and on the role of higher‐order assumptions. The new method is illustrated with an analysis of data on impaired pulmonary function.