Handling missing data in numeric analyses

ABSTRACT

Social science datasets usually have missing cases, and missing values. All such missing data has the potential to bias future research findings. However, many research reports ignore the issue of missing data, only consider some aspects of it, or do not report how it is handled. This paper rehearses the damage caused by missing data. The paper then briefly considers eight different approaches to handling missing data so as to minimise that damage, their underlying assumptions and the likely costs and benefits. These approaches include complete case analysis, complete variable analysis, single imputation, multiple imputation, maximum likelihood estimation, default replacement values, weighting, and sensitivity analyses. Using only complete cases should be avoided wherever possible. The paper suggests that the more complex, modelling approaches to replacing missing data are based on questionable methodological and philosophical assumptions. And they may anyway not have clear advantages over simpler approaches like default replacements. It makes sense to report all possible forms of missing data, report everything that is known about the characteristics of cases missing values, conduct simple sensitivity analyses of the potential impact of missing data on the substantive results, and retain the knowledge of missingness when using any form of replacement value.     

Analytic strategies for longitudinal networks with missing data

Missing data are often problematic when analyzing complete longitudinal social network data. We review approaches for accommodating missing data when analyzing longitudinal network data with stochastic actor-based models. One common practice is to restrict analyses to participants observed at most or all time points, to achieve model convergence. We propose and evaluate an alternative, more inclusive approach to sub-setting and analyzing longitudinal network data, using data from a school friendship network observed at four waves (N = 694). Compared to standard practices, our approach retained more information from partially observed participants, generated a more representative analytic sample, and led to less biased model estimates for this case study. The implications and potential applications for longitudinal network analysis are discussed.     

Using modern methods for missing data analysis with the social relations model: A bridge to social network analysis

Social network analysis identifies social ties, and perceptual measures identify peer norms. The social relations model (SRM) can decompose interval-level perceptual measures among all dyads in a network into multiple person- and dyad-level components. This study demonstrates how to accommodate missing round-robin data using Bayesian data augmentation, including how to incorporate partially observed covariates as auxiliary correlates or as substantive predictors. We discuss how data augmentation opens the possibility to fit SRM to network ties (potentially without boundaries) rather than round-robin data. An illustrative application explores the relationship between sorority members’ self-reported body comparisons and perceptions of friends’ body talk.     

Cluster analysis of multiplex networks: Defining composite network measures

Social relations are multiplex by nature: actors in a group are tied together by various types of relationships. To understand and explain group processes it is, therefore, important to study multiple social networks simultaneously in a given group. However, with multiplexity the complexity of data also increases. Although some multivariate network methods (e.g. Exponential Random Graph Models, Stochastic Actor-oriented Models) allow to jointly analyze multiple networks, modeling becomes complicated when it focuses on more than a few (2–4) network dimensions. In such cases, dimension reduction methods are called for to obtain a manageable set of variables. Drawing on existing statistical methods and measures, we propose a procedure to reduce the dimensions of multiplex network data measured in multiple groups. We achieve this by clustering the networks using their pairwise similarities, and constructing composite network measures as combinations of the networks in each resulting cluster. The procedure is demonstrated on a dataset of 21 interpersonal network dimensions in 18 Hungarian high-school classrooms. The results indicate that the network items organize into three well-interpretable clusters: positive, negative, and social role attributions. We show that the composite networks defined on these three relationship groups overlap but do not fully coincide with the network measures most often used in adolescent research, such as friendship and dislike.     

Introduction to stochastic actor-based models for network dynamics

Stochastic actor-based models are models for network dynamics that can represent a wide variety of influences on network change, and allow to estimate parameters expressing such influences, and test corresponding hypotheses. The nodes in the network represent social actors, and the collection of ties represents a social relation. The assumptions posit that the network evolves as a stochastic process ‘driven by the actors’, i.e., the model lends itself especially for representing theories about how actors change their outgoing ties. The probabilities of tie changes are in part endogenously determined, i.e., as a function of the current network structure itself, and in part exogenously, as a function of characteristics of the nodes (‘actor covariates’) and of characteristics of pairs of nodes (‘dyadic covariates’). In an extended form, stochastic actor-based models can be used to analyze longitudinal data on social networks jointly with changing attributes of the actors: dynamics of networks and behavior.     

Small networks and selectivity bias in the analysis of survey network data

Selectivity bias is a danger whenever observations are systematically excluded from a data set on the basis of a dependent variable, whether this exclusion is explicit or implicit. If present, the problem has severe consequences for the validity of statistical estimates of effects. The problem is of importance to the analysis of survey network data, since many network measures (such as density) are available only for persons having networks of size two or larger, while others (such as percent kin) are defined only for those having networks of size one or more. Analysts can adjust for selectivity bias by estimating the risk of exclusion (in this case, of having a network of size 0 or 1), and including the modeled risk as a control in substantive equations. Such estimates are presented for the 1985 General Social Survey network data; in the course of this results of Fischer and Phillips on social isolation are replicated. Other ways of guarding against selection bias are also discussed; at a minimum, network size should be included among the set of regressors in analyses of survey network data, as a methodological control if not as a substantive variable.     

A Liang–Zeger Method for Modeling Dyadic Interdependence in the Analysis of Social Networks

This article introduces a modified Liang–Zeger method for the estimation of the variance–covariance matrix of parameter estimates for models of social network data that include variables to characterize dyadic nonindependence. While the pseudolikelihood method has been used recently to estimate parameters for such models, the issue of estimating their standard errors, or the variance–covariance matrix more generally, has been neglected. This article addresses the issue by proposing a method for such estimation and also presents an illustrative application of the method to empirical social network data.     

Asking about the future: methodological insights from energy biographies

Temporality is fundamental to qualitative longitudinal (QLL) research, inherent in the design of returning to participants over time, often to explore moments of change. Previous research has indicated that talking about the future can be difficult, yet there has been insufficient discussion of methodological developments to address these challenges. This paper presents insights from the Energy Biographies project, which has taken a QLL and multimodal approach to investigating how everyday energy use can be understood in relation to biographical pasts and imagined futures. In particular, we detail innovative techniques developed within the project (e.g. SMS photograph activities) to elicit data on anticipated futures, in ways that engender thinking about participants’ own biographical futures and wider societal changes. We conclude by considering some of the significant benefits and challenges such techniques present. These methodological insights have a wider relevance beyond the substantive topic for those interested in eliciting data about futures in qualitative research.     

Three Likelihood-Based Methods For Mean and Covariance Structure Analysis With Nonnormal Missing Data

Survey and longitudinal studies in the social and behavioral sciences generally contain missing data. Mean and covariance structure models play an important role in analyzing such data. Two promising methods for dealing with missing data are a direct maximum-likelihood and a two-stage approach based on the unstructured mean and covariance estimates obtained by the EM-algorithm. Typical assumptions under these two methods are ignorable nonresponse and normality of data. However, data sets in social and behavioral sciences are seldom normal, and experience with these procedures indicates that normal theory based methods for nonnormal data very often lead to incorrect model evaluations. By dropping the normal distribution assumption, we develop more accurate procedures for model inference. Based on the theory of generalized estimating equations, a way to obtain consistent standard errors of the two-stage estimates is given. The asymptotic efficiencies of different estimators are compared under various assumptions. We also propose a minimum chi-square approach and show that the estimator obtained by this approach is asymptotically at least as efficient as the two likelihood-based estimators for either normal or nonnormal data. The major contribution of this paper is that for each estimator, we give a test statistic whose asymptotic distribution is chi-square as long as the underlying sampling distribution enjoys finite fourth-order moments. We also give a characterization for each of the two likelihood ratio test statistics when the underlying distribution is nonnormal. Modifications to the likelihood ratio statistics are also given. Our working assumption is that the missing data mechanism is missing completely at random. Examples and Monte Carlo studies indicate that, for commonly encountered nonnormal distributions, the procedures developed in this paper are quite reliable even for samples with missing data that are missing at random.     

Missing data in networks: exponential random graph (p1) models for networks with non-respondents

Survey studies of complete social networks often involve non-respondents, whereby certain people within the “boundary” of a network do not complete a sociometric questionnaire—either by their own choice or by the design of the study—yet are still nominated by other respondents as network partners. We develop exponential random graph (p¹) models for network data with non-respondents. We model respondents and non-respondents as two different types of nodes, distinguishing ties between respondents from ties that link respondents to non-respondents. Moreover, if we assume that the non-respondents are missing at random, we invoke homogeneity across certain network configurations to infer effects as applicable to the entire set of network actors. Using an example from a well-known network dataset, we show that treating a sizeable proportion of nodes as non-respondents may still result in estimates, and inferences about structural effects, consistent with those for the entire network.If, on the other hand, the principal research focus is on the respondent-only structure, with non-respondents clearly not missing at random, we incorporate the information about ties to non-respondents as exogenous. We illustrate this model with an example of a network within and between organizational departments. Because in this second class of models the number of non-respondents may be large, values of parameter estimates may not be directly comparable to those for models that exclude non-respondents. In the context of discussing recent technical developments in exponential random graph models, we present a heuristic method based on pseudo-likelihood estimation to infer whether certain structural effects may contribute substantially to the predictive capacity of a model, thereby enabling comparisons of important effects between models with differently sized node sets.     

Statistical estimation of co-offending youth networks

The relevance and potential of network approaches in criminology are well known. Friendship networks and antisocial influences conveyed by them have an impact on the spread of criminal behavior. However, there are relatively few studies reporting on the use of statistical network models in the analysis of delinquency data. The purpose here is to discuss statistical network models that might be appropriate for estimating joint participation in crime and the structure of co-offending youth networks. The discussion is largely based on problems encountered in some recent exploratory studies of juvenile crime in Stockholm with register data from the police on suspected offenders. A bipartite graph model of such data is built from assumptions about crime reporting and offender detection for a specific type of crimes committed in a certain area during a certain time period. It is shown how the model parameters and the numbers of crimes of different sizes, the numbers of offenders of different activities, and the total number of offences can be estimated from data about reported crimes and police identified offenders. The illustrations also show the need for further statistical development.     

Network inference,error, and informant (in)accuracy: a Bayesian approach

Much, if not most, social network data is derived from informant reports; past research, however, has indicated that such reports are in fact highly inaccurate representations of social interaction. In this paper, a family of hierarchical Bayesian models is developed which allows for the simultaneous inference of informant accuracy and social structure in the presence of measurement error and missing data. Posterior simulation for these models using Markov Chain Monte Carlo methods is outlined. Robustness of the models to structurally correlated error rates, implications of the Bayesian modeling framework for improved data collection strategies, and the validity of the criterion graph are also discussed.     

Substantive Meanings of Missing Data in Family Research: Does “Don't Know” Matter?

In this article we analyze “don't know” responses from three sources of longitudinal data: the National Longitudinal Study of Adolescent Health (n = 14,528), the National Survey of Families and Households (n = 5,488), and the National Health Interview Survey Second Longitudinal Study of Aging (n = 1,131). We asked whether these responses are meaningful in family research, and, if so, how evaluating these responses can contribute to the development of theory, the discovery of novel findings, and identification of sensible methods for analyzing these nebulous responses. We found that “don't know” responses to questions about family members predicted less educational attainment, poor marital quality, and earlier mortality. Results suggest that this response category may have substantive meanings rather than indicating neutral responses or being missing data.     

Conditionally Independent Dyads (CID) network models: A latent variable approach to statistical social network analysis

Latent variable network models that accommodate edge correlations implicitly, by assuming an underlying latent factor, are increasing in popularity. Although, these models are examples of what is a growing body of research, much of the research is focused on proposing new models or extending others. There has been very little work on unifying the models in a single framework. In this paper, we present a complete framework that organizes existing latent variable network models within an integrative generalized additive model. Our framework is called Conditionally Independent Dyad (CID) models, and includes existing network models that assume dyad (or edge) independence conditional on latent variables and other components in the model. We further discuss practical aspects of model fitting such as posterior parameter estimation via MCMC, identifiability of parameters, approaches to handle missing data and model selection via cross-validation, for the proposed additive CID models. Finally, by presenting several data examples, we illustrate the utility of the proposed framework and provide advice on selecting components for building new CID models.     

Formal network studies and the anthropological perspective

Anthropological social network studies are primarily of interest for an original formulation of the classic sociological problem of reconciling structural and action aspects of social organization. In general, however, these studies have produced disappointing substantive results owing to serious methodological and theoretical difficulties. Within the anthropological tradition are two types of research, viz., structural kinship studies and cognitive anthropological decision models, which have produced sound substantive results and which, if generalized and properly combined, could provide the methodological and theoretical tools which eluded the network scholars.     

Missing Data in Sociological Research: An Overview of Recent Trends and an Illustration for Controversial Questions, Active Nonrespondents and Targeted Samples

In an age of telemarketers, spam emails, and pop-up advertisements, sociologists are finding it increasingly difficult to achieve high response rates for their surveys. Compounding these issues, the current political and social climate has decreased many survey respondents’ likelihood of responding to controversial questions, which are often at the heart of much research in the discipline. Here we discuss such implications for survey research in sociology using: a content analysis of the prevalence of missing data and survey research methods in the most cited articles in top sociology journals, a case study highlighting the extraction of meaningful information through an example of potential mechanisms driving the non-random missing data patterns in the Religion Among Academic Scientists dataset, and qualitative responses from non-responders in this same case. Implications are likely to increase in importance given the ubiquitous nature of survey research, missing data, and privacy concerns in sociological research.     

Covariance Models For Latent Structure In Longitudinal Data

We present several approaches to modeling latent structure in longitudinal studies when the covariance itself is the primary focus of the analysis. This is a departure from much of the work on longitudinal data analysis, in which attention is focused solely on the cross-sectional mean and the influence of covariates on the mean. Such analyses are particularly important in policy-related studies, in which the heterogeneity of the population is of interest. We describe several traditional approaches to this modeling and introduce a flexible, parsimonious class of covariance models appropriate to such analyses. This class, while rooted in the tradition of mixed effects and random coefficient models, merges several disparate modeling philosophies into what we view as a hybrid approach to longitudinal data modeling. We discuss the implications of this approach and its alternatives especially on model interpretation. We compare several implementations of this class to more commonly employed mixed effects models to describe the strengths and limitations of each. These alternatives are compared in an application to long-term trends in wage inequality for young workers. The findings provide additional guidance for the model formulation process in both statistical and substantive senses.     

Negotiating identities: the lives of Pakistani and Bangladeshi young disabled people

Research has generally amalgamated minority ethnic (all called 'Asian' or 'black') disabled young people's experiences and failed to acknowledge the multiple aspects of Asian and black disabled identities, for example how the combined attributes of race, ethnicity, religion, gender, culture, class and disability shape their perspectives and experiences. In an attempt to address this issue my doctoral research explored the experiences and perspectives of 13 young Pakistani and Bangladeshi disabled people. By drawing on the substantive and theoretical findings which emerged from my analysis in this paper I shall consider how multiple aspects of identity, such as ethnicity, disability and gender, affect this population's identity and self-image and how this makes their experiences different from white disabled young people and other minority groups' experiences.     

The Statistical Evaluation of Social Network Dynamics

A class of statistical models is proposed for longitudinal network data. The dependent variable is the changing (or evolving) relation network, represented by two or more observations of a directed graph with a fixed set of actors. The network evolution is modeled as the consequence of the actors making new choices, or withdrawing existing choices, on the basis of functions, with fixed and random components, that the actors try to maximize. Individual and dyadic exogenous variables can be used as covariates. The change in the network is modeled as the stochastic result of network effects (reciprocity, transitivity, etc.) and these covariates. The existing network structure is a dynamic constraint for the evolution of the structure itself. The models are continuous-time Markov chain models that can be implemented as simulation models. The model parameters are estimated from observed data. For estimating and testing these models, statistical procedures are proposed that are based on the method of moments. The statistical procedures are implemented using a stochastic approximation algorithm based on computer simulations of the network evolution process.     

Actor-based analysis of peer influence in A Stop Smoking In Schools Trial (ASSIST)

As shown by the success of network intervention studies that exploit the occurrence of peer influence in their target group, the reliable assessment of peer influence processes can be important for informing public health policy and practice. A recently developed tool for assessing peer influence in longitudinal social network data is stochastic actor-based modeling. The body of the literature in which this method is applied is growing, but how reliable are the results? In this paper, we identify two shortcomings in this literature: the questionable assumption of temporal homogeneity, and the potential dependence of results on the inclusion of nuisance parameters in the model specification. These issues are resolved by analyzing the data of three schools selected from ASSIST, a large UK-based trial of a school-based smoking prevention intervention. Results show that the co-evolution of friendship and smoking is a time heterogeneous process, and that results are sensitive to specification details. However, the peer influence parameter is not affected by either, but emerges as surprisingly stable over time and robust to model variation. This establishes confidence in the method and encourages detailed future investigations of peer influence in ASSIST.