Does the online collection of ego-centered network data reduce data quality? An experimental comparison

We analyze whether differences in kind and quality of ego-centered network data are related to whether the data are collected online or offline. We report the results of two studies. In the first study respondents could choose between filling out ego-centered data through a web questionnaire and being probed about their network in a personalized interview. The second study used a design in which respondents were allocated at random to either online or offline data collection. Our results show that the data quality suffers from the online data collection and the findings indicate that this is the consequence of the respondents answering “mechanically”. We conclude that network researchers should avoid to simply copy traditional network items into a web questionnaire. More research is needed about how new design elements specific for web questionnaires can motivate respondents to fill out network questions properly.     

Reliability and validity of egocentered network data collected via web: A meta-analysis of multilevel multitrait multimethod studies

Our goal in this article is to assess reliability and validity of egocentered network data collected through web surveys using multilevel confirmatory factor analysis under the multitrait multimethod approach.In this study, we analyze a questionnaire of social support of Ph.D. students in three European countries. The traits used are the frequency of social contact questions. The methods used are web survey design variants.We consider egocentered network data as hierarchical; therefore, a multilevel analysis is required. Within and between-ego reliabilities and validities are defined and interpreted.Afterwards, we proceed to a meta-analysis of the results of the three countries where within and between-ego validities and reliabilities are predicted from survey design variables which have to do with question order (by questions or by alters), response category labels (end labels or all labels) and lay-out of the questionnaire (graphical display or plain text). Results show that question order by questions, all-labeled response categories and a graphical display lay-out with images lead to a better data quality. Our basic approach consisting on multilevel and meta-analysis can be applied to evaluate the quality of any type of egocentered network questionnaire, regardless of the data collection mode.     

EFFECTS OF QUESTIONNAIRE DESIGN ON THE QUALITY OF SURVEY DATA

The importance of format, graphic layout, and question routinginstructions on the quality of survey data has been acknowledgedfor self-administered survey instruments, but the significanceof these factors for questionnaires administered by interviewershas not been recognized equally. This paper examines two studiesin which interviewers used different questionnaire designs toadminister the same set of survey questions to randomly selectedadult respondents in housing units that made up area probabilitysamples of the Detroit metropolitan area. The paper presentsempirical evidence of specific effects (questions skipped inerror, unprobed answers) directly attributable to the choiceof questionnaire design. The analysis shows that question nairedesign choices can either help or hurt the quality of data collectedby interviewers. Furthermore, the behaviors of experienced andinexperienced interviewers are affected in similar ways. Inother words, interviewing experience does not compensate forformat deficits in the design of survey instruments.     

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Recent experimental research has shown that respondents to forced-choicequestions endorse significantly more options than respondentsto check-all questions. This research has challenged the commonassumption that these two question formats can be used interchangeablybut has been limited to comparisons within a single survey mode.In this paper we use data from a 2004 random sample survey ofuniversity students to compare the forced-choice and check-allquestion formats across web self-administered and telephoneinterviewer-administered surveys as they are commonly used insurvey practice. We find that the within-mode question formateffects revealed by previous research and reaffirmed in thecurrent study appear to persist across modes as well; the telephoneforced-choice format produces higher endorsement than the webcheck-all format. These results provide further support forthe argument that the check-all and forced-choice question formatsdo not produce comparable results and are not interchangeableformats. Additional comparisons show that the forced-choiceformat performs similarly across telephone and web modes.     

Using egocentered networks in survey research. A methodological preview on an application of social network analysis in the area of family research

Can questions about families be adequately answered with household survey data? This question leads us to the problem to defining the family. To help us answer this question we use network analysis as a tool to examine the family without a preconceived definition. To get information about the population of family configurations in Federal Republic of Germany, we choose a survey technique that measured ego-centered networks (10–12 network generators, 4–6 network interpreters). Based on two pretests (n₁ = 98, n₂ = 534) and the main survey (n = 10,000), we discuss some problems encountered during data collection. We also examine the validity and reliability of the data. We used variations of the measurement instrument and asked a group of 99 respondents twice to test reliability. Then we compare our results with census data and compatible US and German studies to test validity. The results show: (1) It is possible to get reliable network data by using survey technique. (2) Size of networks, named persons, named relations are reasonably stable. (3) The instrument to measure networks is constant against minor changes in formulation of questionnaire and against variation of collecting techniques. These findings hold both on the macro level (aggregate data), and also on the micro level (person related data).     

Effects on reliability and validity of egocentered network measurements

This paper examines the reliability and validity of egocentered networks. Reliability and validity are estimated by the multitrait-multimethod (MTMM) approach. A split ballot MTMM design [Saris, W.E., 1999. Forced choice or agree/disagree questions? An evaluation by the split ballot MTMM experiment. In: Proceeding of the Meeting of the IRMCS, pp. 122–146; Kogovšek, T., Ferligoj, A., Coenders, G. Saris, W. E., 2002. Estimating the reliability and validity of personal support measures: full information ml estimation with planned incomplete data. Social Networks 24, 1–20] is used, in which separate groups of respondents received different combinations of two methods. The effect of factors such as the methods used and the personal characteristics of respondents that can affect the quality of data was estimated by a meta analysis.Measurement method, type of question, network size, age, gender, extraversion and emotional stability all had statistically significant effects on the validity of measurement. After the list of alters is obtained with name generators, name interpreter questions can be asked in two ways. One way (“by alters”) is to take each alter individually and to ask all the questions about him/her, going alter by alter until the end of the list of alters. The other way (“by questions”) is to take the question and ask this question for all alters on the list, going question by question until the end of the list of name interpreter questions. Telephone interviewing (both by alters and by questions) gave more valid measurements than face-to-face interviews.Behavioral questions were more valid than questions with emotional content. The characteristics of ties were more validly measured in smaller networks. With reference to respondents’ personal characteristics younger respondents, men, extraverted and emotionally stable respondents all had more valid measurements. Reliability was significantly affected by the measurement method, the type of question and age. The telephone/by alters method was the most reliable measurement method. Behavioral questions were more reliable than questions with emotional content. Measurements among younger respondents were also more reliable.     

The content of the General Social Survey discussion networks: an exploration of General Social Survey discussion name generator in a Chinese context

This study focuses on the discussion network name generator used in the 1985 General Social Survey (GSS). Based on the data from a Chinese survey, it explores the content of the discussion networks by examining the overlapping of names generated by the GSS discussion name generator and the exchange name generators. It finds that the GSS discussion question has generated a range of social ties, which accounts for an important part of a Chinese personal network. Specifically, the people with whom the Chinese respondents discussed important matters are also likely to spend leisure time with the respondents and to be the confidants for personal matters. Some of them are expected to offer substantial help or to possess important social resources; however, they are least involved in what are considered family affairs.     

At the intersection of criminal justice involvement and sexual orientation: Dynamic networks and health among a population-based sample of young Black men who have sex with men

Mass incarceration of young Black men has a significant impact on their network composition and stability that, in turn, may have major implications for health and well-being. A sub-group of young Black men with criminal justice involvement (CJI) identify as gay, bisexual or are non-identified men who have sex with men (hereafter MSM). This paper focuses on the potential effects of CJI on the composition of Black MSM social and sexual networks, their stability over time, and concomitant health and social outcomes. We use data from the UConnect study, a population-based cohort of young Black MSM 16–29 years of age (n = 618) selected using respondent-driven sampling in Chicago from 2013 to 2016. Both confidant and sexual network name generators and interpreters were administered at 9-month intervals over three waves of data collection. Ego and dyadic-level data were collected on behaviors prevalent among MSM and including factors associated with network CJI, network stability, and health outcomes. Generalized Structural Equation Models (GSEM) were utilized to determine the relationship between CJI network composition, network stability and behaviors prevalent among young Black MSM and their networks. In the UConnect cohort, 46% had at least once been detained, arrested or spent time in jail or prison. In addition, 20% of participants reported incident CJI over the study period. Respondents with a history of CJI were significantly more likely to have CJI homophily in their confidant and sexual networks. Multivariate analyses demonstrate that the association between one’s history of CJI, housing instability and drug use is partially explained by one’s network CJI. In addition, a higher prevalence of network CJI is associated with increased turnover in the confidant network, and this network instability is also related to important health and social outcomes. This analysis describes the networks of criminal justice involved men among a representative sample of young Black MSM and demonstrates the relationship between CJI network homophily, network stability and their impact on several key health and social outcomes relevant to this population.     

Small networks and high isolation? A reexamination of American discussion networks

Recent findings from the General Social Survey suggest that discussion networks have shrunk, and social isolation has increased, since 1985. These data have been threatened by possible artifacts, however, and may be unreliable. This paper reports on a nationally representative experiment that estimates discussion network size and evaluates several proposed explanations for the shrinking networks finding. It finds that modern discussion networks have decreased in size, but that social isolation has not become more prevalent. It also disconfirms several explanations for shrinking networks, and validates a method for increasing the number of names provided in response to name generators.     

To tell the truth: Measuring concordance in multiply reported network data

Social network data must accurately reflect actors’ relationships to properly estimate network features. Here, we examine multiple reports of sexual, drug-sharing and social tie data on high-risk networks in Colorado Springs. By comparing multiple reports on the same ties, we can evaluate the reliability of this study's network data. Our findings suggest that these data have a high level of reporting agreement. From these findings, we discuss implications for analysis of these and similar data and provide suggestions for future social network data collection efforts.     

Analyzing event stream dynamics in two-mode networks: An exploratory analysis of private communication in a question and answer community

Information about social networks can often be collected as event stream data. However, most methods in social network analysis are defined for static network snapshots or for panel data. We propose an actor oriented Markov process framework to analyze the structural dynamics in event streams. Estimated parameters are similar to what is known from exponential random graph models or stochastic actor oriented models as implemented in SIENA. We apply the methodology on a question and answer web community and show how the relevance of different kinds of one- and two-mode network structures can be tested using a new software.     

An exploratory comparison of name generator content: Data from rural India

Since the 1970s sociologists have explored the best means for measuring social networks, although few name generator analyses have used sociocentric data or data from developing countries, partly because sociocentric studies in developing countries have been scant. Here, we analyze 12 different name generators used in a sociocentric network study conducted in 75 villages in rural Karnataka, India. Having unusual sociocentric data from a non-Western context allowed us to extend previous name generator research through the unique analyses of network structural measures, an extensive consideration of homophily, and investigation of status difference between egos and alters. We found that domestic interaction questions generated networks that were highly clustered and highly centralized. Similarity between respondents and their nominated contacts was strongest for gender, caste, and religion. We also found that domestic interaction name generators yielded the most homogeneous ties, while advice questions yielded the most heterogeneous. Participants were generally more likely to nominate those of higher social status, although certain questions, such as who participants talk to uncovered more egalitarian relationships, while other name generators elicited the names of social contacts distinctly higher or lower in status than the respondent. Some questions also seemed to uncover networks that were specific to the cultural context, suggesting that network researchers should balance local relevance with global generalizability when choosing name generators.     

Question-order effects in social network name generators

Social network surveys are an important tool for empirical research in a variety of fields, including the study of social capital and the evaluation of educational and social policy. A growing body of methodological research sheds light on the validity and reliability of social network survey data regarding a single relation, but much less attention has been paid to the measurement of multiplex networks and the validity of comparisons among criterion relations. In this paper, we identify ways that surveys designed to collect multiplex social network data might be vulnerable to question-order effects. We then test several hypotheses using a split-ballot experiment embedded in an online multiple name generator survey of teachers’ advice networks, collected for a study of complete networks. We conclude by discussing implications for the design of multiple name generator social network surveys.     

VERY LOCAL STRUCTURE IN SOCIAL NETWORKS

Triadic configurations are fundamental to many social structural processes and provide the basis for a variety of social network theories and methodologies. This paper addresses the question of how much of the patterning of triads is accounted for by lower-order properties pertaining to nodes and dyads. The empirical base is a collection of 82 social networks representing a number of different species (humans, baboons, macaques, bison, cattle, goats, sparrows, caribou, and more) and an assortment of social relations (friendship, negative sentiments, choice of work partners, advice seeking, reported social interactions, victories in agonistic encounters, dominance, and co-observation). Methodology uses low dimensional representations of triad censuses for these social networks, as compared to censuses expected given four lower-order social network properties. Results show that triadic structure is largely accounted for by properties more local than triads: network density, nodal indegree and outdegree distributions, and the dyad census. These findings reinforce the observation that structural configurations that can be realized in empirical social networks are severely constrained by very local network properties, making some configurations extremely improbable.     

An analysis of cyclically dense data using EBLOC

Everett (1982a) has proposed a graph theoretic technique for analysing social networks. It is not usually possible to apply this method to data which contain a large number of interlocking cycles. We demonstrate a technique which overcomes this problem by referring to the data analysed by MacRae (1960).     

Task-based learning approaches for supporting the development of social science researchers’ critical data skills

Conducting social science research is a complex process, and social science students face a number of learning challenges in order to develop their skills. Moreover, new sources and types of data including so-called big data are providing new opportunities for research, but also pose methodological challenges. In this article, we explore a task-based learning approach for teaching social science research methods. We draw on evidence from two case study learning tasks: (i) the collection and analysis of Twitter data; and (ii) designing and conducting a face-to-face and online survey. The students were guided to co-lead the tasks, apply their knowledge and then to critically reflect. The pedagogical framework of task-based learning provides opportunities to embed learning in new ways by integrating knowledge, practice and critical reflection. Task-based learning can create a dynamic learning environment and can empower students to develop their identities as researchers.     

Name order effects in measuring adolescent social networks using rosters

Recent studies have found order effects in social network data collection, where later names on a roster receive fewer nominations. Some thus argue for randomizing name orders or sampling peer names for survey rosters. We model order effects as biases in nomination choices and demonstrate observational and experimental methods for assessing these biases and illuminating their mechanisms. Employing these lenses, we find little evidence of order effects on eight sociometric questions in four middle school cohorts over six waves. To inform future work, we investigate aspects of the survey situation that may amplify or attenuate order effects. Analyzing these moderating forces offers guidance for detecting, understanding, and mitigating order effects in future research.     

Helping Respondents Get It Right the First Time: The Influence of Words, Symbols, and Graphics in Web Surveys

We utilize and apply visual design theory to experimentallytest ways to improve the likelihood that web respondents reportdate answers in a particular format desired by the researcher,thus reducing possible deleterious effects of error messagesor requests for corrections. These experiments were embeddedin a series of web surveys of random samples of university students.We seek to examine the sequential and cumulative effects ofvisually manipulating the size and proximity of the answer spaces,the use of symbols instead of words, the verbal language ofthe question stem, and the graphical location of the symbolicinstruction. Our results show that the successive series ofvisual language manipulations improve respondents’ useof the desired format (two digits for the month and four digitsfor the year) from 45 percent to 96 percent. These results suggestthat writing effective questions for web surveys may dependas much or more on the presentation of the answer categories/spacesas the question wording itself.     

Network Canvas: Key decisions in the design of an interviewer-assisted network data collection software suite

Self-reported social network analysis studies are often complex and burdensome, both during the interview process itself, and when conducting data management following the interview. Through funding obtained from the National Institute on Drug Abuse (NIDA/NIH), our team developed the Network Canvas suite of software – a set of complementary tools that are designed to simplify the collection and storage of complex social network data, with an emphasis on usability and accessibility across platforms and devices, and guided by the practical needs of researchers. The suite consists of three applications: Architect: an application for researchers to design and export interview protocols; Interviewer: a touch-optimized application for loading and administering interview protocols to study participants; and Server: an application for researchers to manage the interview deployment process and export their data for analysis. Together, they enable researchers with minimal technological expertise to access a complete research workflow, by building their own network interview protocols, deploying these protocols widely within a variety of contexts, and immediately attaining the resulting data from a secure central location. In this paper, we outline the critical decisions taken in developing this suite of tools for the network research community. We also describe the work which guides our decision-making, including prior experiences and key discovery events. We focus on key design choices, taken for theoretical, philosophical, and pragmatic reasons, and outline their strengths and limitations.     

Collecting experimental network data from interventions on critical links in workplace networks

This article describes and discusses challenges associated with interventionist network data gathering in organizational settings, with a special focus on dyadic interventions. While pointing out major risks of these approaches, we argue that collecting data in combination with dyadic network alteration methods can enable social network researchers to explore network mechanisms from a new angle and potentially benefit the members of the targeted social networks and the entire collectives, if certain research design and implementation principles are followed. We introduce a facilitated self-disclosure method for strengthening critical dyads in social networks in combination with longitudinal and retrospective network measurement. We assess the participants’ perceptions of the different stages of this process by qualitative interviews. The study illustrates that experimental network data collection includes some extra challenges in addition to the many challenges of observational network data collection but participants also reported practical benefits that would not be gained through observational network surveys alone. The results highlight the importance of early and continuous communication during the data collection process with all research participants, not just the management, and the benefits of sharing more of the preliminary results. The lessons learnt through this study can inform the design of experimental network data collection to prioritize the preferences of the participants and their benefits.