Does the online collection of ego-centered network data reduce data quality? An experimental comparison

We analyze whether differences in kind and quality of ego-centered network data are related to whether the data are collected online or offline. We report the results of two studies. In the first study respondents could choose between filling out ego-centered data through a web questionnaire and being probed about their network in a personalized interview. The second study used a design in which respondents were allocated at random to either online or offline data collection. Our results show that the data quality suffers from the online data collection and the findings indicate that this is the consequence of the respondents answering “mechanically”. We conclude that network researchers should avoid to simply copy traditional network items into a web questionnaire. More research is needed about how new design elements specific for web questionnaires can motivate respondents to fill out network questions properly.     

Measuring ego-centered social networks on the web: Questionnaire design issues

Collecting survey data on ego-centered social networks is a difficult task, owing to the complex questionnaire format. Usually, the interviewer handles the dynamics of the question–answer exchange, motivates the respondent and ensures the proper recording of the data. Self-administered modes of data collection, especially web data collection, are more problematic, as the respondents are left alone with a complex and burdensome questionnaire. Therefore, questionnaire layout is crucial for ensuring cooperation and data quality. In this paper we examined three key components of the corresponding web questionnaire: the number of name boxes using a single name generator, question format for assessing alter characteristics (i.e. alter-wise vs. question-wise) and number of name interpreters (i.e. alter characteristics). The number of name boxes was found to be essential for the reported size of social networks and also for some aspects of data quality. Specific data quality effects were also found with respect to variations in question format, where question-wise format performed better than alter-wise. The number of name interpreters had a relatively minor effect. Suggestions for possible standardization of the web interface layout are also given, so that equivalence with other data collection modes can be established.     

EFFECTS OF QUESTIONNAIRE DESIGN ON THE QUALITY OF SURVEY DATA

The importance of format, graphic layout, and question routinginstructions on the quality of survey data has been acknowledgedfor self-administered survey instruments, but the significanceof these factors for questionnaires administered by interviewershas not been recognized equally. This paper examines two studiesin which interviewers used different questionnaire designs toadminister the same set of survey questions to randomly selectedadult respondents in housing units that made up area probabilitysamples of the Detroit metropolitan area. The paper presentsempirical evidence of specific effects (questions skipped inerror, unprobed answers) directly attributable to the choiceof questionnaire design. The analysis shows that question nairedesign choices can either help or hurt the quality of data collectedby interviewers. Furthermore, the behaviors of experienced andinexperienced interviewers are affected in similar ways. Inother words, interviewing experience does not compensate forformat deficits in the design of survey instruments.     

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Recent experimental research has shown that respondents to forced-choicequestions endorse significantly more options than respondentsto check-all questions. This research has challenged the commonassumption that these two question formats can be used interchangeablybut has been limited to comparisons within a single survey mode.In this paper we use data from a 2004 random sample survey ofuniversity students to compare the forced-choice and check-allquestion formats across web self-administered and telephoneinterviewer-administered surveys as they are commonly used insurvey practice. We find that the within-mode question formateffects revealed by previous research and reaffirmed in thecurrent study appear to persist across modes as well; the telephoneforced-choice format produces higher endorsement than the webcheck-all format. These results provide further support forthe argument that the check-all and forced-choice question formatsdo not produce comparable results and are not interchangeableformats. Additional comparisons show that the forced-choiceformat performs similarly across telephone and web modes.     

Using egocentered networks in survey research. A methodological preview on an application of social network analysis in the area of family research

Can questions about families be adequately answered with household survey data? This question leads us to the problem to defining the family. To help us answer this question we use network analysis as a tool to examine the family without a preconceived definition. To get information about the population of family configurations in Federal Republic of Germany, we choose a survey technique that measured ego-centered networks (10–12 network generators, 4–6 network interpreters). Based on two pretests (n₁ = 98, n₂ = 534) and the main survey (n = 10,000), we discuss some problems encountered during data collection. We also examine the validity and reliability of the data. We used variations of the measurement instrument and asked a group of 99 respondents twice to test reliability. Then we compare our results with census data and compatible US and German studies to test validity. The results show: (1) It is possible to get reliable network data by using survey technique. (2) Size of networks, named persons, named relations are reasonably stable. (3) The instrument to measure networks is constant against minor changes in formulation of questionnaire and against variation of collecting techniques. These findings hold both on the macro level (aggregate data), and also on the micro level (person related data).     

Effects on reliability and validity of egocentered network measurements

This paper examines the reliability and validity of egocentered networks. Reliability and validity are estimated by the multitrait-multimethod (MTMM) approach. A split ballot MTMM design [Saris, W.E., 1999. Forced choice or agree/disagree questions? An evaluation by the split ballot MTMM experiment. In: Proceeding of the Meeting of the IRMCS, pp. 122–146; Kogovšek, T., Ferligoj, A., Coenders, G. Saris, W. E., 2002. Estimating the reliability and validity of personal support measures: full information ml estimation with planned incomplete data. Social Networks 24, 1–20] is used, in which separate groups of respondents received different combinations of two methods. The effect of factors such as the methods used and the personal characteristics of respondents that can affect the quality of data was estimated by a meta analysis.Measurement method, type of question, network size, age, gender, extraversion and emotional stability all had statistically significant effects on the validity of measurement. After the list of alters is obtained with name generators, name interpreter questions can be asked in two ways. One way (“by alters”) is to take each alter individually and to ask all the questions about him/her, going alter by alter until the end of the list of alters. The other way (“by questions”) is to take the question and ask this question for all alters on the list, going question by question until the end of the list of name interpreter questions. Telephone interviewing (both by alters and by questions) gave more valid measurements than face-to-face interviews.Behavioral questions were more valid than questions with emotional content. The characteristics of ties were more validly measured in smaller networks. With reference to respondents’ personal characteristics younger respondents, men, extraverted and emotionally stable respondents all had more valid measurements. Reliability was significantly affected by the measurement method, the type of question and age. The telephone/by alters method was the most reliable measurement method. Behavioral questions were more reliable than questions with emotional content. Measurements among younger respondents were also more reliable.     

GENSI: A new graphical tool to collect ego-centered network data

This study (1) tested the effectiveness of a new survey tool to collect ego-centered network data and (2) assessed the impact of giving people feedback about their network on subsequent responses. The new tool, GENSI (Graphical Ego-centered Network Survey Interface), allows respondents to describe all network contacts at once via a graphical representation of their networks. In an online experiment, 434 American adults were randomly assigned to answer traditional network questions or GENSI and were randomly assigned to receive feedback about their network or not. The traditional questionnaire and GENSI took the same amount of time to complete, and measurements of racial composition of the network showed equivalent convergent validity in both survey tools. However, the new tool appears to solve what past researchers have considered to be a problem with online administration: exaggerated numbers of network connections. Moreover, respondents reported enjoying GENSI more than the traditional tool. Thus, using a graphical interface to collect ego-centered network data seems to be promising. However, telling respondents how their network compared to the average Americans reduced the convergent validity of measures administered after the feedback was provided, suggesting that such feedback should be avoided.     

Duocentered networks

When a pair of individuals is central to a research problem (e.g., husband and wife, PhD student and supervisor) the concept of “duocentered” networks can be defined as a useful extension of egocentered networks. This new structure consists of a pair of central egos and their direct links with alters, instead of just one central ego as in the egocentered networks or multiple egos as in complete networks. The key point in this kind of network is that ties exist between the central pair of egos and between them and all alters, but the ties among alters are not considered. Duocentered networks can also be considered as a compromise between egocentered and complete networks. Complete network measurements are often costly to obtain and tend to contain a large proportion of missing data (especially for peripheral actors). Egocentered network data are less costly but a lot of information is lost with their use when a pair of individuals is the relevant unit of analysis.     

Helping Respondents Get It Right the First Time: The Influence of Words, Symbols, and Graphics in Web Surveys

We utilize and apply visual design theory to experimentallytest ways to improve the likelihood that web respondents reportdate answers in a particular format desired by the researcher,thus reducing possible deleterious effects of error messagesor requests for corrections. These experiments were embeddedin a series of web surveys of random samples of university students.We seek to examine the sequential and cumulative effects ofvisually manipulating the size and proximity of the answer spaces,the use of symbols instead of words, the verbal language ofthe question stem, and the graphical location of the symbolicinstruction. Our results show that the successive series ofvisual language manipulations improve respondents’ useof the desired format (two digits for the month and four digitsfor the year) from 45 percent to 96 percent. These results suggestthat writing effective questions for web surveys may dependas much or more on the presentation of the answer categories/spacesas the question wording itself.     

Analyzing event stream dynamics in two-mode networks: An exploratory analysis of private communication in a question and answer community

Information about social networks can often be collected as event stream data. However, most methods in social network analysis are defined for static network snapshots or for panel data. We propose an actor oriented Markov process framework to analyze the structural dynamics in event streams. Estimated parameters are similar to what is known from exponential random graph models or stochastic actor oriented models as implemented in SIENA. We apply the methodology on a question and answer web community and show how the relevance of different kinds of one- and two-mode network structures can be tested using a new software.     

Propensity to consent to data linkage: experimental evidence on the role of three survey design features in a UK longitudinal panel

When performing data linkage, survey respondents need to provide their informed consent. Since not all respondents agree to this request, the linked data-set will have fewer observations than the survey data-set alone and bias may be introduced. By focusing on the role that survey design features play in gaining respondents’ consent, this paper provides an innovative contribution to the studies in this field. Analysing experimental data collected in a nationally representative household panel survey of the British population, we find that interview features such as question format (dependent/independent questions) and placement of the consent question within the questionnaire have an impact on consent rates.     

The effect of personalization on response rates and data quality in web surveys

Personalizing correspondence has often shown to significantly increase survey response rates in mail surveys. This study experimentally tests whether personalization of email invitations acts correspondingly to web survey response rates. Also, it is investigated whether personalization influences data quality. The results of the study, using a large student sample, show that personalization significantly increases the web survey response rate by 8.6 percentage points. The data quality does not appear to be affected in any major way by personalizing the email invitations. However, the analyses do show that respondents of the personalization condition tend to respond with more social desirability bias to sensitive questions. Therefore, it is concluded that personalization has positive effects on the survey response rate, but one should carefully consider whether or not to personalize when a survey on sensitive topics is conducted.     

Information seeking in secondary schools: A multilevel network approach

In this study, we investigate information seeking interactions in secondary schools from a multilevel network approach. Based on network-related theories, we examine the facilitating role of formal subunits. We apply exponential random graph models for multilevel networks and summarize our findings by using a meta-analysis technique. Our results indicate that formal subunits (e.g. subject departments) can, to some extent, facilitate interactions, in loosely coupled organizations (e.g. secondary schools). Finally, this study shows that a multilevel network approach can provide a more informative representation of information seeking ties in knowledge-intensive organizations.     

Has protest increased since the 1970s? How a survey question can construct a spurious trend

The literature on political participation asserts that protest has increased over the last four decades, all over the world. This trend is derived from surveys asking questions about participation in various forms of protest, including demonstrations, boycotts, and unofficial strikes. The latter question made sense in the context in which it was formulated, Britain in the early 1970s, and with regard to the original methodological aim, measuring ‘protest potential’. The absence of a generic question on strikes, however, distorts our understanding of protest. Two sources of data on Britain in the 1980s and 1990s – a population survey and an event catalogue – comprehensively measure strikes. They show that strikes greatly outnumbered demonstrations and other forms of protest. Another claim in the literature, that protesters are highly educated, no longer holds once strikes are properly counted. Strikes in Britain, as in many countries, have dramatically declined since the 1980s. This decline more than offsets any increase in demonstrations and boycotts, meaning that the total volume of protest has decreased. The episode illustrates how survey questions, when replicated without scrutiny, can misconstrue social trends.     

Advancing Social Capital Measurement: Using General Social Survey Cycles to Develop an Efficient Survey Instrument

The General Social Survey (GSS) has provided significant insight about the state of societies around the world through the International Social Survey Programme (ISSP). Although broadly applicable, in this study the context for developing a new social capital survey instrument covers the Canadian GSS since 1985. Questions in the GSS are often used by researchers who are interested in specific questions or themes for use in some form of re-sampling. Given significant social and relational themes, GSS cycles from 1985 to 2013 were reviewed using a social capital framework to produce a new Social Capital GSS survey instrument. A process of vetting and sorting questions led to development of a 41 question instrument that includes 16 demographic and context variables and 25 social network and trust variables. After testing and refinement, the instrument was used with a randomly sampled group across selected East Hamilton Census Tracts that are differentiated by median income and a standard deviation above and below respectively (n?=?97).     

Neighborhood disadvantage, network social capital, and depressive symptoms

Research on why neighborhood disadvantage matters for health focuses on the capacity of neighborhoods to regulate residents' behavior through informal social control. The authors extend this research by conducting a multilevel analysis of data from a 1995 telephone survey of 497 residents of 32 neighborhoods in a U.S. city. The authors find that network social capital mediates the contextual effect of neighborhood disadvantage on depressive symptoms and that health effects of network social capital persist when perceived neighborhood disorder, a standard indicator of low informal social control, is controlled for. The findings demonstrate the value of a conceptualization and measurement of network social capital that (1) considers ties that transcend neighborhood boundaries, (2) investigates health benefits of network social capital in the forms of closure and embedded support resources and range and embedded instrumental resources, and (3) uses network data on specific network members with strong and weak ties to respondents.     

The influence of interviewer characteristics on support for democracy and political engagement in Sub-Saharan Africa*

This study investigates how an interviewer’s characteristics affect how respondents answer survey questions about democracy and political engagement. I analyze data from the 2008 Afrobarometer surveys, in which 810 interviewers surveyed 27,713 respondents across 20 countries in sub-Saharan Africa. Using these data, I study how interviewer education, age, and gender affect two outcomes: (1) response distributions to attitudinal and behavioral survey questions and (2) the likelihood of respondents saying ‘don’t know’ to a survey question. The analysis also investigates how the respondent’s perception of who sponsored the survey (NGO, private sector, government) affects attitudes. The results show that these interviewer characteristics affect the quality of survey data on political attitudes and behaviors. In the discussion, I consider the implications for research based on public opinion data about democracy and political engagement.     

Presentation matters: how mode effects in item non-response depend on the presentation of response options

Comparability of data across modes is an important issue in survey research. In this paper we discuss item non-response to attitudinal questions in telephone and web surveys. We present results from a survey experiment conducted in Italy and in Spain that compares different presentations of response options in an online setting with a benchmark telephone survey. In line with earlier studies we find that (A), the share of non substantial answers in the online survey depends on how the response option is presented. Comparing different presentations in an online survey to the standard approach of telephone surveys using propensity score matching, we find that (B), the share and pattern of non substantial answers, is most similar across the two modes in the online survey when the it is captured in an instruction on each screen for the online survey. Our findings are of particular relevance for the design of multiple mode or mix-mode surveys of attitudinal questions using online and telephone modes.     

Which Label? An Investigation into the Effects of Terminology on Public Perceptions of and Attitudes towards People with Learning Difficulties

There is much rhetoric around the damaging effect of certain labels applied to people with learning difficulties. This empirical study investigated the effect of three different labels in current usage on public perceptions of the groups so labelled. The labels contrasted were 'mentally subnormal', 'mentally handicapped' and 'people with learning difficulties'. Three independent groups of subjects were presented with an attitude questionnaire. This included a semantic differential scale to ascertain a stereotype of the labelled group, a social distance scale, and a series of questions to tap attitudes regarding people's abilities and rights. Each group was questioned with reference to one of the three labels only. A total of 111 subjects took part. The main findings supported the view that the term 'learning difficulties' is associated with more positive attitudes than either of the other labels which do not significantly differ from each other. However there was also evidence that people labelled as having 'learning difficulties' were thought to be less deserving of special provision. The implications of these findings are discussed in relation to present and future uses of labels.