The effect of interviewer evaluation of respondents’ attitudes and understanding on future survey participation in a panel study

This study explored whether the interviewers’ judgments of respondents’ performances in an initial survey can predict survey participation for a second survey among the same group of respondents in a panel study. Specifically, this study examined two interviewer variables: the interviewers’ judgments of (1) the respondents’ understanding of survey questions and (2) the respondents’ attitudes toward the survey. The analyses revealed that respondents who were rated to have an excellent understanding of the survey questions and friendly attitudes were more likely to respond to the second interview request than those with a poor understanding and unfriendly attitudes, after controlling for the characteristics of the respondents. Of these two, the judgment of the respondents’ attitudes was a stronger predictor. However, interviewer’s judgments are not correlated with the primary survey outcome or the data quality. This suggests limited value of the interviewer observation.     

Effects on reliability and validity of egocentered network measurements

This paper examines the reliability and validity of egocentered networks. Reliability and validity are estimated by the multitrait-multimethod (MTMM) approach. A split ballot MTMM design [Saris, W.E., 1999. Forced choice or agree/disagree questions? An evaluation by the split ballot MTMM experiment. In: Proceeding of the Meeting of the IRMCS, pp. 122–146; Kogovšek, T., Ferligoj, A., Coenders, G. Saris, W. E., 2002. Estimating the reliability and validity of personal support measures: full information ml estimation with planned incomplete data. Social Networks 24, 1–20] is used, in which separate groups of respondents received different combinations of two methods. The effect of factors such as the methods used and the personal characteristics of respondents that can affect the quality of data was estimated by a meta analysis.Measurement method, type of question, network size, age, gender, extraversion and emotional stability all had statistically significant effects on the validity of measurement. After the list of alters is obtained with name generators, name interpreter questions can be asked in two ways. One way (“by alters”) is to take each alter individually and to ask all the questions about him/her, going alter by alter until the end of the list of alters. The other way (“by questions”) is to take the question and ask this question for all alters on the list, going question by question until the end of the list of name interpreter questions. Telephone interviewing (both by alters and by questions) gave more valid measurements than face-to-face interviews.Behavioral questions were more valid than questions with emotional content. The characteristics of ties were more validly measured in smaller networks. With reference to respondents’ personal characteristics younger respondents, men, extraverted and emotionally stable respondents all had more valid measurements. Reliability was significantly affected by the measurement method, the type of question and age. The telephone/by alters method was the most reliable measurement method. Behavioral questions were more reliable than questions with emotional content. Measurements among younger respondents were also more reliable.     

EFFECTS OF INTERVIEWER GENDER, INTERVIEWER CHOICE, AND ITEM WORDING ON RESPONSES TO QUESTIONS CONCERNING SEXUAL BEHAVIOR

We examined factors influencing responses to questions on sexualbehavior among adult respondents 18–49 years old (unweightedN = 2,030) obtained through a random-digit dialing survey. Basedon self-disclosure and perceived control the ory, we hypothesizedthat giving people a choice in selecting the gender of theirinterviewer rather than being assigned an inter viewer, andusing questions that are "supportive" of what may be perceivedof as nonnormative behavior (enhanced items), would increasedata quality relative to, respectively, matched-or opposite-genderinterviewer conditions and standard worded items. The enhanceditems facilitated responding to a number of sensitive topics.However, the effects of item wording on item response are oftenmediated by interviewer conditions. The "choice" results suggestthat giving respondents greater control decreases question threat.However, the overall findings argue for matching respondentsand interviewers on gender over opposite-gender interviewersor allowing respondents to select their interviewer's gender.Wording and interviewer manipulations reduced the discrepanciesbetween men's and women's self-reports of sexual behavior, butthey did not eliminate them, and in some cases they had no effect.The present findings suggest that males tend to be influencedby variations in item wording, interviewer gender, and respondentcontrol across a somewhat wider range of sexual topics. In general,the findings recommend matching respondents and interviewerson gender and the use of more supportive wording in sexual behaviorquestions. However, for assessment of some topics (e.g., sexualviolence) in particular segments of the population (e.g., men),other procedures, such as increasing respondent control, maybe a better choice. Overall, the data support the view thatin terms of preferred procedures, not all sexual topics arecreated equal.     

ISSUE INVOLVEMENT AND RESPONSE EFFECTS IN PUBLIC OPINION SURVEYS

This research tests the widespread assumption that responseeffects due to variations in question form, wording, or contextwill be greatest among respondents who are least involved withan issue. A meta-analysis of results from 15 split-ballot experimentsconducted over a five-year period indicates that the responseeffects of using counterarguments or middle alternatives insurvey questions are significantly larger, as would be expected,among respondents who are less involved with a given issue thanamong those who are highly involved with it. But the effectsof question order and response order appear to be largely unrelatedto how involved a respondent is with a particular issue. Issueinvolvement, then, appears to specify some response effects,but not others.     

Scale direction effects in agree/disagree and item-specific questions: a comparison of question formats

The effect of the response scale direction on response behavior is a well-known phenomenon in survey research. While there are several approaches to explaining how such response order effects occur, the literature reports mixed evidence. Furthermore, different question formats seem to vary in their susceptibility to these effects. We therefore investigate the occurrence of response order effects in Agree/Disagree (A/D) and Item-Specific (IS) questions. We conducted an experiment among n = 930 students in which we varied the scale direction (decremental vs. incremental) within A/D and IS questions and asked respondents to evaluate the questionnaires. The results reveal response order effects within the A/D but not within the IS question format. Furthermore, respondent’s evaluations suggest that completion of the IS questionnaires requires more consideration than the completion of the A/D questionnaires. Altogether, our findings indicate that IS questions are more robust against response order effects than A/D questions.     

Telephone survey respondents' reactions to questions regarding interpersonal violence

Concerns have been raised regarding the appropriateness of asking about violence victimization in telephone interviews and whether asking such questions increases respondents' distress or risk for harm. However, no large-scale studies have evaluated the impact of asking such questions during a telephone interview. This study explored respondents' reactions to questions regarding violence in two large recently completed telephone surveys. After respondents were asked about violence, they were asked if they thought surveys should ask such questions and whether they felt upset or afraid because of the questions. In both surveys, the majority of respondents (regardless of their victimization history) were willing to answer questions about violence and were not upset or afraid because of the questions. More than 92% of respondents thought such questions should be asked. These results challenge commonly held beliefs and assumptions and provide some assurance to those concerned with the ethical collection of data on violent victimization.     

The measurement of personal values in survey research: a test of alternative rating procedures

When survey researchers are interested in measuring the personal values of respondents, they often use a rating rather than a ranking method because it is easier and faster to administer and yields data that are amenable to parametric statistical analyses. However, because personal values are inherently positive constructs, respondents often exhibit little differentiation among the values and end-pile their ratings toward the positive end of the scale. Such lack of differentiation may potentially affect the statistical properties of the values and the ability to detect relationships with other variables. Two experiments were conducted via mail surveys to general population samples to test alternative rating methods designed to increase differentiation and reduce end-piling in the rating of personal values. The results suggest that a procedure in which respondents first pick their most and least important values, then rate them (most-least), provides more differentiation and less end-piling than a simple rating procedure (rate-only). Increased differentiation for the most-least method influenced the fit of latent structure and resulted in more robust relations between the values ratings and other criterion variables. These results generalized across type of values scale, number of values rated, and number of rating points.     

Uncertainty and framing in a valuation task

Many stated preference studies report framing effects in responses to valuation questions. Framing in stated preference studies occurs when respondents use irrelevant information contained in the question to help them value the good. This may occur because respondents are uncertain or do not hold well-formed preferences for the good in question. We investigate if respondent certainty explains framing effects in a contingent valuation study, using data from a double bounded dichotomous elicitation format and a follow-up certainty question. We investigate if respondent certainty influences anchoring and the shift effect. We find evidence that the anchoring effect is stronger for respondents who are less certain about their response to the contingent valuation question compared to respondents who are very certain. However, the shift effect is significant and negative only for respondents who are very certain. Our results indicate that certain respondents are more consistent with the predictions of rational behaviour than uncertain respondents.     

Between Sacral Belief and Moral Community: A Multidimensional Approach to the Relationship Between Religion and Alcohol Among Whites and Blacks

This study explores the conditions under which two dimensions of religion first specified by Durkheim (1995), the normative and the integrative, may be related to a particular aspect of alcohol use among white and black adults. The analysis compares those denominations that oppose the use of alcohol to those that do not, while simultaneously examining the effects of the relative strength of the commitment to the religious community on the risk for dependency. Because a substantially large sample drawn from 41 mid-sized cities nationwide is employed, the study is able to expand the set of denominations considered, and, more importantly, to conduct a more refined analysis of differences between black and white respondents than is typically possible. The findings indicate that although effects due to religious denomination are certainly a factor in the risk for dependency, this is much more so the case among whites than among blacks. By contrast, among blacks, frequency of church attendance, a measure of integration, is a more powerful predictor of risk than it is for whites. These results suggest that to more fully comprehend the manner in which alcohol use and religion are linked, especially in the black community, the organizational nature of the church—and not solely its denomination—needs to be considered in the attempt to assess the risk of alcohol dependency.     

Promoting responsible gambling via prevention messages: insights from the evaluation of actual European messages

Prevention messages are short sentences supposed to broadcast preventive intentions. Three types of messages are noticeable: messages correcting erroneous beliefs, messages informing about the risks and messages promoting responsible gambling. While the results in the literature about messages promoting results are alerting, they are frequently used in European prevention campaigns. The present study relied on language and semantics models to evaluate the communicative and preventive values of 14 messages: 7 were actual European prevention messages and 7 messages were created for the study. Overall, 339 participants answered an online questionnaire in which they evaluated the communicative value of one of the messages and then ranked all the messages according to their preventive level. Results showed that messages informing about the risks and messages correcting erroneous beliefs have a higher level of communicative value than messages promoting responsible gambling. Indeed, the latter are judged as more ambiguous and as less preventive than the two other types of messages. As models of conversational pragmatics suggest that ambiguous messages convey at least two interpretations, the article discusses the possibility that individuals comprehend these messages as incentives to control their impulses (prevention) and incentives to control the game (promotion).     

Spacing, Position, and Order: Interpretive Heuristics for Visual Features of Survey Questions

We present the results of six experiments that demonstrate theimpact of visual features of survey questions on the responsesthey elicit, the response process they initiate, or both. Allsix experiments were embedded in Web surveys. Experiments 1and 2 investigate the effects of the placement of nonsubstantiveresponse options (for example, "No opinion" and "Don’tknow" answer options) in relation to the substantive options.The results suggest that when these options are not differentiatedvisually (by a line or a space) from the substantive options,respondents may be misled about the midpoint of the scale; respondentsseemed to use the visual rather than the conceptual midpointof the scale as a reference point for responding. Experiment3, which varied the spacing of the substantive options, showeda similar result. Responses were pushed in the direction ofthe visual midpoint when it fell to one side of the conceptualmidpoint of the response scale. Experiment 4 examined the effectsof varying whether the response options, which were arrayedvertically, followed a logical progression from top to bottom.Respondents answered more quickly when the options followeda logical order. Experiment 5 examined the effects of the placementof an unfamiliar item among a series of similar items. For example,one set of items asked respondents to say whether several makesand models of cars were expensive or not. The answers for theunfamiliar items depended on the items that were nearby on thelist. Our last experiment varied whether a battery of relateditems was administered on a single screen, across two screens,or with each item on its own screen. The intercorrelations amongthe items were highest when they were all on the same screen.Respondents seem to apply interpretive heuristics in assigningmeaning to visual cues in questionnaires. They see the visualmidpoint of a scale as representing the typical or middle response;they expect options to be arrayed in a progression beginningwith the leftmost or topmost item; and they expect items thatare physically close to be related to each other conceptually.     

Can Demographic Variables Predict Lottery and Pari-mutuel Losses? An Empirical Investigation

Using data from the 2004 and 2005 Consumer Expenditure Surveys (CES) comprising of 15,000 respondents, this study examines two research questions. The first of these considers the demographic differences between households whose members lose money playing the lottery and/or engaging in pari-mutuel betting and those whose members do not lose money participating in such activities. The second assesses demographic differences among households whose members lose money playing the lottery and/or engaging in pari-mutuel betting. It was found that respondents living in money-losing households are slightly older, better off financially, more likely to be married or divorced, more likely to live in a state with at least one legal casino and more likely to live in the Northeast than respondents living in non-money-losing households. Among those living in money-losing households, the least wealthy and African American respondents are more likely to lose a higher proportion of their respective incomes purchasing lottery tickets and engaging in pari-mutuel betting than wealthier respondents and whites.     

Homeless families in St. Louis: A report from the field

In May, June, and September of 1998, 169 residents of the largest homeless shelter in Missouri were surveyed in response to the Institute for Children and Poverty's February 1998 request to continue research on homeless families in other areas of the country. Our findings corroborate the New York City data of the Institute for Children and Poverty (ICP), with these notable differences: ” For the women in our study, the experience of childhood violence and abuse was significantly higher than that reported in the ICP study. ” The number of children in homeless single-parent families in which the parent is beyond childbearing years was higher than the ICP-reported average (2.7 children). ” Older respondents were more likely to have graduated from high school than younger respondents. ” Compared to the ICP´ is respondents, fewer of our survey's respondents had previously lived on their own and more had stayed with family and friends. ” Significantly more residents in our study had no medical insurance. In addition, ” We gathered information on how many non-high school graduates had earned their GED and how many had attended or graduated from college. ” Our respondents were asked in what career fields they wanted to work. ” We gathered data on nicotine, alcohol, and drug abuse among our residents. ” We asked respondents if their children had the same father. Based on this survey, we concluded that programs for homeless children aimed at keeping them in school, improving their education, and helping them to form and realize career goals—as well as services such as counseling and mentoring—are necessary to keep homeless children from becoming homeless adults.     

Eye-Tracking Data: New Insights on Response Order Effects and Other Cognitive Shortcuts in Survey Responding

Survey researchers since Cannell have worried that respondents may take various shortcuts to reduce the effort needed to complete a survey. The evidence for such shortcuts is often indirect. For instance, preferences for earlier versus later response options have been interpreted as evidence that respondents do not read beyond the first few options. This is really only a hypothesis, however, that is not supported by direct evidence regarding the allocation of respondent attention. In the current study, we used a new method to more directly observe what respondents do and do not look at by recording their eye movements while they answered questions in a Web survey. The eye-tracking data indicate that respondents do in fact spend more time looking at the first few options in a list of response options than those at the end of the list; this helps explain their tendency to select the options presented first regardless of their content. In addition, the eye-tracking data reveal that respondents are reluctant to invest effort in reading definitions of survey concepts that are only a mouse click away or paying attention to initially hidden response options. It is clear from the eye-tracking data that some respondents are more prone to these and other cognitive shortcuts than others, providing relatively direct evidence for what had been suspected based on more conventional measures.     

The Effect of Question Framing and Response Options on the Relationship between Racial Attitudes and Beliefs about Genes as Causes of Behavior

Prior research suggests that the attribution of individual and group differences to genetic causes is correlated with prejudiced attitudes toward minority groups. Our study suggests that these findings may be due to the wording of the questions and to the choice of response options. Using a series of vignettes in an online survey, we find a relationship between racial attitudes and genetic attributions when respondents are asked to make causal attributions of differences between racial groups. However, when they are asked to make causal attributions for characteristics shown by individuals, no such relationship is found. The response scale used appears to make less, if any, difference in the results. These findings indicate that the way questions about genetic causation of behavior are framed makes a significant contribution to the answers obtained because it significantly changes the meaning of the questions. We argue that such framing needs to be carefully attended to, not only in posing research questions but also in discourse about genetics more generally.     

Former welfare clients in prison: education and self-reported learning problems

The aim of this study is to investigate educational background, educational wishes, actual participation in prison education, and self-reported learning problems among former child welfare clients in prison. A significant concern is that prisoners' educational level is far lower than for the general population. The same concern has also been expressed regarding former child welfare clients. The question is whether prisoners with a background from the child welfare system are especially vulnerable. The survey reported in this paper addressed the entire Norwegian prison population. We distributed a questionnaire containing questions about whether the prisoner or their family had been in contact with the welfare system, educational issues such as educational background, whether they wanted to participate in prison education and actually participated, and to what extent the participants had self-reported learning problems. We also investigated whether such learning problems affected participation in prison education. The results showed that around one-third of the 2065 respondents were former child welfare clients. The mean age of the respondents in the child welfare group was 31 years and 7.9% were women. The results further revealed that those who had been child welfare clients had a lower educational level and more self-reported learning problems than the general prison population, but most of them still wished to start an education while incarcerated.     

The Impact of Gender and Marital Status on Attitudes and Responses to Unemployment

The study examined differences in job search intensity, attitudes toward unemployment, and related responses among a sample of Israelis, based on gender and marital status (single vs. married). The findings indicate that gender-based differences were greater than those based on marital status. Regarding the impact of gender, women were more likely than men to reject jobs because of conflict with family responsibilities or unsuitable working conditions. Both men and women mentioned gender atypical characteristics of occupations as a reason for rejecting employement, although this tendency was especially prevalent among married women. Women also reported more stress reactions and decline in health as an outcome of unemployment. Regarding the impact of marital status, married men tended to seek jobs more intensively than respondents in the other research groups. Single respondents of both sexes were more likely than their married counterparts to view unemployment as a personal advantage because it gives them more time to themselves. Moreover, married respondents of both sexes were more likely to reject job offers because of conflict with family responsibilities or unsuitable working conditions. In light of the findings, practical recommendations for counselors are provided.     

Predicting ecstasy use among young people at risk: a prospective study of initially ecstasy-naive subjects

Our aim is to identify predictors of first-time ecstasy use in a prospective study among young people at risk. As part of the multidisciplinary Netherlands XTC Toxicity Study (NeXT), we monitored 188 subjects aged > or = 18 who were ecstasy-naive at baseline but seemed likely to start taking ecstasy in the near future. After an 11- to 26-month follow-up period, 160 respondents remained (85.1%; mean age 21.0 years, 58.1% females): 65 who took ecstasy at least once (ecstasy users) and 95 non-users. At baseline and four times during follow-up, respondents completed self-report questionnaires. Cox regression analysis was used to examine the effects of baseline respondent characteristics on incident ecstasy use. Development of peer group ecstasy use was analyzed by logistic regression. Intention to use ecstasy, low education, and current weekly cannabis use independently increased the hazard rate for first ecstasy use. Although ecstasy use among peers at baseline was not a predictor, the proportion of ecstasy users with ecstasy-using peers increased markedly during the study. Our results suggest that targeted prevention activities should focus in particular on young people who have strong intentions to take ecstasy, especially if they are also regular smokers of cannabis.     

INTERVIEW MODE EFFECTS IN SURVEYS OF DRUG AND ALCOHOL USE: A FIELD EXPERIMENT

This research explores the impact of interview mode on respondents'willingness to reveal illicit or undesirable behavior and mechanismsby which interview mode influences response tendencies. A fieldexperiment was designed to control mode effects due to samplingand screening so that the impact of response anonymity (throughuse of self-administered questionnaires ([SAQs]) and socialdistance in the interviewer-respondent relationship (telephonevs. personal communication) could be tested. Respondents aged18–45 were randomly as-signed to interview mode: telephone,face-to-face, or self-administered. Admission of illicit druguse and alcohol use was most likely in the personal mode withSAQs, slightly less likely in personal mode without SAQs, andleast likely in the telephone mode. The magnitude of the modedifferentials was larger for blacks than for whites, and largeramong respondents who are more mistrustful of others. Resultssupport the notion that response effects due to mode of interviewderive, at least in part, from interview mode differences inability to assuage respondents' confidentiality concerns. Greatersocial distance between interviewer and respondent in the telephoneinterview, compared with face-to-face communication, makes itmore difficult for the researcher to make convincing confidentialityguarantees. The response anonymity provided by SAQs also appearsto increase respondents' willingness to reveal sensitive behavior,especially among racial/ethnic minorities.     

The deteriorating work-related well-being among statutory social workers in a rigorous economic context

This article examines the connection between economic pressures and impaired well-being at work experienced by Finnish statutory social workers. The survey data were collected during the years 2011–2012. The respondents (N = 817) were social workers in statutory social services. The overall response rate was 46.5 and nearly 60% of all Finnish municipalities participated in the study. This study affirms the worrying state of the well-being at work of statutory social workers. Up to 43% of the respondents reported worsening of their work-related well-being during the past two years. On average, they had experienced impaired work-related well-being at least once a month. Nearly 18% reported experiencing this at least a few times a week or even daily. According to the respondents, economic pressures had increased while the opportunities to practice ethically responsible social work had decreased. Increasing demands for efficiency predicted not only impaired work-related well-being directly, but also through decreasing opportunities to do ethically responsible social work. The role of the budget constraints in predicting impaired well-being became meaningful when these constraints were connected to decreasing opportunities to do ethically responsible social work.