The effect of interviewer evaluation of respondents’ attitudes and understanding on future survey participation in a panel study

This study explored whether the interviewers’ judgments of respondents’ performances in an initial survey can predict survey participation for a second survey among the same group of respondents in a panel study. Specifically, this study examined two interviewer variables: the interviewers’ judgments of (1) the respondents’ understanding of survey questions and (2) the respondents’ attitudes toward the survey. The analyses revealed that respondents who were rated to have an excellent understanding of the survey questions and friendly attitudes were more likely to respond to the second interview request than those with a poor understanding and unfriendly attitudes, after controlling for the characteristics of the respondents. Of these two, the judgment of the respondents’ attitudes was a stronger predictor. However, interviewer’s judgments are not correlated with the primary survey outcome or the data quality. This suggests limited value of the interviewer observation.     

Interviewing Changes Attitudes--Sometimes

To examine the effects of interviewing respondents were questionedabout either cancer or burglary prevention in an interview,and later they received either relevant information or no information.Changes in attitudes were measured in a second interview, andthe results showed that asking questions about cancer changedrespondents' attitudes toward cancer, but interviewing aboutburglary prevention did not change attitudes toward crime. Theevidence suggests that interview effects will occur when therespondent's attitudes and information are unfocused or ambiguousand the topic is important. Theoretically, this can be attributedto competence motivation and a postulated desire to maintainself-esteem in the interview situation     

Incorporating eye tracking into cognitive interviewing to pretest survey questions

In this study, we investigated whether incorporating eye tracking into cognitive interviewing is effective when pretesting survey questions. In the control condition, a cognitive interview was conducted using a standardized interview protocol that included pre-defined probing questions for about one-quarter of the questions in a 52-item questionnaire. In the experimental condition, participants’ eye movements were tracked while they completed an online version of the questionnaire. Simultaneously, their reading patterns were monitored for evidence of response problems. Afterward, a cognitive interview was conducted using an interview protocol identical to that in the control condition. We compared both approaches with regard to the number and types of problems they detected. We found support for our hypothesis that cognitive interviewing and eye tracking complement each other effectively. As expected, the hybrid method was more productive in identifying both questionnaire problems and problematic questions than applying cognitive interviewing alone.     

Impact of Repeated Questioning on Interviewers: Learning From a Forensic Interview Training Project

Forensic interviewers have a difficult job with high risk for career burnout and secondary trauma. Few studies have addressed how new forensic interviewers or trainees experience repeated questioning and multiple interviews. This study simulated the process of training new forensic interviewers through the creation of two interview videos in which social work graduate students participated as actors portraying the roles of interviewer and child. These films served as instructional aids preparing graduate social work students for professional child welfare roles while promoting research-based approaches to interviewing children about sexual abuse allegations. Qualitative data from two cohorts of student actors were collected to analyze interviewers’ perspectives on repeated questioning and interviews in child sexual abuse cases. Two themes were extracted from the subjects’ experiences: “It is emotionally taxing” and “Navigating the interviewer role is unexpectedly complex.” Exposure to repeated questions and multiple interviews affected the performance and confidence of the interviewers.     

Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias

The last 50 years have seen a gradual replacement of face-to-faceinterviewing with telephone interviewing as the dominant modeof survey data collection in the United States. But some ofthe most expensive and large-scale nationally funded, long-termsurvey research projects involving national area-probabilitysamples and long questionnaires retain face-to-face interviewingas their mode. In this article, we propose two ways in whichshifting such surveys to random digit dialing (RDD) telephoneinterviewing might affect the quality of data acquired, andwe test these hypotheses using data from three national modeexperiments. Random digit dialing telephone respondents weremore likely to satisfice (as evidenced by no-opinion responding,nondifferentiation, and acquiescence), to be less cooperativeand engaged in the interview, and were more likely to expressdissatisfaction with the length of the interview than were face-to-facerespondents, despite the fact that the telephone interviewswere completed more quickly than the face-to-face interviews.Telephone respondents were also more suspicious about the interviewprocess and more likely to present themselves in socially desirableways than were face-to-face respondents. These findings shedlight on the nature of the survey response process, on the costsand benefits associated with particular survey modes, and onthe nature of social interaction generally.     

EFFECTS OF QUESTIONNAIRE DESIGN ON THE QUALITY OF SURVEY DATA

The importance of format, graphic layout, and question routinginstructions on the quality of survey data has been acknowledgedfor self-administered survey instruments, but the significanceof these factors for questionnaires administered by interviewershas not been recognized equally. This paper examines two studiesin which interviewers used different questionnaire designs toadminister the same set of survey questions to randomly selectedadult respondents in housing units that made up area probabilitysamples of the Detroit metropolitan area. The paper presentsempirical evidence of specific effects (questions skipped inerror, unprobed answers) directly attributable to the choiceof questionnaire design. The analysis shows that question nairedesign choices can either help or hurt the quality of data collectedby interviewers. Furthermore, the behaviors of experienced andinexperienced interviewers are affected in similar ways. Inother words, interviewing experience does not compensate forformat deficits in the design of survey instruments.     

Integrating qualitative and quantitative methods in survey research on aging: An assessment

The benefits of triangulation have gained widespread acceptance and several notable studies have taken a combined-methods approach. However, to date there have been few analyses of attempts to extend survey research to include a qualitative component. This paper reports on what is essentially a survey research design in which three types of interviewing are triangulated: close-ended questions in a structured interview; open-ended questions embedded in the interview; and follow-up conversational interviews employing open-ended questions. This combined-methods approach is assessed in relation to elaboration of answers, interpretation and context, discovery in the field, and divergence. The author concludes that each source of data, as well as the divergence in findings among them, makes a valuable contribution to the research process.     

A Study of Experimental Techniques for Telephone Interviewing

This investigation focused on the effects of experimental interviewingprocedures, which were developed in face-to-face interviews,on reporting in telephone contacts. Respondents in a nationalRDD sample (N=1,1054) were randomly assigned to be interviewedwith one of two experimental procedures, or a control technique.The experimental procedures involved two different combinationsof three interviewing techniques which had been found to beeffective in personal interviewing studies—commitment,instructions, and feedback. The techniques are designed to betterinform respondents about reporting tasks and to motivate themto perform the tasks well. The findings suggest that the experimentalprocedures can improve reporting in telephone interview, althoughthe effects in this study were not as strong as we expected.This research may be seen as a beginning step in the developmentof optimal telephone interviewing procedures.     

Assessing comprehension in a survey of public reactions to complex issues

This paper reports results of research to develop householdinterview procedures for obtaining reasonably informed publicjudgments about ethical issues in biomedical research. We describe(1) an interviewing strategy designed to enhance comprehensionamong survey respondents, and (2) a composite comprehensionindex based on three types of information from the interview.Using data from a cross-section sample, we identify subgroupsthat differ significantly with respect to scores on the comprehensionindex, and we then describe the observed levels of comprehensionin terms of the operations defining the index. The results demonstraleboth the feasibility and limits of obtaining informed ratherthan naive judgments about complex issues from lay respondents.     

Event history calendars and question list surveys: a direct comparison of interviewing methods

The research reported in this article provides the first direct experimental comparison between Event History Calendar (EHC; N=309; 84.4 percent response rate) and standardized state-of-the-art question list (Q-list; N=307; 84.1 percent response rate) interviewing methodologies. Respondents and 20 interviewers were randomly assigned to EHC and Q-list interviews that were conducted via telephone in the spring of 1998. All interviews asked for retrospective reports on social and economic behaviors that occurred during the calendar years of 1996 and 1997. Using data from the same respondents collected 1 year earlier on events reported during 1996 as a standard of comparison, the quality of retrospective reports on 1996 events from the 1998 administration of EHC and Q-list interviews was assessed. In comparison to the Q-list, the EHC condition led to better-quality retrospective reports on moves, income, weeks unemployed, and weeks missing work resulting from self illness, the illness of another, or missing work for these reasons in combination with other ones. For reports of household members entering the residence, and number of jobs, the EHC led to significantly more overreporting than the Q-list. Contingent on additional research that examines a wider range of reference periods and different modes of interviewing, the EHC may become a viable and potentially superior method to the Q-list in the collection of self-reported retrospective information.     

Comparing results from different styles of cognitive interviewing: ‘verbal probing’ vs. ‘thinking aloud’

Two styles of cognitive interviewing exist, thinking aloud (TA) and verbal probing (VP), but there have been few studies undertaken which compare the two. This paper reports on such a comparison, which was undertaken in order to understand the merits and drawbacks of each approach. VP finds more problems than TA. TA appears to be better at examining respondents’ understandings of the underlying concepts behind each question; VP is better for exploring specific terms in each question. TA interviews appeared to generate more information about how respondents used information from their past experiences to select codes in closed‐ended questions. However, in terms of the eventual redrafting of the questions respondents were asked about, there was very little difference between the VP and TA interviews.     

Privileging Fieldwork Over Interviews: Consequences for Identity and Practice

Qualitative sociologists typically privilege fieldwork over interviews. What happens to fieldworkers who now ask questions but no longer hang out? What about those who rely exclusively on intensive interviewing while participant observation remains the standard? The authors examine the negative consequences of privileging fieldwork for identity and practice, the unique contributions of in-depth interviewing, and the differences in the tales that fieldworkers and interviewers tell. An inclusive identity anchored to the analytical assumptions fieldworkers and interviewers share would increase qualitative researchers' confidence and lead them to do better work.     

The Effect of Interviewer Characteristics and Expectations on Response

This study reports on two sets of findings related to interviewereffects, derived from a national RDD sample of the adult population.The first of these concerns the effect of interviewer characteristicsand expectations on overall cooperation rates; the second, theeffect of interviewer characteristics and expectations on itemnonresponse and response quality. We found that interviewers'age, the size of the interviewing assignment, and interviewers'expectations all had a strong effect on overall cooperationrates; the relation of experience to response rate was curvilinearin this sample. Age and education have consistent but statisticallyinsignificant effects on item nonresponse. The effect of interviewers'expectations on responses within the interview resembles thatin earlier studies, but is less pronounced and less consistent.     

What influences respondents to behave consistently when asked to consent to health record linkage on repeat occasions?

This study constitutes the first longitudinal exploration of consent to link survey and administrative data. It examines variations in consent over time and explores the influence of the respondents’ characteristics (both observed and latent) and the impact of the interviewers on consent co-operation. Respondent inclination to consent is modelled as a latent construct. Most respondents behave consistently over time. However, this consistency is not driven by a strong inclination to consent but rather by the circumstances of the respondents at the time of the interview and by the impact of the interviewers themselves. The findings also show that the change in consent behaviour over time is a clear indication that consent should be treated as a dynamic phenomenon at the individual level.     

Interviewer Effect in a Survey of Drinking Practices*

This study investigates the relationship of interviewer characteristics to the responses of their interviewees in a survey in a relatively sensitive area—drinking practices. Generally, respondents seem to have reacted to easily ascertainable characteristics of the interviewers, such as sex, status (as measured by education, occupation, and clothing), and religion. The attitudes of the interviewer toward his interviewing were also related to respondents' reported drinking. It is concluded that interviewing is a social relationship of brief duration in which the respondent's performance reflects norms that tend to emerge based on the information each of the participants is able to learn about the other.     

Which probes are most useful when undertaking cognitive interviews?

This paper reports the use of verbal probes in cognitive interviews (CIs) undertaken to test the usefulness, validity and reliability of survey questions. Through examining the use of probes by three interviewers undertaking interviews as part the piloting of a cross-national crime survey, we examine which of the various types of probes used in CIs produce the most useful information. Other influences on interview quality are examined, including differences between interviewers and respondents themselves. The analyses rely on multi-level modelling and suggest that anticipated, emergent and conditional probes provide the most useful data. Furthermore, age, gender and educational levels appear to have no bearing on the quality of the data generated.     

Police interviews of sexual assault reporters: do attitudes matter?

Sexual assault is underreported in the United States. Survivors are often reluctant to make police reports for various reasons; one is fear of revictimization by criminal justice professionals. Conversely, police officers often lack skills for interviewing crime victims. Posttraumatic stress reactions among victims can exacerbate the problem. Although some victims prefer female interviewers, it is not known whether they are more skilled. A sample of 429 police officers completed a written survey testing their rape myth acceptance and knowledge of how to interview rape reporters. A significant relationship between rape myth acceptance and interviewing skill was discovered. Although officer gender was related to interviewing skill, the effect was mediated by rape myth acceptance. Specific officer behaviors related to high rape myth acceptance were identified. Implications for selection of police to conduct victim interviews were discussed.     

Research note The study of sensitive subjects

This note examines some research issues raised by studies of sensitive topics and the disclosure of highly personal and confidential information. The first part focuses on the research method of in-depth interviewing and covers some theoretical and methodological problems associated with the method, together with a discussion of some strategies adopted by researchers and respondents in the interview setting. The second part discusses some of the hazards of this type of research and argues for the need to protect respondents and researchers alike. These issues are discussed with reference to the author's own fieldwork experiences of three research projects.     

Interviewing Older Adults: A Comparison of Telephone and Face-to-Face Modalities

This article examines telephone interviewing of older adultsand compares it with face-to-face interviews. Specifically,the following issues are examined in several national surveys:(1) differences in age distributions between the samples ofadults reached in both modes; (2) explanations for potentialdifferences in age distributions; (3) differences between thetwo modes in demographic characteristics in the adults reached,in interview process and in response quality, and how thesemode differences vary by age of the respondents. Telephone surveystend to underrepresent older persons, and older persons whodo participate in a telephone survey are disproportion atelywell educated. Implications of the lower response rate amongolder persons are softened by the fact that reponse distributionsacross a range of questions show little difference by interviewmode between older persons and persons of other age groups.     

Shaping Clients' Answers: Departures from neutrality in care-staff interviews with people with a learning disability

What can happen when care staff interview clients with a learning disability? We examine tape-recordings of five questionnaire-based interviews designed to yield information on the clients' perceptions of the quality of the service provided to them. Of interest was the way in which the care staff, who were not formally trained in interview skills, delivered the 42-item questionnaire that formed the basis for the interview. It was discovered that interviewers replicated a number of non-neutral practices previously identified in a set of similar interviews administered by formally-trained professionals. They also introduced further deviations from neutral interviewing. The effect of these practices on the information recorded as the respondents' answers is discussed. We note that any interviewer is faced with a dilemma of choosing between literal (but potentially robotic and insensitive) and tailored (but potentially unstandardised and invalid) administration of a questionnaire. We argue that the deviations we see here show the interviewers falling on the side of 'liberal' administration. The net effect was arguably to prompt 'better' answers. When what is being recorded is an 'audit' of services provided to respondents, there is a real-life danger that their perceptions are being improved by what is ostensibly a neutral interview.