A comparison of methods for collecting self-report data on sensitive topics

Insufficient attention has been paid to whether disclosure rates of sensitive or stigmatizing information vary as a function of method of inquiry. Methods vary both in terms of the anonymity afforded the participant and the opportunity to make a connection with the researcher, both of which might affect participants' willingness to disclose such information. In this investigation, 215 undergraduate students were randomly assigned to complete identical questionnaires using one of the three most common methods of data collection (in-person interview, telephone interview, and paper-and-pencil questionnaire) or an automated telephonic data collection (ATDC) system. Questions on six topic areas of increasing social sensitivity (study habits, substance use, physical and sexual aggression, victimization and perpetration) were included. The results indicated that there were no differences in disclosure rates due to methods and no method by topic interaction, but the two telephonic methods both produced significantly higher participation rates than the two other methods. The results suggest that, at least for a college student sample, an automated telephonic system produces data comparable to that of more traditional methods, while offering greater convenience, economy, and participation.     

The relative costs and benefits of telephone interviews versus self-administered diaries for daily data collection

This article compares two methods of collecting daily data: self-administered diaries and telephone interviews. Study participants included 44 men and 56 women between the ages of 16 and 35 who participated in a larger study of drinking, drug use, and sexual activity. Participants were randomly assigned to either the written diary or the telephone interview conditions; question wording and format were identical in both conditions. Daily data were collected for a period of 8 weeks. Results indicate that although telephone interviews resulted in slightly more missed days of data collection, they generally yielded less item-level missing data, produced cleaner data and therefore were less costly to process, and were as palatable to participants as self-administered diaries. Except for reports of drinking and vegetable consumption, telephone and diary conditions did not differ in the amount of behavior reported; more drinking and vegetable consumption were reported with telephone interviews, however. Telephone interviews also imposed considerably higher overall personnel costs.     

Participant responses to retrospective surveys of child maltreatment: does mode of assessment matter?

This study examines the impact that different methods of assessing child maltreatment history may have on adult participants. A total of 334 female undergraduate students were randomly assigned to complete a retrospective measure of child sexual and physical abuse in one of three conditions: paper-and-pencil questionnaire, face-to-face interview, or computer-administered survey. Disclosure rates of abuse, psychological distress and mood change, preferences for assessment format, and perceptions of confidentiality were examined across the three assessment formats. Although disclosure did not vary by condition, participants with a history of abuse reported more distress and mood change than did nonvictims, particularly in the computer condition. Nevertheless, the computer condition was rated as the most preferred format and was viewed by participants as the most confidential means of assessing maltreatment history. Participants reporting abuse through interviews were more likely than those in other conditions to state a preference for another type of assessment format. The implications of these findings for abuse history research are discussed.     

Individual interviews or focus groups? Interview format and women’s self-disclosure

Individual interviews have traditionally been an important method of data collection in multiple disciplines, including psychology. However, research comparing individual interviews with focus groups has generated mixed results regarding which method is more effective in investigating sensitive topics. The purpose of the present study was to directly compare the two methods with respect to young women’s verbalizations on a sensitive topic, namely their thoughts and feelings related to body image. Female undergraduate students were randomly assigned to an individual interview (n = 32) or focus group (n = 33). Participants in the individual interviews disclosed more personal thoughts and feelings about body image and had more positive perceptions of the interpersonal climate of their sessions than participants in the focus groups. These findings suggest that individual interviews may provide women with a more supportive environment in which to discuss sensitive topics.     

Intimacy,rapport, and exceptional disclosure: a comparison of in-person and mediated interview contexts

While qualitative researchers increasingly accept online video interviews as a reliable method, many maintain concerns about rapport and data quality. Drawing on two separate interview projects conducted in private in-person settings, public in-person settings, and privately via Skype, we compare interview contexts with regard to rapport, suitability to sensitive topics, interview duration, and scheduling concerns raised by prior research. Analytical comparison of these two corpuses of data suggest, largely in contrast to previous literature, that (1) interviews conducted in private settings (either in-person or via Skype) result in more sharing of deeply personal experiences, and there is little difference in this exceptional disclosure between Skype and in-person private interviews; (2) interviewing via Skype produces neither reduction nor inappropriate excesses of rapport; and (3) Skype interviews are a popular choice among participants, did not result in shorter interview duration, and were not subject to greater rescheduling or cancellation.     

Survey mode effects on measured income inequality

We study the effect of interview modes on estimates of economic inequality which are based on survey data. We exploit variation in interview modes in the Austrian EU-SILC panel, where between 2007 and 2008 the interview mode was switched from personal interviews to telephone interviews for some but not all participants. We combine methods from the program evaluation literature with methods from the distributional decomposition literature to obtain causal estimates of the effect of interview mode on estimated inequality. We find that the interview mode has a large effect on estimated inequality, where telephone interviews lead to a larger downward bias. The effect of the mode is much smaller for robust inequality measures such as interquantile ranges, as these are not sensitive to the tails of the distribution. The magnitude of effects we find are of a similar order as the differences in many international and intertemporal comparisons of inequality.     

Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias

The last 50 years have seen a gradual replacement of face-to-faceinterviewing with telephone interviewing as the dominant modeof survey data collection in the United States. But some ofthe most expensive and large-scale nationally funded, long-termsurvey research projects involving national area-probabilitysamples and long questionnaires retain face-to-face interviewingas their mode. In this article, we propose two ways in whichshifting such surveys to random digit dialing (RDD) telephoneinterviewing might affect the quality of data acquired, andwe test these hypotheses using data from three national modeexperiments. Random digit dialing telephone respondents weremore likely to satisfice (as evidenced by no-opinion responding,nondifferentiation, and acquiescence), to be less cooperativeand engaged in the interview, and were more likely to expressdissatisfaction with the length of the interview than were face-to-facerespondents, despite the fact that the telephone interviewswere completed more quickly than the face-to-face interviews.Telephone respondents were also more suspicious about the interviewprocess and more likely to present themselves in socially desirableways than were face-to-face respondents. These findings shedlight on the nature of the survey response process, on the costsand benefits associated with particular survey modes, and onthe nature of social interaction generally.     

A Preliminary Investigation of a Telephone Administration of the Quality of Marriage Index

Follow‐up data collection presents many challenges for longitudinal research. Validating paper‐and‐pencil assessments for use via telephone may alleviate these issues. This study evaluates psychometric properties of the Quality of Marriage Index (QMI; Norton, 1983, Journal of Marriage and the Family, 45, 141) when administered by telephone and compares scores with the paper‐and‐pencil version. Participants included women with major depressive disorder and dyadic discord and their male spouses (n = 24 couples). Results suggest excellent scale reliability for the verbal QMI. However, a significant mean difference was found: participants scored on average 3 points higher on the telephone‐administered QMI than on the written QMI. Recommendations for future research and clinical implications are discussed.     

Mode effects in cultural domain analysis: comparing pile sort data collected via internet versus face-to-face interviews

This article tests whether collecting pile-sort data online produces results similar to those obtained with face-to-face methods. We collected pile sorts from 227 university students in the cultural domain of emotions. To test for mode and design effects, we randomly assigned participants to face-to-face or internet modes and to either a 15- or 30-item sort. We found no evidence of mode effects in semantic structure, but the level of agreement among respondents varied by mode, depending on the number of items in the task. Agreement was higher among online respondents in the 15-item task but among face-to-face respondents in the 30-item one. Because the central purpose of pile sorting is to elicit semantic structure, we conclude that online data collection is a viable option for most researchers, but the mode effect in level of agreement implies that other design elements may affect comparability of face-to-face and web-based methods.     

Methodological Comparisons Between CATI Event History Calendar and Standardized Conventional Questionnaire Instruments

Six hundred and twenty-six participants from the Panel Studyof Income Dynamics (PSID) were interviewed via telephone retrospectivelyabout life course events by either computer-assisted (CATI)event history calendar (EHC) or standardized CATI conventionalquestionnaire (CQ) methods, randomly assigned. Experimentalretrospective reports, for a reference period up to 30 years,were validated against reports provided annually from the PSIDcore interviews. Data quality outcome measures included variablesassociated with marriage, cohabitation, employment, unemployment,residential changes, and cigarette smoking. The EHC providedhigher quality retrospective reports for cohabitation, employment,unemployment, and smoking histories; the CQ provided betterdata quality for marriage history, although what variable wasbeing measured, instead of which method was being used, hadthe biggest impact on differences in data quality. Both EHCand CQ interviews lasted on average around one hour, with theEHC interviews being on average 10 percent longer. Interviewerspreferred the EHC interviews. In both EHC and CQ conditions,respondents generally enjoyed the interviews, and did not findquestions difficult. The costs and benefits of both EHC andCQ methods in the collection of life course retrospective reportsare discussed. Received for publication December 19, 2005. Revision received September 25, 2007. Accepted for publication September 25, 2007.     

A Study of Experimental Techniques for Telephone Interviewing

This investigation focused on the effects of experimental interviewingprocedures, which were developed in face-to-face interviews,on reporting in telephone contacts. Respondents in a nationalRDD sample (N=1,1054) were randomly assigned to be interviewedwith one of two experimental procedures, or a control technique.The experimental procedures involved two different combinationsof three interviewing techniques which had been found to beeffective in personal interviewing studies—commitment,instructions, and feedback. The techniques are designed to betterinform respondents about reporting tasks and to motivate themto perform the tasks well. The findings suggest that the experimentalprocedures can improve reporting in telephone interview, althoughthe effects in this study were not as strong as we expected.This research may be seen as a beginning step in the developmentof optimal telephone interviewing procedures.     

Use of Collateral Contacts in Child Custody Evaluations

ABSTRACT

Four hundred nineteen members of the Association of Family and Conciliation Courts (AFCC) identify themselves as custody evaluators in the 1999–2000 membership directory. One hundred of these custody evaluators were randomly selected for inclusion in this survey of the use of collateral contact interviews in child custody evaluations. Fifty-three of those selected responded to a brief questionnaire concerning their use and views of collaterals. All evaluators reported use of the collateral interview. Most practitioners reported using both personal and telephone interviews. Respondents indicated that they interview family and friends, but routinely place greater trust in more objective sources that have no obvious, vested interest in outcome. More experienced evaluators reported evolving toward use of written questionnaires for use in interviewing collateral sources for purposes of risk management as well as increasing relevance and convergent validity.     

Measuring depression in the community: a comparison of telephone and personal interviews

Interview method effects in response to the Center for Epidemiologic Studies-Depression Scale (CES-D) are compared among a community sample (N = 546) of adults randomly assigned to either telephone or in-person interviews. The interviews were conducted in Los Angeles during 1980. No significant differences were observed between the two interview methods in nonresponse to symptom items, preference for specific response categories, reliability, mean level of depression, or proportion classified as depressed. Furthermore, no significant interactions were found between sociodemographic characteristics and the method of interview.     

Telephone survey respondents' reactions to questions regarding interpersonal violence

Concerns have been raised regarding the appropriateness of asking about violence victimization in telephone interviews and whether asking such questions increases respondents' distress or risk for harm. However, no large-scale studies have evaluated the impact of asking such questions during a telephone interview. This study explored respondents' reactions to questions regarding violence in two large recently completed telephone surveys. After respondents were asked about violence, they were asked if they thought surveys should ask such questions and whether they felt upset or afraid because of the questions. In both surveys, the majority of respondents (regardless of their victimization history) were willing to answer questions about violence and were not upset or afraid because of the questions. More than 92% of respondents thought such questions should be asked. These results challenge commonly held beliefs and assumptions and provide some assurance to those concerned with the ethical collection of data on violent victimization.     

Effects and Costs of Tracing Strategies on Nonresponse Bias in a Survey of Workers with Low-Back Injury

Declining rates of participation are an increasing challengefor studies that involve telephone surveys. This study examinedthe costs of a telephone survey methodology that used increasinglyintensive tracing methods to track a pool of claimants who hadsustained occupational back injuries. It also compared the respondentsample to people who refused the survey and/or were not locatedor contacted. 3,181 claimants were drawn from a database maintainedby the Missouri Division of Workers’ Compensation (DWC)and 1,475 completed a telephone interview. The DWC databaseprovided data reflecting monetary and disability outcomes forall potential participants; telephone interviews provided additionalself-reported data. More intensive tracing strategies improvedthe representativeness of the sample, yielding more women andminority participants. Relative to less intensive techniques,advanced tracing efforts located people whose DWC records weremore similar to claimants not located. While the hands-on tracingefforts reduced the apparent bias of the respondent sample,costs increased as tracing strategies intensified – over$98,000 was spent to trace 1,027 claimants who were never locatedor contacted. The results suggest some guidelines that may facilitatedecision-making for researchers and funders who must balancethe trade-off between costs and nonresponse bias when planningsurveys.     

Interview effects in the reporting of domestic violence

Victims of domestic abuse sometimes have the opportunity to discuss their experiences in interviews with researchers and subsequently to report their victimization. Beyond the reporting decision—a non-trivial information–revelation decision—they frequently must decide whether to have someone they know present at the interview and whether to conduct the interview in person or by telephone. Having an associate present at the interview may provide a respondent victim greater comfort or social support than she might have conducting the interview by herself, but perhaps only to the extent that the abusive partner does not pose a continuing household threat. Undertaking a personal interview may facilitate a greater degree of empathy or deeper understanding from the information gatherer likely unavailable over the telephone, but personal interviews generally take more time and incur more significant marginal opportunity costs. This paper examines conceptually and empirically how the parameters of these interviews, endogenously determined by victims themselves, affect the decision to report domestic violence.     

INTERVIEW MODE EFFECTS IN SURVEYS OF DRUG AND ALCOHOL USE: A FIELD EXPERIMENT

This research explores the impact of interview mode on respondents'willingness to reveal illicit or undesirable behavior and mechanismsby which interview mode influences response tendencies. A fieldexperiment was designed to control mode effects due to samplingand screening so that the impact of response anonymity (throughuse of self-administered questionnaires ([SAQs]) and socialdistance in the interviewer-respondent relationship (telephonevs. personal communication) could be tested. Respondents aged18–45 were randomly as-signed to interview mode: telephone,face-to-face, or self-administered. Admission of illicit druguse and alcohol use was most likely in the personal mode withSAQs, slightly less likely in personal mode without SAQs, andleast likely in the telephone mode. The magnitude of the modedifferentials was larger for blacks than for whites, and largeramong respondents who are more mistrustful of others. Resultssupport the notion that response effects due to mode of interviewderive, at least in part, from interview mode differences inability to assuage respondents' confidentiality concerns. Greatersocial distance between interviewer and respondent in the telephoneinterview, compared with face-to-face communication, makes itmore difficult for the researcher to make convincing confidentialityguarantees. The response anonymity provided by SAQs also appearsto increase respondents' willingness to reveal sensitive behavior,especially among racial/ethnic minorities.     

Qualitative bereavement research: incongruity between the perspectives of participants and research ethics boards

A central feature of the majority of qualitative research is the interactive nature of data collection which generally involves direct and meaningful communication between the individuals conducting the research and the individuals participating in the research. This core aspect of data collection, however, is frequently flagged as the most concerning or potentially harmful aspect of qualitative bereavement research by research ethics boards. Further, there has been a tendency to conceptualize the bereaved as vulnerable and in need of protection in the research process. Instead of thinking that a research interview which explores the complex, personal issues of grief would potentially harm participants, it may be seen as potentially beneficial to participants when the therapeutic aspects of the interview are considered. Participants’ responses to the interview process in two bereavement studies are offered as illustrations to complement the literature on the potential for the researcher‐participant relationship and the research interview to be perceived as beneficial by bereavement research participants.     

Cognitive Appraisal of Performance Capability in the Prevention of Drunken Driving: A Test of Self-Efficacy Theory

This experiment tested predictions derived from self-efficacy theory by exposing participants to one of two public service announcements based on either symbolic modeling or persuasive efficacy information. Each message was designed to heighten participants' self-efficacy to prevent a friend from driving drunk. Participants in the symbolic modeling condition viewed a public service announcement that demonstrated how to dissuade a friend from driving drunk, and those in the verbal persuasion condition viewed an announcement that only advocated performing the task. A control announcement mentioned the consequences of arrest for drunken driving but contained no efficacy information. Data were gathered during laboratory sessions and during follow-up interviews 1 month later. Overall, laboratory findings supported the hypothesized ordered effects for sources of efficacy information: Symbolic modeling engendered greater efficacy expectations and behavioral intentions than did persuasive efficacy information, which in turn surpassed the control condition on some measures of self-efficacy, but not on behavioral intentions where neither condition differed. Follow-up data indicated that participants in the efficacy-information treatments were equally successful at dissuading a friend from driving drunk, whereas the controls were not.