Interviewer Effects in Telephone Surveys

The interviewer effects for selected items from a number ofnational polls conducted by CBS News and The New York Timesin 1980 were examined. These effects were found to be generallyquite small and somewhat inconsistent from poll to poll. Theinconsistencies were explained by variable associations withthe nonrandom regional distribution of respondents and the politicalcontext in which the mea surements were obtained. There wassome evidence of respondent-interviewer interactions for certainitems.     

The Effects of Material Incentives in Mail Surveys: Two Studies

Two studies were undertaken on the effects of including a materialnonmonetary incentive in mail surveys, using various samplesof the general public in the Netherlands. The results show thatnonmonetary incentives produce a higher initial response rate,but follow-ups reduced the effect of the incentive to a nonsignificantratio. Inclusion of the incentive produced no response biasand little volunteer bias. Results from these studies offera possible explanation for why past studies on nonmonetary incentiveshave often shown positive effects: they were conducted usingmethods that produced low response rates. When methods thatproduce high response rates are used, the effect of nonmonetaryincentives on response rate disappears. Finally, the use ofmonetary incentives in mail surveys with a high base responseis discussed.     

Estimating Response to Follow-ups in Mail Surveys

A simple method for estimating the effectiveness of successivefollow-ups in mail surveys is presented.     

Interviewers' Voices and Refusal Rates in Telephone Surveys

Interviewers' voice characteristics are related to refusal ratesin telephone interviews. Judges reliably rated vocal and personalcharacteristics of telephone interviewers from recordings ofthe first one-half minute of interview introductions. Interviewerswith low refusal rates were judged as having comparatively higherpitched voices, greater ranges of variation in pitch, greaterloudness, faster rates of speaking, and clearer and more distinctpronunciation. They also were judged as more competent and ashaving a more positive approach to the respondent and the interview.     

Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys

The high costs and largely unknown error properties of cellulartelephone interviews make screening for cell-only adults a potentiallyattractive option in dual frame RDD surveys. Screening out adultswith landline telephones from the cellular sample does not affectthe coverage properties of a dual frame survey, but it may affectother sources of error, especially nonresponse. In this study,data from a 2006 dual frame RDD survey conducted for the PewResearch Center, the Associated Press, and AOL are used to evaluatethe effects of implementing a cell-only screener on both thebias and variance of weighted survey estimates. The effect ofscreening appears to be minimal so long as an adjustment fortelephone service is included in the weighting method. Resultsof an attempt to correct for residual nonresponse due to inaccessibilityare also discussed.     

Actors and Questions in Telephone and Personal Interview Surveys

This paperconcentratesontwoaspectsof telephone survey administration:(1) the respondent's reactions to the request for an interviewand to the interview itself, and (2) the properties of questionsusing response cards in personalinterviews and adaptations ofthese questions for telephone use.     

The Consequences ofLarge Monetary Incentivesin Mail Surveys of Elites

Research concerning incentives and responses to mail questionnairesusually deals with the effects on response rates of paymentsof less than a dollar and with mass respondents. This articlereports the results of an international mail survey of eliterespondents using incentives ranging from 0 to 50 dollars. Thestudy found that larger incentives have a slight positive effecton the response rate and a substantial effect on response quality.The implications of these findings for cost-effective researchare discussed.     

Question Order and Fair Play: Evidence of Even-Handedness in Rural Surveys1

The norm of even-handedness is a question-order effect known to occur in face-to-face interviews when respondents favor one of two complementary or competing opinion questions, both written at the same level of specificity. After defining and illustrating the concept, evidence of even-handedness in opinion questions is explored in a Montana mail survey and in recall of recent behavior in Iowa telephone interviews. The Montana survey provides evidence of education effects that are not consistent with previous theories of cognitive sophistication, suggesting that future research must consider the interaction between substantive issues and cognitive processes. The Iowa poll provides evidence that respondents' recall of their recent behaviors and the behaviors of their partner also are subject to context effects. This poll also provides evidence that even-handedness operates independently of social desirablity, especially among women.     

Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults

The proportion of adults with only wireless telephones is growingrapidly. Using 2006 data from the National Center for HealthStatistics’ National Health Interview Survey, this articleis among the first to reveal that noncoverage of this populationcan result in nonnegligible bias for traditional random-digit-diallandline telephone surveys that do not call wireless telephonenumbers. In 2006 in the United States, 17 percent of low-incomeadults with household income below 200 percent of the federalpoverty thresholds, 25 percent of young adults aged 18–29years, and 32 percent of low-income young adults lived in householdswith only wireless telephones. Within each of these three subgroups,we compared wireless-only adults and adults with landline telephoneson demographic characteristics and 13 key indicators of healthstatus, health behaviors, health care service use, and healthcare access. Even after statistical adjustments that accountfor demographic differences between adults living in householdswith and without landlines, telephone surveys of landlines willunderestimate the prevalence of health behaviors, such as bingedrinking, smoking, and HIV testing. Obesity may be overestimatedand physical activity may be underestimated for low-income youngadults. No significant bias is predicted for other measuresof health status and health insurance coverage. Sample weightingprocedures that incorporate adjustments for multiple demographiccharacteristics are necessary to help attenuate coverage biasin traditional telephone surveys, but may not be sufficientfor behavioral risk factor surveys of low-income and young adults.     

Estimating the Nonresponse BiasDue to Refusals in Telephone Surveys

This paper attempts to aid the process of accumulating the necessaryinformation for making more informed judgments about the effectsof nonresponse under different conditions. Two measures, whichpermit quantifiable nonsubjective assessment of the effectsof nonresponse on sample estimates, are introduced and are usedto examine the effects of respondent refusals in a random-digit-dialedgeneral population telephone survey of over 1,200 householdsas the response rate isincreased from 74.5 percent to 86.8 percent.By applying these measures under a wide range of conditions,the adequacy of various response rates may be assessed and morerational decisions made about the costs and benefits of devotingextraordinary resources to minimizing nonresponse.     

The Influence of Advance Letters on Response in Telephone Surveys: A Meta-Analysis

Recently, the leading position of telephone surveys as the majormode of data collection has been challenged. Telephone surveyssuffer from a growing nonresponse, partly due to the generalnonresponse trend for all surveys and partly due to changesin society and technology influencing contactability and willingnessto answer. One way to counteract the increasing nonresponseis the use of an advance letter. In mail and face-to-face surveys,advance letters have been proven effective. Based on the proveneffectiveness in face-to-face and mail surveys, survey handbooksadvise the use of advance letters in telephone surveys. Thisstudy reviews the evidence for this advice and presents a quantitativesummary of empirical studies on the effectiveness of advanceletters in raising the response rate for telephone surveys.The major conclusion is that advance letters are also an effectivetool in telephone surveys, with an average increase in responserate (RR1) from 58 percent (no letter) to 66 percent (advanceletter), and an average increase in cooperation rate (COOP1)from 64 percent (no letter) to 75 percent (advance letter).     

RRT Meets RDD: Statistical Strategies for Assuring Response Privacy in Telephone Surveys

Despite its growing role in survey research, the telephone surveyhas been largely neglected with respect to the design and testingof statistical procedures for assuring response privacy (e.g.,randomized response techniques). This paper demonstrates thatsuch procedures are no less feasible to administer in telephonesurveys than in face-to-face surveys. Both randomized responsetechniques and alternatives to randomized response are considered.     

Race-of-Interviewer Effects in Telephone Interviews

Previous studies have found a race-of-interviewer effect onsurvey questions dealing with racial issues. This effect hasbeen found in both personal interviews and on questionnairesfilled out in the presence of an interviewer. This study examineswhether a race-of-interviewer effect is also present in telephoneinterviews. The results show that a race-of-interviewer effectdoes occur in telephone interviews on racial questions.     

Response Effects in Sex Surveys

Two potential problems in sex surveys—respondent refusalsand invalid reporting—are examined for national and areaprobability samples of males and females. Little support isfound for the contention that sexual topics constitute a sensitiveresearch area in terms of respondent cooperation and candor.It is found that (1) respondent cooperation, as indicated byrefusals and terminations, is neither more nor less problematicin sex research than in surveys on other topics; (2) respondentreports of sexual matters are not systematically related tointerviewer characteristics such as gender, sexual experience,or technical competence; (3) respondent candor, when self-reported,also is not associated systematically with either interviewercharacteristics or demographic and sexual characteristics ofrespondents. This analysis does identify, however, evidencethat (4) sexual matters often are sensitive topics for interviewersand principal investigators, leading both to interviewer turnoverand the creative construction of unobtrusive data collectionprocedures.     

Increasing Response Rates to Telephone Surveys: A Door in the Face for Foot-in-the-Door?

Several experimental studies have noted heightened cooperationwith a request for those who initially are given a smaller request.The technique, labeled the foot-in-the-door procedure, is appliedto a telephone survey design which asks two questions of samplehouseholds in a call prior to the contact requesting the fullinterview.     

Visual Context Effects in Web Surveys

There are many examples of context effects in survey measurement.Responses to survey questions can be shaped by the order ofquestions, the format of response options, the broader surveyenvironment, and so on. For Web surveys, the inclusion of visualimages is a trivial design issue, but may have consequencesfor the responses obtained because they change the visual context.We report a series of experiments examining how responses maybe affected by the use of images in Web surveys. Specifically,we examine the effect that pictures of a healthy woman exercisingversus a sick woman in a hospital bed have on self-rated health.We replicated the experiments in three different surveys, varyingsuch factors as the size and placement of the image and thelocation of the question within the questionnaire. In general,we find that when exposed to a picture of a fit woman, respondentsconsistently rate their own health lower than when exposed toa picture of a sick woman.     

Question Order Effects on Subjective Measures of Quality of Life1

The effects of question order on respondents' ratings of general and specific aspects of community life were assessed using data from separate mail surveys in Montana and Pennsylvania, The samples differed in locale, composition, and size and the relevant questions varied in number, format, and specific focus. Nevertheless, for both data sets the general question was more likely to be answered and more likely to receive positive responses when it was asked after—rather than before—the specific questions. There was some indication that carryover from the specific items to the general question responses were somewhat greater for those questions asked most recently and less for those asked earlier. Implications of these findings are discussed.     

Surveys on Surveys: Limitations and Potentialities

Data from two surveys on surveys in Waterloo, Ontario were usedto assess attitudes toward the survey. Questions in the twosurveys inquired into motives for refusing/consenting to interviews;attitudes toward past experience with surveys; likes/dislikesof form of contact, question format, and topic; perception ofsocial pressure to respond to surveys; views on the social legitimacyof polling and on legislative controls for surveying. The surveysalso collected response histories. Analysis of the attitudesbegan with a factor analysis of 12 items, which identified fourfactors descending from the general to the particular. It wasfound that attitudes are related to exposure to surveying, i.e.,the more times a person reports requests for survey cooperation,the more unfavorable is his or her attitude toward the method.On the whole, however, the author finds that the evidence presentedbelies pessimistic views about surveys on surveys.     

Personalization of Mail Surveys for General Public and Populations with a Group Identity: Results from Nine Studies*

Abstract The effect of personalization on mail survey response rates was examined in nine studies that included 17 comparisons under several research conditions. A study of this variable across multiple experiments in five agricultural experiment stations was undertaken because of conflicting results from previous research and from concern that the effectiveness of personalization might have decreased over time. Results show that, while response to general public surveys appeared to increase modestly across all treatment groups, there was no positive effect for populations in which a group identity (e.g., Dear Oregon Gardner or Dear ATV Owner) is employed to address respondents in cover letters. Personalization appears to remain useful for improving response in surveys of the general public.