The Influence of Advance Letters on Response in Telephone Surveys: A Meta-Analysis

Recently, the leading position of telephone surveys as the majormode of data collection has been challenged. Telephone surveyssuffer from a growing nonresponse, partly due to the generalnonresponse trend for all surveys and partly due to changesin society and technology influencing contactability and willingnessto answer. One way to counteract the increasing nonresponseis the use of an advance letter. In mail and face-to-face surveys,advance letters have been proven effective. Based on the proveneffectiveness in face-to-face and mail surveys, survey handbooksadvise the use of advance letters in telephone surveys. Thisstudy reviews the evidence for this advice and presents a quantitativesummary of empirical studies on the effectiveness of advanceletters in raising the response rate for telephone surveys.The major conclusion is that advance letters are also an effectivetool in telephone surveys, with an average increase in responserate (RR1) from 58 percent (no letter) to 66 percent (advanceletter), and an average increase in cooperation rate (COOP1)from 64 percent (no letter) to 75 percent (advance letter).     

Experiments with incentives in telephone surveys

In an effort to counter the problem of noncooperation, survey organizations are offering incentives to respondents with increasing frequency, some at the outset of the survey, as has traditionally been done in mail surveys, and some only after the person has refused, in an attempt to convert the refusal. This article reports on a series of experiments carried out over a period of about 2 years with a monthly telephone survey, the Survey of Consumer Attitudes, in an effort to increase response rates or reduce interviewer effort. We report on experiments with prepaid versus promised incentives; advance letters; and advance letters with prepaid incentives; and we also report on the effects of incentives on response quality, sample composition, response bias, interviewer and respondent expectations, and costs.     

Effects of an introductory letter on response rates to a teen/parent telephone health survey

The authors conducted a pilot study in preparation for a larger investigation that will rely on telephone surveys to assess select health behaviors of teens and their parents, with a focus on indoor tanning. This study used a randomized design to assess the impact of a presurvey letter on response rates to a telephone survey, as well as prevalence estimates of ever having used an indoor tanning facility. The letter did not have a significant effect on response rates or prevalence estimates in the two cities under study. Findings suggest that researchers should not assume that a letter sent in advance of a telephone survey will necessarily increase response rates enough to justify their use.     

USING DUAL FRAME DESIGNS TO REDUCE NONRESPONSE IN TELEPHONE SURVEYS

This article reports on the results of a series of experimentsdesigned to improve response rates for telephone surveys. Inthree surveys telephone households were selected using bothstandard random digit dialing (RDD) techniques and lists oftelephone numbers purchased from a commercial firm. In the RDDportions of the samples "cold contact" interviewing methodswere used; in the list frame portions advance letters were mailed,and the listed household name was used in the introduction.Experiments were designed to test the effects on response ratesof the advance letters and use of the listed household nameas a means of establishing rapport. The advance letters increasedresponse rates, but no difference could be attributed to theuse of names. The mixture of RDD and list sampling techniquesis also used to evaluate the effects of relative response rateson substantive findings. The cost consequences of these dualframe designs are assessed along a number of dimensions, andthe cost and error components of these designs are discussed.     

Do Advance Letters Improve Preelection Forecast Accuracy?

The survey methodology literature has debated whether advanceletters to potential survey respondents will reduce nonresponsebias and thereby improve the accuracy of preelection forecasts.This research note analyzes the results of experiments conductedin Maryland, New York, and Pennsylvania in which advance letterswere sent to a random sample of potential survey respondentsto 2002 preelection surveys. We find a significant increasein the overall response rate, although notably less than inpast studies. However, the advance letters did not improve therepresentativeness of survey respondents or the accuracy ofthe election forecasts.     

Personalization of Mail Surveys for General Public and Populations with a Group Identity: Results from Nine Studies*

Abstract The effect of personalization on mail survey response rates was examined in nine studies that included 17 comparisons under several research conditions. A study of this variable across multiple experiments in five agricultural experiment stations was undertaken because of conflicting results from previous research and from concern that the effectiveness of personalization might have decreased over time. Results show that, while response to general public surveys appeared to increase modestly across all treatment groups, there was no positive effect for populations in which a group identity (e.g., Dear Oregon Gardner or Dear ATV Owner) is employed to address respondents in cover letters. Personalization appears to remain useful for improving response in surveys of the general public.     

Response Results of Lottery Buyer Behavior Surveys: In-Home vs. Point-of-Purchase Interviews

This paper reports the response results of surveys on lotterypurchasing behavior. Data are based on answers generated fromtwo probability samples: a national, random sample of 1,646households (in-home survey) and a point-of-purchase sample of520 respondents. Results show that answers to the point-of-purchasesurvey were more accurate at half the cost of in-home interviews.The surveys were undertaken in Israel, but have implicationsfor those wishing to sample lottery purchasers in other countries.     

Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

An experiment to test the feasibility and quality of a web-based questionnaire of teachers

The use of web-based surveys to gather information from teachers has become increasingly common primarily based on the premise that they can reduce costs. Yet, relatively, little is known about the quality or cost effectiveness of web-based versus mail surveys for teachers. To study the efficacy of web-based teacher surveys, the author randomly assigned a nationally representative sample of 877 elementary school teachers to a paper or web survey mode. The cost savings from the web administration were not enough to offset the loss of sample due to missing/inaccurate email addresses and the lower response rates that resulted from the web survey.     

Homeless Affiliation,Location, and Sender Address in Return of Lost Letters

A field study used 1008 “lost letters” to test the hypotheses that the differences in returned responses would be attributable to the addressee (i.e., ending homelessnesscontrol or preventing it among youth, veterans, and lesbians) and especially to a socially deviant addressee, that cultural differences between distribution locations might influence returned responses of lost letters, and that sender address (i.e., urban, rural, control) would also influence returned responses. Both addressees’ affiliations and distribution location were associated with returned responses, with the lowest rates being for the Prevent Lesbian Homelessness Program affiliate overall and for the city of Pensacola, respectively. Antigay prejudice toward a deviant group may still be strong even when a helping act can be performed without intimate contact involving members of a deviant group. Neither distribution location nor the sender or return address conditions were associated with different returned responses. Further testing of the lost letter technique, as a measure of public response, may be warranted.     

Distal and Proximal Influences on the Risk of Extramarital Sex: A Prospective Study of Longer Duration Marriages

Previous models of the risk of extramarital sex (EMS) rely largely on cross-section samples and retrospective reporting. This may well conflate causes with consequences of EMS in the same model. Instead, this study employs panel data with an event-history approach to re-assess the influences on the risk of EMS. The sample consists of 1,270 married respondents, with no prior history of EMS, who were followed up in five subsequent surveys spanning a 20-year period. The quality of the conjugal bond emerged as a paramount influence on the outcome. The hazard of EMS was higher for respondents who had ever experienced a trial separation, reported marital violence, scored higher on a marital instability index, or spent less time in activities with the spouse. The risk of EMS was lower the longer respondents had been married at baseline, the longer the duration since baseline, and the greater the respondent's religiosity.     

Suicidal behavior among members of Gamblers Anonymous

A national sample of 500 members of Gamblers Anonymous was surveyed by mail in order to gather information on suicidal history. One hundred sixty two usable surveys were returned representing 32.4% of the original sample. Compulsive gamblers who had a history of suicidal preoccupation began gambling at an earlier age than nonsuicidal gamblers and were more likely to have stolen to support their gambling. They also tended to have addicted relatives and children more than nonsuicidal gamblers did. The data suggest that those gamblers who had been suicidal tend to be more serious gamblers than nonsuicidal respondents.     

Does Telephone Audio Computer-Assisted Self-Interviewing Improve the Accuracy of Prevalence Estimates of Youth Smoking? Evidence from the UMass Tobacco Study

Despite their advantage for obtaining representative samplesof adolescents, telephone surveys have been regarded as an inferiormethod for collecting data on youth tobacco use because theyyield lower estimates than school-based, self-administered surveys.Although no gold standard for smoking estimates exists, thelower estimates in telephone surveys have been attributed tounderreporting due to youths’ concern that parents orothers may overhear their responses. Telephone audio computer-assistedself-interviewing (T-ACASI) is a cost-effective method for obtaininga representative sample of youths and provides increased privacyfor the respondent. We hypothesize that using T-ACASI wouldencourage youths to report more fully smoking behavior comparedto traditional interviewer-administered telephone methods. Ouranalysis further assesses whether respondent age, gender, race/ethnicity,and parental attitude toward smoking moderated the relationshipbetween survey mode and smoking reports. Using data from a statewidetobacco use survey that randomly assigned youth respondentsto either T-ACASI or interviewer-administered modes, we findthat youths were more likely to report smoking behaviors inT-ACASI mode and that this was especially true for girls, particularlythose who believed their parents would disapprove strongly oftheir smoking. The findings suggest that traditional telephonesurveys may underestimate smoking prevalence in most girls bya factor of two, and that a technique for ensuring privacy forthese respondents is an important component of effective telephonesurvey methodology.     

RESIDENTIAL MOBILITY AND ATTEMPTED SUICIDE AMONG ADOLESCENTS: An Individual-Level Analysis

Building on the Durkheimian legacy's emphasis on social integration as a determinant of suicidal behavior, many macrolevel studies have observed an association between aggregate rates of geographical mobility and suicide, but little research has explored this connection at the individual level. We use data from 9,594 respondents who participated in two waves of the National Longitudinal Study of Adolescent Health to examine the effect of recent residential mobility on attempted suicide one year later and to explore the mechanisms that potentially transmit this effect. We find that among adolescent girls, recent movers are about 60 percent more likely than nonmovers to report having attempted suicide during the following year and that this difference cannot be readily explained by mover versus nonmover differences in preexisting demographic and family background characteristics. Some of the apparent effect of residential mobility on females' risk of attempting suicide operates through higher rates of victimization and delinquency, lower levels of school attachment, higher rates of social isolation, and a tendency for movers to associate with peers who exhibit delinquent behaviors and who themselves have attempted suicide. In contrast, we find no evidence that mobile female adolescents' deficit of parental social capital or lower levels of school engagement can account for the difference in attempted suicide risk between movers and nonmovers. We also find that residential mobility is not significantly associated with suicide attempts among adolescent males.     

The effects of response rate changes on the index of consumer sentiment

From 1979 to 1996, the Survey of Consumer Attitudes response rate remained roughly 70 percent. But number of calls to complete an interview and proportion of interviews requiring refusal conversion doubled. Using call-record histories, we explore what the consequences of lower response rates would have been if these additional efforts had not been undertaken. Both number of calls and initially cooperating (vs. initially refusing) are related to the Index of Consumer Sentiment (ICS), but only number of calls survives a control for demographic characteristics. We assess the impact of excluding respondents who required refusal conversion (which reduces the response rate 5-10 percentage points), respondents who required more than five calls to complete the interview (reducing the response rate about 25 percentage points), and those who required more than two calls (a reduction of about 50 percentage points). We found no effect of excluding any of these respondent groups on cross-sectional estimates of the ICS using monthly samples of hundreds of cases. For yearly estimates, based on thousands of cases, the exclusion of respondents who required more calls (though not of initial refusers) had an effect, but a very small one. One of the exclusions generally affected estimates of change over time in the ICS, irrespective of sample size.     

Response Effects in the Electronic Survey

This report examines the electronic survey as a research tool.In an electronic survey, respondents use a text processing programto self-administer a computer-based questionnaire. As more peoplehave access to computers, electronic surveys may become widespread.The electronic survey can reduce processing costs because itautomates the transformation of raw data into computer-readableform. It can combine advantages of interviews (e.g., prompts,complex branching) with those of paper mail surveys (e.g., standardization,anonymity). An important issue is how the electronic surveyaffects the responses of people who use it. We conducted anexperimental sample survey on health attitudes, behaviors, andpersonal traits using two forms of administration: electronicand paper mail. Closed-end responses in the electronic surveywere less socially desirable and tended to be more extreme thanwere responses in the paper survey. Open-ended responses thatcould be edited by respondents were relatively long and disclosing.These findings are consistent with other research on computer-mediatedcommunication, raising general issues about using computersto collect self-report data.     

The Cost-Effectiveness of Alternative Advance Mailings in a Telephone Survey

Advance letters are being used increasingly to combat the declinein response rates, but their effectiveness depends partly ontheir being read. In the face of the post-9/11 anthrax mailingscare, the Office for Survey Research (OSR) converted to usingadvance postcards rather than advance letters. Subsequently,the OSR conducted an experiment to test the effectiveness ofsending letters, postcards, or nothing to directory-listed numbersselected in a large random digit dial (RDD) telephone survey.In terms of response rates, the results clearly indicate thatsending advance letters is more effective than sending postcards,which, in turn, is more effective than sending nothing. A detailedcost-effectiveness evaluation indicates that sending lettersactually saves money, despite the increased costs associatedwith the mailings, compared to sending postcards; sending postcardsis also more cost-effective than not sending anything.     

Möglichkeiten und Probleme des Einsatzes postalischer Befragungen

The research focuses on the strategies in mail surveys that lead to high response rates and on the social characteristics of the respondents that are recruited into the survey through reminders. Empirical basis are surveys in the East German cities of Chemnitz, Dresden and Leipzig with an original sample size of ca 2000 persons per city and up to three reminders. The response rate varies between 68 and 70 percent and surpasses the usual response rates in face to face and telephone surveys done in German cities. In contrast to common assumptions in the literature it has no effect on the response rate whether stamps are used on the address and return envelope, neither do original versus scanned signatures make a difference and whether financial incentives are used in the late reminders. Through the reminders especially respondents with low education and of little political interest are recruited into the survey.     

Problem Gambling Within a Chinese Speaking Community

The purpose of this study was to obtain preliminary data on pathological gambling rates within a metropolitan Chinese community to determine if more costly extensive epidemiological surveys were justified. Two thousand survey questionnaires were distributed to parents through children attending a local Chinese speaking school. A response rate of 27.4% was obtained. Over a quarter of respondents were born overseas in either Mainland China or Hong Kong. Results indicated that gambling was not a popular activity with 60.0% of respondents stating that they never gambled. Of those who gambled, a third of respondents identified lotto as the preferred form. Using a Chinese translation version of the SOGS and a cut-off score of 10, a prevalence estimate of 2.9% for pathological gambling was found with males showing a higher rate (4.3%) as compared to females (1.6%). Respondents reporting a prior history of gambling in their country of origin were more likely to be classified as probable pathological gamblers. Despite the use of a sample of convenience, changes to the wording of two items in the Chinese translation of the SOGS and the possibility of false positive cases in the present study, it is concluded that further research into problem gambling in this community should be undertaken.     

Disability and going green: a comparison of the environmental values and behaviours of persons with and without disability

Some studies have suggested that socially excluded groups exhibit different environmental values and behaviours from mainstream society. To consider this in the context of disability, a survey was implemented to investigate environmental values and ecological behaviour. A high percentage of participants considered themselves to have a disability, long‐standing illness or infirmity. Over half of the sample had restricted physical mobility. The study used the revised New Ecological Paradigm Scale, a widely used measure of pro‐environmental orientation, and the General Ecological Behaviour scale to assess ecological behaviours. No significant differences were found between respondents with/without disability for either their general environmental attitudes or their general ecological behaviour. However, a positive correlation was found between level of physical mobility and general ecological behaviour. The results also show significant differences in the ecological behaviours of those respondents living dependently and those living independently; respondents living in care scored lower on the ecological behaviour scale.