Does Telephone Audio Computer-Assisted Self-Interviewing Improve the Accuracy of Prevalence Estimates of Youth Smoking? Evidence from the UMass Tobacco Study

Despite their advantage for obtaining representative samplesof adolescents, telephone surveys have been regarded as an inferiormethod for collecting data on youth tobacco use because theyyield lower estimates than school-based, self-administered surveys.Although no gold standard for smoking estimates exists, thelower estimates in telephone surveys have been attributed tounderreporting due to youths’ concern that parents orothers may overhear their responses. Telephone audio computer-assistedself-interviewing (T-ACASI) is a cost-effective method for obtaininga representative sample of youths and provides increased privacyfor the respondent. We hypothesize that using T-ACASI wouldencourage youths to report more fully smoking behavior comparedto traditional interviewer-administered telephone methods. Ouranalysis further assesses whether respondent age, gender, race/ethnicity,and parental attitude toward smoking moderated the relationshipbetween survey mode and smoking reports. Using data from a statewidetobacco use survey that randomly assigned youth respondentsto either T-ACASI or interviewer-administered modes, we findthat youths were more likely to report smoking behaviors inT-ACASI mode and that this was especially true for girls, particularlythose who believed their parents would disapprove strongly oftheir smoking. The findings suggest that traditional telephonesurveys may underestimate smoking prevalence in most girls bya factor of two, and that a technique for ensuring privacy forthese respondents is an important component of effective telephonesurvey methodology.     

A Comparison of Survey Techniques on Sensitive Sexual Behavior in Italy

This article compares two national surveys carried out through the most commonly used procedures in Italy: CATI (computer-assisted telephone interviews) and SAQ-FI (self-answered questionnaires following interviews). Both surveys ask two identical questions concerning sensitive sexual behavior: early age at first intercourse and same-sex attraction. The SAQ-FI survey had both unit non-response and item non-response rates much lower than the CATI survey. Moreover, in the CATI survey, the groups with highest item non-response rates were also the groups with the lowest proportions of early intercourse and homosexual attraction. In addition, a differential analysis of the respondents produced diverse results for the two surveys. This is especially true of results by gender for same-sex attraction: Such behavior is more common among men (3.1%) than women (2.9%), according to the CATI survey, whereas the opposite is true of the SAQ-FI survey (6.1% of men vs. 7.7% women). In Italy at the beginning of the 21st century, CATI surveys reveal a lower level of early intercourse and same-sex attraction than SAQ-FI surveys. This article argues that the CATI survey underestimates the true level of these sensitive sexual behaviors in the Italian population.     

Monitoring youth behavior in response to structural changes. Alternative approaches for measuring adolescent drinking

Random digit dial (RDD) telephone and self-administered school-based surveys of drinking and drinking and driving were compared for adolescents from two Northern California communities. The RDD and school-based surveys resulted in very different samples. The telephone sample contained a greater proportion of European Americans and a smaller proportion of Asian Americans and "other" ethnicities. Respondents to the telephone sample also tended to be older and of higher socioeconomic status (SES). The telephone survey captured very few school dropouts. Moreover, it resulted in lower self-reports of drinking and drinking and driving. Survey mode appeared to influence respondents equally, regardless of their gender, age, ethnicity, or SES.     

SAME-GENDER SEX IN THE UNITED STATES IMPACT OF T-ACASI ON PREVALENCE ESTIMATES

Well-conducted telephone surveys provide an economical meansof estimating the prevalence of sexual and reproductive behaviorsin a population. There is, however, a nontrivial potential forbias since respondents must report sensitive information toa human interviewer. The National STD and Behavior MeasurementExperiment (NSBME) evaluates a new survey technology—telephoneaudio computer-assisted self-interviewing (T-ACASI)—thateliminates this requirement. The NSBME embedded a randomizedexperiment in a survey of probability samples of 1,543 U.S.and 744 Baltimore adults ages 18 to 45. Compared with NSBMErespondents interviewed by human interviewers, respondents interviewedby T-ACASI were 1.5 to 1.6 times more likely to report same-gendersexual attraction, experience, and genital contact. The impactof T-ACASI was more pronounced (odds ratio = 2.5) for residentsof locales that have historically been less tolerant of same-gendersexual behaviors and for respondents in households with children(odds ratio = 3.0).     

ADOLESCENTS' INCONSISTENCY IN SELF-REPORTED SMOKING: A COMPARISON OF REPORTS IN SCHOOL AND IN HOUSEHOLD SETTINGS

Extent and sources of inconsistency in self-reported cigarette smoking between self-administered school surveys and household interviews was examined in two longitudinal multiethnic adolescent samples, the urban Transition to Nicotine Dependence in Adolescence (TND) (N = 832) and the National Longitudinal Study of Adolescent Health (Add Health) (N = 4,414). Inconsistency was defined as a positive report of smoking in school followed by a negative report in the household. Smoking questions were ascertained with paper-and-pencil instruments (PAPI-SAQ) in school in both studies, and computer-assisted personal interviewing (CAPI) in TND but audio computer-assisted self-interviewing (ACASI) in Add Health in the household. In TND, 23.5 percent of youths who reported smoking lifetime and 20.4 percent of those who reported smoking the last 12 months in the school survey reported in the household never having smoked; in Add Health, the latter was 8.6 percent. Logistic regressions identified five common correlates of inconsistency across the two studies: younger age, ethnic minority status, lesser involvement in deviant activities, having nonsmoking parents and friends. In TND, interviewing of youth and parent by the same interviewer increased inconsistent reporting. Matching the definition of inconsistent reporting and the age, gender and race/ethnic distributions of TND on an urban Add Health subsample reduced the predicted rate of inconsistency in TND. The estimated bias attributable to CAPI compared with ACASI methodology did not reach significance in the aggregated matched samples suggesting that irrespective of administration mode, household interviews decrease reporting of smoking, especially among younger, minority and more conventional youths embedded in a social network of nonsmokers.     

Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults

The proportion of adults with only wireless telephones is growingrapidly. Using 2006 data from the National Center for HealthStatistics’ National Health Interview Survey, this articleis among the first to reveal that noncoverage of this populationcan result in nonnegligible bias for traditional random-digit-diallandline telephone surveys that do not call wireless telephonenumbers. In 2006 in the United States, 17 percent of low-incomeadults with household income below 200 percent of the federalpoverty thresholds, 25 percent of young adults aged 18–29years, and 32 percent of low-income young adults lived in householdswith only wireless telephones. Within each of these three subgroups,we compared wireless-only adults and adults with landline telephoneson demographic characteristics and 13 key indicators of healthstatus, health behaviors, health care service use, and healthcare access. Even after statistical adjustments that accountfor demographic differences between adults living in householdswith and without landlines, telephone surveys of landlines willunderestimate the prevalence of health behaviors, such as bingedrinking, smoking, and HIV testing. Obesity may be overestimatedand physical activity may be underestimated for low-income youngadults. No significant bias is predicted for other measuresof health status and health insurance coverage. Sample weightingprocedures that incorporate adjustments for multiple demographiccharacteristics are necessary to help attenuate coverage biasin traditional telephone surveys, but may not be sufficientfor behavioral risk factor surveys of low-income and young adults.     

The Influence of Advance Letters on Response in Telephone Surveys: A Meta-Analysis

Recently, the leading position of telephone surveys as the majormode of data collection has been challenged. Telephone surveyssuffer from a growing nonresponse, partly due to the generalnonresponse trend for all surveys and partly due to changesin society and technology influencing contactability and willingnessto answer. One way to counteract the increasing nonresponseis the use of an advance letter. In mail and face-to-face surveys,advance letters have been proven effective. Based on the proveneffectiveness in face-to-face and mail surveys, survey handbooksadvise the use of advance letters in telephone surveys. Thisstudy reviews the evidence for this advice and presents a quantitativesummary of empirical studies on the effectiveness of advanceletters in raising the response rate for telephone surveys.The major conclusion is that advance letters are also an effectivetool in telephone surveys, with an average increase in responserate (RR1) from 58 percent (no letter) to 66 percent (advanceletter), and an average increase in cooperation rate (COOP1)from 64 percent (no letter) to 75 percent (advance letter).     

Methodological Comparisons Between CATI Event History Calendar and Standardized Conventional Questionnaire Instruments

Six hundred and twenty-six participants from the Panel Studyof Income Dynamics (PSID) were interviewed via telephone retrospectivelyabout life course events by either computer-assisted (CATI)event history calendar (EHC) or standardized CATI conventionalquestionnaire (CQ) methods, randomly assigned. Experimentalretrospective reports, for a reference period up to 30 years,were validated against reports provided annually from the PSIDcore interviews. Data quality outcome measures included variablesassociated with marriage, cohabitation, employment, unemployment,residential changes, and cigarette smoking. The EHC providedhigher quality retrospective reports for cohabitation, employment,unemployment, and smoking histories; the CQ provided betterdata quality for marriage history, although what variable wasbeing measured, instead of which method was being used, hadthe biggest impact on differences in data quality. Both EHCand CQ interviews lasted on average around one hour, with theEHC interviews being on average 10 percent longer. Interviewerspreferred the EHC interviews. In both EHC and CQ conditions,respondents generally enjoyed the interviews, and did not findquestions difficult. The costs and benefits of both EHC andCQ methods in the collection of life course retrospective reportsare discussed. Received for publication December 19, 2005. Revision received September 25, 2007. Accepted for publication September 25, 2007.     

Telephone survey respondents' reactions to questions regarding interpersonal violence

Concerns have been raised regarding the appropriateness of asking about violence victimization in telephone interviews and whether asking such questions increases respondents' distress or risk for harm. However, no large-scale studies have evaluated the impact of asking such questions during a telephone interview. This study explored respondents' reactions to questions regarding violence in two large recently completed telephone surveys. After respondents were asked about violence, they were asked if they thought surveys should ask such questions and whether they felt upset or afraid because of the questions. In both surveys, the majority of respondents (regardless of their victimization history) were willing to answer questions about violence and were not upset or afraid because of the questions. More than 92% of respondents thought such questions should be asked. These results challenge commonly held beliefs and assumptions and provide some assurance to those concerned with the ethical collection of data on violent victimization.     

Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

EVALUATION OF A COMPUTER-ASSISTED SELF-INTERVIEW COMPONENT IN A COMPUTER-ASSISTED PERSONAL INTERVIEW SURVEY

This article examines respondent reactions to and performanceon a self-administered component of a computer-assisted personalinterview survey on self-images, conducted in the Detroit areain 1992. Although respondents were encouraged to complete thecomputer-assisted self-interview items them selves, only 79percent of those interviewed actually did so, while for 14 percentthe interviewer completed the items and for the remaining 7percent the interviewer read the questions while the respondententered the responses. We find that the decision to self-completethe computer-assisted self-interview items is re lated to factorssuch as respondents' age, education and com puter experience.This suggests that both capacity (literacy and vision) and motivationaffect the choice of completion. We also find differences insubstantive responses and data quality be tween self-completersand those who use interviewer assistance. These findings haveimplications for nonresponse error and data quality for self-administeredcomponents of interviews, particu larly those using computer-administration.     

Telephone sampling bias in surveying disability

Results of the California Disability Survey indicate that telephone interviewing is well suited for undertaking disability studies that provide (1) estimates of subgroups of the disabled population, including those that are statistically rare: (2) information on current and anticipated areas of policy concern; and (3) information for geographic areas important in rehabilitation program planning. Although these objectives necessitated a large sample size and a complex instrument, the costs and timeliness of telephone interviewing enabled the survey objectives to be reached. This paper discusses the advantages of telephone interviewing, as well as biases inherent in its use. The magnitude of the bias from the omission of nontelephone households is assessed, and the results confirm that the omission of nontelephone households introduces only minor biases into estimates for the total working-age population. A method of weighting is developed and illustrated.     

Impact of T-ACASI on Survey Measurements of Subjective Phenomena

Numerous studies have shown that audio-computer-assisted self-interviewing (audio-CASI) and telephone audio-CASI (T-ACASI) technologies yield increased reporting of sensitive and stigmatized objective phenomena such as sexual and drug use behaviors. Little attention has been given, however, to the impact of these technologies on the measurement of subjective phenomena (attitudes, opinions, feelings, etc.). This article reports results for the seven subjective measurements included in the National STD and Behavior Measurement Experiment (NSBME). NSBME drew probability samples of USA and Baltimore adults (Ns = 1,543 and 744, respectively) and randomized these respondents to be interviewed by T-ACASI or telephone interviewer-administered questioning (T-IAQ). Response distributions for all subjective measurements obtained by T-ACASI diverge from those obtained by human telephone interviewers. For six of our seven ordinal-scaled measurements, this divergence involved shifting responses directionally along the ordinal scale, as opposed to a nondirectional redistribution among response categories. When interviewed by T-ACASI, respondents were more supportive of traditional gender roles and corporal punishment, less supportive of integrated neighborhoods and same-gender sex, and more likely to agree that occasional marijuana use is harmless and to describe themselves as attractive. The majority of these results suggest that telephone survey respondents may provide more "tolerant" and "socially liberal" responses to human interviewers than to a T-ACASI computer. Similarly, although the evidence is not entirely consistent, the impact of T-ACASI appears to increase with the social vulnerability of the population surveyed.     

Effects of Category Order on Answers in Mail and Telephone Surveys1

Abstract Past research suggests that mail surveys encourage a primacy effect, which is a tendency to choose the first answers from a list, whereas telephone surveys encourage a recency effect, a tendency to choose the last answers from a list. This paper summarizes results from 82 new experiments conducted in 12 separate surveys in seven states. Only four of 33 mail survey comparisons exhibited significant primacy effects, while five of 26 experiments in telephone surveys exhibited recency effects. In addition, only three of 23 cross-method comparisons produced a significant primacy/recency effect in the expected manner. The conclusion is that the prevalence of primacy and recency effects has been over-estimated by past research and a new theoretical approach that takes into account multiple causation is needed for examining these effects.     

Reaching the U.S. Cell Phone Generation: Comparison of Cell Phone Survey Results with an Ongoing Landline Telephone Survey

Noncoverage rates in U.S. landline-based telephone samples dueto cell phone only households (i.e., households with no landlinebut accessible by cell phone) and the corresponding potentialfor bias in estimates from surveys that sample only from landlineframes are growing issues. Building on some of the few publishedstudies that focus on this problem, a study was conducted inthree states (Georgia, New Mexico, and Pennsylvania) as partof the Behavioral Risk Factor Surveillance System (BRFSS), theworld's largest ongoing public health telephone survey, to evaluatethe effectiveness of conducting the BRFSS interview with a sampledrawn from dedicated cell phone telephone exchanges and mixed-use(landline and cell phone) exchanges. Approximately 600 interviewswere conducted in each of two groups: cell phone only adults(n = 572) and adults with both a landline and a cell phone (n= 592). Making comparisons with data from the ongoing, landline-basedBRFSS survey, we report on response rates, demographic characteristicsof respondents, key survey estimates of health conditions andrisk behaviors, and survey costs. The methods used in this studyhave wide application for other U.S. telephone surveys.     

Interviewing Older Adults: A Comparison of Telephone and Face-to-Face Modalities

This article examines telephone interviewing of older adultsand compares it with face-to-face interviews. Specifically,the following issues are examined in several national surveys:(1) differences in age distributions between the samples ofadults reached in both modes; (2) explanations for potentialdifferences in age distributions; (3) differences between thetwo modes in demographic characteristics in the adults reached,in interview process and in response quality, and how thesemode differences vary by age of the respondents. Telephone surveystend to underrepresent older persons, and older persons whodo participate in a telephone survey are disproportion atelywell educated. Implications of the lower response rate amongolder persons are softened by the fact that reponse distributionsacross a range of questions show little difference by interviewmode between older persons and persons of other age groups.     

Presentation matters: how mode effects in item non-response depend on the presentation of response options

Comparability of data across modes is an important issue in survey research. In this paper we discuss item non-response to attitudinal questions in telephone and web surveys. We present results from a survey experiment conducted in Italy and in Spain that compares different presentations of response options in an online setting with a benchmark telephone survey. In line with earlier studies we find that (A), the share of non substantial answers in the online survey depends on how the response option is presented. Comparing different presentations in an online survey to the standard approach of telephone surveys using propensity score matching, we find that (B), the share and pattern of non substantial answers, is most similar across the two modes in the online survey when the it is captured in an instruction on each screen for the online survey. Our findings are of particular relevance for the design of multiple mode or mix-mode surveys of attitudinal questions using online and telephone modes.     

Using Community-Level Correlates to Evaluate Nonresponse Effects in a Telephone Survey

Understanding the relationship between nonresponse processesand key research variables is central to evaluating if and hownonresponse introduces bias into survey estimates. In most telephonesurveys, however, little information is available with whichto estimate these effects. We report a procedure for examiningthe potential effects of nonresponse via analyses that (1) investigatethe linkages between community-level (zip code) variables andsurvey nonresponse and (2) examine the associations betweenthese community-level variables and key survey measures. Wedemonstrate these procedures using hierarchical modeling toanalyze data from a state-wide telephone survey in Illinois.One zip code–level indicator of concentrated disadvantage—thepercentage of the population below poverty level—was foundto be positively associated with nonresponse and, among respondents,with both current physical disability status and lack of healthinsurance coverage, suggesting that both may have been underestimatedin this survey. This inexpensive approach has the potentialof enabling researchers to routinely evaluate nonresponse effectsin their survey data.     

Actors and Questions in Telephone and Personal Interview Surveys

This paperconcentratesontwoaspectsof telephone survey administration:(1) the respondent's reactions to the request for an interviewand to the interview itself, and (2) the properties of questionsusing response cards in personalinterviews and adaptations ofthese questions for telephone use.     

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Recent experimental research has shown that respondents to forced-choicequestions endorse significantly more options than respondentsto check-all questions. This research has challenged the commonassumption that these two question formats can be used interchangeablybut has been limited to comparisons within a single survey mode.In this paper we use data from a 2004 random sample survey ofuniversity students to compare the forced-choice and check-allquestion formats across web self-administered and telephoneinterviewer-administered surveys as they are commonly used insurvey practice. We find that the within-mode question formateffects revealed by previous research and reaffirmed in thecurrent study appear to persist across modes as well; the telephoneforced-choice format produces higher endorsement than the webcheck-all format. These results provide further support forthe argument that the check-all and forced-choice question formatsdo not produce comparable results and are not interchangeableformats. Additional comparisons show that the forced-choiceformat performs similarly across telephone and web modes.