ESTIMATING TELEPHONE NONCOVERAGE BIAS WITH A TELEPHONE SURVEY

Nontelephone households are implicitly treated as a static populationin discussions of sampling frame noncoverage. Yet telephoneservice is known to be episodic for many households, who maygain or lose service as their financial situation changes orwhen they move. Thus the population of telephone householdsat any given time includes households that were recently a partof the nontelephone population. These households may be usedto characterize the nature of some noncoverage errors and evento estimate their magnitude. Using a panel constructed withthe 1992-93 Current Population Survey, "transient" telephonehouseholds-those who gained or lost service over the year coveredby the panel-are shown to comprise over half of the panel householdsreporting no telephone service in either the 1992 or 1993 surveys.These households are compared with the total nonphone populationand found to be similar on a variety of key demographic characteristics.Several statewide Virginia telephone surveys are used to comparehouseholds reporting "intermittent" phone service with nontelephonehouseholds surveyed through in-person interviews. Householdsreporting intermittent telephone service were very similar tonontelephone households in terms of health insurance coverageand other variables known to be related to telephone status.     

Household Telephone Service and Usage Patterns in the United states in 2004: Implications for Telephone Samples

Changes in the U.S. telephone system, especially the rapid growthin the prevalence and use of cell phones, raise concerns aboutundercoverage error in random digit dial (RDD) telephone samples.A supplement to the Current Population Survey (CPS) was conductedin 2004 to examine telephone service and usage in U.S. households.This article explores the potential for biases in RDD surveysresulting from the increases in cell phones by presenting estimatesof the percentage of households with different types of telephoneservice, including the percentage of cell-only households, andgiving demographic profiles of households by type of telephoneservice. Logistic regression models examine variables that predictwhether households are without a telephone or only have cellphones. These predictors may be used for weighting adjustmentsto reduce undercoverage biases. We address some additional issues,including the wording of questions for measuring telephone service,that are relevant if telephone-sampling methods are revisedto include cell phones. The estimates from the CPS supplementare also used to help understand some of the new sampling andweighting problems associated with selecting samples from cellphone numbers.     

A Study of Experimental Techniques for Telephone Interviewing

This investigation focused on the effects of experimental interviewingprocedures, which were developed in face-to-face interviews,on reporting in telephone contacts. Respondents in a nationalRDD sample (N=1,1054) were randomly assigned to be interviewedwith one of two experimental procedures, or a control technique.The experimental procedures involved two different combinationsof three interviewing techniques which had been found to beeffective in personal interviewing studies—commitment,instructions, and feedback. The techniques are designed to betterinform respondents about reporting tasks and to motivate themto perform the tasks well. The findings suggest that the experimentalprocedures can improve reporting in telephone interview, althoughthe effects in this study were not as strong as we expected.This research may be seen as a beginning step in the developmentof optimal telephone interviewing procedures.     

Reaching the U.S. Cell Phone Generation: Comparison of Cell Phone Survey Results with an Ongoing Landline Telephone Survey

Noncoverage rates in U.S. landline-based telephone samples dueto cell phone only households (i.e., households with no landlinebut accessible by cell phone) and the corresponding potentialfor bias in estimates from surveys that sample only from landlineframes are growing issues. Building on some of the few publishedstudies that focus on this problem, a study was conducted inthree states (Georgia, New Mexico, and Pennsylvania) as partof the Behavioral Risk Factor Surveillance System (BRFSS), theworld's largest ongoing public health telephone survey, to evaluatethe effectiveness of conducting the BRFSS interview with a sampledrawn from dedicated cell phone telephone exchanges and mixed-use(landline and cell phone) exchanges. Approximately 600 interviewswere conducted in each of two groups: cell phone only adults(n = 572) and adults with both a landline and a cell phone (n= 592). Making comparisons with data from the ongoing, landline-basedBRFSS survey, we report on response rates, demographic characteristicsof respondents, key survey estimates of health conditions andrisk behaviors, and survey costs. The methods used in this studyhave wide application for other U.S. telephone surveys.     

BIAS IN LIST-ASSISTED TELEPHONE SAMPLES

A number of researchers have suggested list-assisted samplingfor the selection of telephone households to overcome some ofthe operational difficulties associated with the Mitofsky-Waksbergmethods of random digit dialing (RDD). An advantage of a list-assistedmethod of RDD is that an equal probability systematic sampleof telephone numbers can be selected and the variances of estimatesfrom such a sample are usually lower than from a clustered designlike the Mitofsky-Waksberg method. The main disadvantage ofthe list-assisted method is that it excludes some householdsfrom the sample, thus creating a coverage bias in the estimates.This article describes research on the coverage bias for a particularmethod of list-assisted sampling. The two key determinants ofcoverage bias are the proportion of households that are noteligible for the sample and the differences in the characteristicsof the covered and not covered populations. The results showthat about 4 percent of all households are excluded in nationalsamples using this method of sampling. Furthermore, they showthat the differences between the covered and uncovered populationsare generally not large. The coverage bias resulting from theseconditions may often be small.     

THE ANSWERING MACHINE GENERATION: WHO ARE THEY AND WHAT PROBLEM DO THEY POSE FOR SURVEY RESEARCH?

Increased incidence of telephone answering machines and theuse of such devices to screen calls pose a potential threatto the representativeness of samples in telephone surveys. Usingdata from nine statewide surveys, this analysis examines theextent to which answering machines are used to screen callsand the demographic characteristics associated with answeringmachine use and call screening. Results show that at most twoto three percent of households use answering machines to screencalls, and that such screening is more likely to take placein households with higher family incomes, outside rural areas,and which include individuals who are younger and have higherlevels of education. While call screening does not presentlythreaten the representativeness of samples in telephone surveys,the increased incidence of answering machines together withthe increased percentage of households indicating these devicesare sometimes used to screen calls demonstrate that the potentialbias from this source is growing.     

Marital Separation

In an effort to evaluate a procedure designed to learn more about clinical and demographic aspects of marital separation, a telephone survey of 2,001 households in Boulder, Colorado, was conducted. The procedure reached a representative sample of the population and provided valuable information regarding the current prevalence of marital separation, as well as its incidence, duration, and modes of resolution. The likelihood of any married couple experiencing a separation was found to be nearly 5 percent in one year. The risk of separation appears to be far higher among the young than the old. Most terminated separations, after lasting about one year, end in divorce. Implications for marriage and divorce counseling, for community intervention programs, and for additional research are indicated.     

USING DUAL FRAME DESIGNS TO REDUCE NONRESPONSE IN TELEPHONE SURVEYS

This article reports on the results of a series of experimentsdesigned to improve response rates for telephone surveys. Inthree surveys telephone households were selected using bothstandard random digit dialing (RDD) techniques and lists oftelephone numbers purchased from a commercial firm. In the RDDportions of the samples "cold contact" interviewing methodswere used; in the list frame portions advance letters were mailed,and the listed household name was used in the introduction.Experiments were designed to test the effects on response ratesof the advance letters and use of the listed household nameas a means of establishing rapport. The advance letters increasedresponse rates, but no difference could be attributed to theuse of names. The mixture of RDD and list sampling techniquesis also used to evaluate the effects of relative response rateson substantive findings. The cost consequences of these dualframe designs are assessed along a number of dimensions, andthe cost and error components of these designs are discussed.     

Bias in a Directory Sample for a Mail Survey of Rural Households

This study examines bias in a mail survey of rural householdsthat used telephone directories as the sampling frame and comparesit with a parallel survey that used an independent sample selectedfrom field listings. A higher response rate was obtained withthe directory sample, which was primarily attributed to pooreraddress information for the listed sample. Although coverageof the study population was higher for the listed frame thanfor the directory frame, no significant differences were foundfor eight respondent and seven household characteristics. Also,no significant differences were found for seven key contentvariables regarding health behaviors and attitudes. The findingssupport the use of telephone directories as the sampling framefor general population mail surveys of rural areas where suchcoverage of the population is considered adequate.     

The Influence of Information Costs on Donor Decision Making

This article explores whether an individual's information costs influence the information‐gathering strategies that he or she turns to prior to donating to a nonprofit. The data for the study come from a telephone survey of residents in a large county in southern California (n = 1,002). The sample was selected using random‐digit‐dialing technology and a computer‐assisted telephone interviewing system. A series of binomial logit models revealed that information costs significantly influenced the information‐gathering strategies individuals turned to in order to learn about nonprofit performance prior to making a donation. Results also revealed that greater confidence in nonprofits did not lead individuals to forgo efforts to obtain information on nonprofit performance altogether. The findings from this study suggest that it is important for nonprofit administrators to recognize, and then take into account, that not all individuals will rely on the same information sources when seeking to learn about nonprofit performance.     

EFFECTS OF INTERVIEW MODE ON SELF-REPORTED DRUG USE

The effects of interview mode on 18- to 34-year-olds' self-reportedtobacco, alcohol, marijuna, and cocaine use were investigated.RDD telephone and personal drug use surveys of the state ofNew Jersey were conducted in 1986–87. In the personalinterview, respondents recorded their drug use on self-administratedanswer sheets. Compared to the area probability sample in thepersonal interview, RDD yielded a sample of blacks higher inincome and education, and more likely to be married and currentlyemployed; white SES was also higher in the telephone sample,but to a lesser degree than for blacks. Controlling for demographiccharacteristics and RDD's exclusion of non-telephone households,the telephone survey yielded significantly lower estimates ofblacks' alcohol consumption, and lifetime and recent marijuanause. Whites' alcohol consumption was slightly lower by telephone;otherwise, estimates of whites' use of the four substances werenearly identical in the two modes. Sample coverage, respondentdemographic characteristics, and racial matching of interviewerand respondent did not account for the significant mode differences.Characteristics of the interview situation itself, such as provisionof privacy in the self-administered format, may have influencedtendencies toward socially desirable responding to a threateningtopic such as drug use.     

Improving Gambling Survey Research Using Dual-Frame Sampling of Landline and Mobile Phone Numbers

Gambling prevalence studies are typically conducted within a single (landline) telephone sampling frame. This practice continues, despite emerging evidence that significant differences exist between landline and mobile (cell) phone only households. This study utilised a dual-frame (landline and mobile) telephone sampling methodology to cast light on the extent of differences across groups of respondents in respect to demographic, health, and gambling characteristics. A total of 2,014 participants from across Australian states and territories ranging in age from 18 to 96 years participated. Interviews were conducted using computer assisted telephone interviewing technology where 1,012 respondents from the landline sampling frame and 1,002 from the mobile phone sampling frame completed a questionnaire about gambling and other health behaviours. Responses across the landline sampling frame, the mobile phone sampling frame, and the subset of the mobile phone sampling frame that possessed a mobile phone only (MPO) were contrasted. The findings revealed that although respondents in the landline sample (62.7 %) did not significantly differ from respondents in the mobile phone sample (59.2 %) in gambling participation in the previous 12 months, they were significantly more likely to have gambled in the previous 12 months than the MPO sample (56.4 %). There were no significant differences in internet gambling participation over the previous 12 months in the landline sample (4.7 %), mobile phone sample (4.7 %) and the MPO sample (5.0 %). However, endorsement of lifetime problem gambling on the NODS-CLiP was significantly higher within the mobile sample (10.7 %) and the MPO sample (14.8 %) than the landline sample (6.6 %). Our research supports previous findings that reliance on a traditional landline telephone sampling approach effectively excludes distinct subgroups of the population from being represented in research findings. Consequently, we suggest that research best practice necessitates the use of a dual-frame sampling methodology. Despite inherent logistical and cost issues, this approach needs to become the norm in gambling survey research.     

SAME-GENDER SEX IN THE UNITED STATES IMPACT OF T-ACASI ON PREVALENCE ESTIMATES

Well-conducted telephone surveys provide an economical meansof estimating the prevalence of sexual and reproductive behaviorsin a population. There is, however, a nontrivial potential forbias since respondents must report sensitive information toa human interviewer. The National STD and Behavior MeasurementExperiment (NSBME) evaluates a new survey technology—telephoneaudio computer-assisted self-interviewing (T-ACASI)—thateliminates this requirement. The NSBME embedded a randomizedexperiment in a survey of probability samples of 1,543 U.S.and 744 Baltimore adults ages 18 to 45. Compared with NSBMErespondents interviewed by human interviewers, respondents interviewedby T-ACASI were 1.5 to 1.6 times more likely to report same-gendersexual attraction, experience, and genital contact. The impactof T-ACASI was more pronounced (odds ratio = 2.5) for residentsof locales that have historically been less tolerant of same-gendersexual behaviors and for respondents in households with children(odds ratio = 3.0).     

Effects of screening questionnaires on response in a two-phase postal survey

For surveys targeting specific population groups, the two-phase postal approach (screener followed by a topical survey sent to eligible households) has been demonstrated to be more effective at identifying population domains of interest than random digit dial telephone methods considering cost, coverage, and response. An important question is how best to motivate screener response from eligible households. In 2011, we conducted a large-scale field test to empirically test a number of methods for motivating response. We fielded screening surveys that varied content-influencing relevance, and also switched screener questionnaires for following up nonrespondents to the initial postal survey – an approach we have labeled responsive tailoring. In another experiment, we tested the effect of asking for first names in the screener questionnaire. In this article, we describe the effects of these experimental treatments on response to both the screener and the topical survey.     

The changing experience of researching family and intimate relationships

A small-scale study to explore the possibility of obtaining information about small companies' charitable giving through telephone interviews provided challenges on three research design issues: selecting a target sample, telephone interviewing, and sensitive research topics. This article explores the general lessons derived from the study. First, it considers suitable sampling frames for identifying and contacing small companies. Second, it discusses the consequences for response rates of the telephone warfare between telemarketers and small companies. Third, it reviews what contributes to the sensitivity of research topics and ways of overcoming this.     

A Capture–Recapture Approach to Estimation of Refugee Populations

The debate over immigration continues to be one of the most politically charged policy issues in the United States (US). Given the charged nature of this topic, it is vitally important to have reliable data on not only the number of US foreign nationals but also the characteristics of this extremely heterogeneous group – a population comprised not only of immigrants but also of refugees. There exist a small number of data sources for informing policy and practice at the national level. However, such data are often lacking for smaller geographical areas. This paper describes a recent effort to generate serviceable data on the immigrant and refugee population for a medium‐sized metropolitan area in the US. The objectives of this research were twofold. Our first goal was to provide local stakeholders with information to assist them with resettling and obtaining funding for immigrants and refugees. The second aim was to develop better techniques for tabulating diverse refugees and immigrants in a medium‐sized community. By comparing and contrasting three data sources – that is, refugee services, public schools and a local health plan – we are able to generate estimates of the local refugee and immigrant population. During the period from 2005 to 2007, we estimate the total number of immigrants and refugees in the community to be somewhere between 10,938 and 13,282. Although perhaps a bit on the high end due to methodological assumptions, these estimates seem plausible, based on previously cited figures for the region. While such estimates are valuable, a number of shortcomings related to the data prevent us from painting a more complete picture of these populations. We conclude this paper with a number of recommendations that will assist others in planning research designed to inform migration policy and practice in medium‐sized metropolitan areas.     

The impact of touch-tone data entry on reports of HIV and STD risk behaviors in telephone interviews

Respondents concerns about privacy can decrease reporting of HIV and STD risk behaviors in general population telephone surveys. The purpose of this paper is to describe the results of an experimental study evaluating whether one method for increasing privacy, touch-tone data entry (TTDE), is effective in increasing estimates of sexual behaviors from a population-based survey. We conducted a random-digit-dial telephone survey of adults in New Jersey (n = 405), with half the respondents using TTDE for answering sexual behavior questions. TTDE led to increased reports of same-sex sexual behavior, certain HIV and STD risk factors, and concern about one s risk for HIV and STD transmission. TTDE also narrowed the difference between men s and women s reports of the number of different sexual partners over the past 10 years. The feasibility and limitations of TTDE are discussed, as well as possible alternative interpretations that consider the impact of TTDE on the dynamics of the interaction between the respondent and the interviewer.     

Effects of an introductory letter on response rates to a teen/parent telephone health survey

The authors conducted a pilot study in preparation for a larger investigation that will rely on telephone surveys to assess select health behaviors of teens and their parents, with a focus on indoor tanning. This study used a randomized design to assess the impact of a presurvey letter on response rates to a telephone survey, as well as prevalence estimates of ever having used an indoor tanning facility. The letter did not have a significant effect on response rates or prevalence estimates in the two cities under study. Findings suggest that researchers should not assume that a letter sent in advance of a telephone survey will necessarily increase response rates enough to justify their use.     

Neighborhood Selection and Residential Segregation

Residential segregation has long been a topic of investigation for urban scholars. In attempting to understand this phenomenon, most researchers have examined the characteristics of different residential neighborhoods (typically census tracts) and have often compared the composition of the population within the census tract over time. This study looks at the household characteristics of the mobile family, as well as the aggregate characteristics of the census tract into which the family moves. The findings reveal that households display a tendency to move to areas where the existing population has characteristics similar to their own. A multiple logit analysis reveals that the race/ethnicity of the household is generally the best predictor of the neighborhood where the household will move.     

Assessing the Effect of Model Misspecifications on Parameter Estimates in Structural Equation Models

Model misspecifications may have a systematic effect on parameters, causing biases in their estimates. In the application of structural equation models, every interesting model is fallible. When simultaneously evaluating a model, it is of interest to study whether all parameters are affected by a misspecification. This paper provides three procedures for evaluating such an effect: (1) analyzing the path, (2) using a functional relationship, and (3) using a significance test. Analyzing the path is illustrated through a confirmatory factor model. This method is ad hoc but intuitive. A more rigorous approach is built upon the concept of orthogonality of two sets of parameters. When parameter a is orthogonal to parameter b, omitting parameter b will not affect the estimation of parameter a. The functional relationship of two sets of parameters is used to check their orthogonality. The distribution of the difference between estimates based on different models is obtained, which provides a Hausman–like way to check significant parameter differences that are due to biases. Examples illustrate that these procedures can provide valuable information on identifying parameter estimates that are systematically affected by a model misspecification.