Improving Gambling Survey Research Using Dual-Frame Sampling of Landline and Mobile Phone Numbers

Gambling prevalence studies are typically conducted within a single (landline) telephone sampling frame. This practice continues, despite emerging evidence that significant differences exist between landline and mobile (cell) phone only households. This study utilised a dual-frame (landline and mobile) telephone sampling methodology to cast light on the extent of differences across groups of respondents in respect to demographic, health, and gambling characteristics. A total of 2,014 participants from across Australian states and territories ranging in age from 18 to 96 years participated. Interviews were conducted using computer assisted telephone interviewing technology where 1,012 respondents from the landline sampling frame and 1,002 from the mobile phone sampling frame completed a questionnaire about gambling and other health behaviours. Responses across the landline sampling frame, the mobile phone sampling frame, and the subset of the mobile phone sampling frame that possessed a mobile phone only (MPO) were contrasted. The findings revealed that although respondents in the landline sample (62.7 %) did not significantly differ from respondents in the mobile phone sample (59.2 %) in gambling participation in the previous 12 months, they were significantly more likely to have gambled in the previous 12 months than the MPO sample (56.4 %). There were no significant differences in internet gambling participation over the previous 12 months in the landline sample (4.7 %), mobile phone sample (4.7 %) and the MPO sample (5.0 %). However, endorsement of lifetime problem gambling on the NODS-CLiP was significantly higher within the mobile sample (10.7 %) and the MPO sample (14.8 %) than the landline sample (6.6 %). Our research supports previous findings that reliance on a traditional landline telephone sampling approach effectively excludes distinct subgroups of the population from being represented in research findings. Consequently, we suggest that research best practice necessitates the use of a dual-frame sampling methodology. Despite inherent logistical and cost issues, this approach needs to become the norm in gambling survey research.     

Sampling Telephone Numbers and Adults, Interview Length, and Weighting in The California Health Interview Survey Cell Phone Pilot Study

This article describes several features included in a CaliforniaHealth Interview Survey cell phone pilot study that differ fromearlier cell phone surveys conducted in the United States. Onedifference is that the study used a screening design and onlyadults living in cell-only households were interviewed. Mostof the previous studies used dual frame designs. Another differencewas the development and implementation of a within-householdadult sampling procedure to cover adults when cell phones wereshared in the household. The study was also intended to determineif conducting a cell phone interview of the same scope and lengthas a regular telephone interview was feasible. Most previoussurveys were focused on cell phone topics and were shorter thanthe comparable landline surveys. We then explore some interestingproblems that arise in weighting a cell phone survey using ascreening design. We conclude with a discussion of planninga new cell phone survey in 2007 that incorporates findings fromthis study.     

Cell Phone Survey Feasibility in The U.S.: Sampling and Calling Cell Numbers Versus Landline Numbers

In 2004, we conducted a nationwide dual frame survey of landlineand cell phone numbers to evaluate the feasibility of includingcell phone numbers in a random digit dial telephone survey.Households with both landline and cell phones were eligiblefor selection in both samples. This article describes our designand data collection methods; compares the results from the twosamples (with an emphasis on operational characteristics); andpresents the outcomes of two experimental manipulations designedto improve the cell phone response rate.     

The State of Surveying Cell Phone Numbers in the United States: 2007 and Beyond

By the late 1970s, household telephone coverage grew to exceed90 percent in the United States, and by the mid-1980s telephonesurveying of the general public had become commonplace. Nevertheless,20 years later, the ability of researchers to reach representativesamples of the U.S. public via landline (wired) telephone surveysand gather reliable data is being seriously challenged for manyreasons, especially those related to cell phones and the growthof the "cell phone only" population. However, at present thereexists no widely accepted set of Cell Phone Surveying "bestpractices" for U.S. survey researchers to follow. Despite whatsome appear to believe, surveying persons reached on cell phonenumbers in the United States currently is a very complex undertakingif one wants to do it "right," i.e. to do it legally, ethically,and in ways that optimally allocate one's finite resources togather the highest quality data, and to analyze and interpretthose data accurately. This final "wrap-up" article in the specialissue provides a review of the empirical articles in the issuewith a focus on their practical implications for the decisionsthat researchers need to make regarding sampling, coverage,nonresponse, measurement, and weighting in surveys that includeinterviews with persons reached on cell phones. The articlealso highlights the practical implications of a number of legal,ethical, and other issues that relate to surveys in the UnitedStates that include cell phone numbers. Surveying the U.S. cellphone population is possible, if at a higher cost than surveyingtheir landline counterparts, and if with less precision thancurrently can be done surveying the landline population. Thenext five years should see a considerable growth in the methodologicaland statistical know-how that the survey community uses to plan,implement, and interpret cell phone surveys. There is a greatdeal that still must be learned.     

Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers

We conducted a dual frame survey of landline and cell phonenumbers in 2004 to evaluate the feasibility of including cellphone numbers in random digit dial telephone surveys in theUnited States. By sampling cell phone numbers, the coveragebias associated with households that have only cell phones iseliminated. However, we discovered two major sources of nonresponsebias in the dual frame sample. In an attempt to reduce thesebiases, we applied several different estimation schemes. Buta comparison to the 2004 Current Population Survey Cell PhoneSupplement showed that none of the estimation schemes substantiallyreduced the nonresponse bias of the estimates. We suggest othermethods that might be used in future surveys that include cellphones and discuss the need for additional data collection andresearch on this issue.     

A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys

Valid and reliable public health data are becoming more difficultto obtain through random-digit dial (RDD) telephone surveys.As a result, researchers are evaluating different survey designs(i.e., sampling frame and survey mode combinations) as complementsor alternatives to RDD. Traditionally, mail surveys of the generalpublic have been limited due to a lack of a complete samplingframe of households. Recent advances in electronic record keeping,however, have allowed researchers to develop a sample from aframe of addresses (e.g., the U.S. Postal Service Delivery SequenceFile, which appears to provide coverage which rivals or possiblyexceeds that obtained through RDD sampling methods). To testthe use of this frame for surveying adults aged 18 years andolder across a wide geographic area, a pilot study was conductedas part of the 2005 Behavioral Risk Factor Surveillance System(BRFSS). The pilot compared use of a traditional, RDD telephonesurvey methodology to an approach using a mail version of thequestionnaire completed by a random sample of households drawnfrom an address-based frame. The findings indicate that themail survey approach can achieve higher response rates in low-response-ratestates (<40%) than RDD (particularly when two mailings aresent). Additionally, the address frame with mail survey designprovides access to cell phone only households and offers costsavings over the telephone approach. The resulting sample, however,significantly overrepresents non-Hispanic whites and peoplewith higher levels of education.     

What's Missing from National Landline RDD Surveys?: The Impact of the Growing Cell-Only Population

The number of cell phone only households has continued to grow– 12.8 percent of all households by the end of 2006, accordingto the National Health Interview Survey (NHIS). To assess theextent and nature of the potential bias in landline telephonesamples created by the absence of cell-only households, thePew Research Center conducted four independent dual frame studiesin 2006, three of which included at least 200 interviews withcell-only respondents. Despite the fact that there are significantdifferences between cell-only and landline respondents on manyimportant variables, across the four surveys with differentsubstantive content we find that including a cell-only samplewith a landline RDD sample produces general population estimatesthat are nearly identical to those from the landline samplealone. Yet, while the noncoverage problem is currently not damagingestimates for the entire population, we find evidence that itdoes create biased estimates on certain variables for youngadults, 25 percent of whom are cell-only according to the mostrecent government estimate.     

Telephone surveys under different frame constructions—evaluating differences in estimates,precision, and cost

Dual-frame, random-digit-dialing (DFRDD) surveys, using a frame of landline telephone numbers and a frame of cell phone numbers, have become the norm for telephone surveys. Two alternative frame constructions sample exclusively from a cell phone frame, referred to as the ‘single-frame cell,’ and sample from the listed landline frame and the cell phone frame, referred to as ‘listed/cell.’ The listed/cell frame construction differs from the DFRDD in that it does not select unlisted-landline frame members. We observe in our simulated data that the listed/cell frame construction results in estimates that more closely approximate the estimates from DFRDD compared with the single-frame cell, results in an improvement in precision compared with the DFRDD that is not as large as the improvement in the single-frame cell, and has lower cost than both the DFRDD and the single-frame cell. We conclude that a listed/cell frame construction might be a better solution than either a DFRDD or single-frame cell frame construction.     

The Influence of Incremental Increases in Token Cash Incentives on Mail Survey Response: Is There an Optimal Amount?

The research reported is from a large-scale, national, mixed-modesurvey (N = 49,675 addresses), in which token incremental, noncontingentincentives used in the second (mail) stage of the survey weretested in an experimental design. The levels of the incentivestested ranged from $0 to $10, in one-dollar increments, withthe exception that there was no $9 condition. Furthermore, theexperiment was conducted using three types of household addressesthat originally were sampled in the first stage of the mixed-modesurvey via a random digit dial (RDD) frame: (1) households thatagreed to participate in the mail stage of the survey; (2) householdsthat were never contacted in the RDD stage despite numerouscontact attempts; and (3) households that had refused to participatewhen contacted in the RDD stage. It was determined that amongthose households that had previously agreed to participate,even those that received no cash incentive ($0) were more likelyto return completed surveys than the groups of previously noncontactedand previously refusing households that received $10. It wasalso found that the nature of the outcome of the prior contactwith the household mediated (that is, interacted with) the proportionaleffects of the incremental incentives, in ways only partiallyconsistent with Dillman’s social exchange theory. Thefindings, which are consistent with leverage-salience theory,suggest that if there is an optimal amount for an incentiveto use in a mail survey, then it is a variable amount best targetedto the individual household.     

Household Telephone Service and Usage Patterns in the United states in 2004: Implications for Telephone Samples

Changes in the U.S. telephone system, especially the rapid growthin the prevalence and use of cell phones, raise concerns aboutundercoverage error in random digit dial (RDD) telephone samples.A supplement to the Current Population Survey (CPS) was conductedin 2004 to examine telephone service and usage in U.S. households.This article explores the potential for biases in RDD surveysresulting from the increases in cell phones by presenting estimatesof the percentage of households with different types of telephoneservice, including the percentage of cell-only households, andgiving demographic profiles of households by type of telephoneservice. Logistic regression models examine variables that predictwhether households are without a telephone or only have cellphones. These predictors may be used for weighting adjustmentsto reduce undercoverage biases. We address some additional issues,including the wording of questions for measuring telephone service,that are relevant if telephone-sampling methods are revisedto include cell phones. The estimates from the CPS supplementare also used to help understand some of the new sampling andweighting problems associated with selecting samples from cellphone numbers.     

Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

From Agenda Setting to Refusal Setting: Survey Nonresponse as a Function of Media Coverage Across the 2004 Election Cycle

Past research suggests that there is a relationship betweensurvey response and topic salience, namely that individualsresponding to a survey are likely to find the survey topic moresalient than nonrespondents do. For election surveys, nonresponseresulting from a lack of salience can influence findings becauserespondents may be more interested in politics than nonrespondents.The agenda-setting model suggests that media coverage shouldheighten salience. Thus, as media coverage of political campaignsincreases over the course of an election, refusals to a politicalsurvey should decline. Using data from the National AnnenbergElection Survey (NAES), which was conducted nearly continuouslyin 2004, this study investigates the issue of nonresponse ina random digit dial telephone survey across the election cycleby examining daily changes in the refusal rates using time-seriesanalysis. Content analyses of the frequencies of presidentialcampaign stories mentioned in the New York Times and three networknews broadcasts were matched against a time series from theNAES to demonstrate that increases in media coverage of theelection were negatively related to the survey refusal rate. Received for publication January 15, 2005. Revision received March 18, 2007. Accepted for publication May 14, 2007.     

ESTIMATING TELEPHONE NONCOVERAGE BIAS WITH A TELEPHONE SURVEY

Nontelephone households are implicitly treated as a static populationin discussions of sampling frame noncoverage. Yet telephoneservice is known to be episodic for many households, who maygain or lose service as their financial situation changes orwhen they move. Thus the population of telephone householdsat any given time includes households that were recently a partof the nontelephone population. These households may be usedto characterize the nature of some noncoverage errors and evento estimate their magnitude. Using a panel constructed withthe 1992-93 Current Population Survey, "transient" telephonehouseholds-those who gained or lost service over the year coveredby the panel-are shown to comprise over half of the panel householdsreporting no telephone service in either the 1992 or 1993 surveys.These households are compared with the total nonphone populationand found to be similar on a variety of key demographic characteristics.Several statewide Virginia telephone surveys are used to comparehouseholds reporting "intermittent" phone service with nontelephonehouseholds surveyed through in-person interviews. Householdsreporting intermittent telephone service were very similar tonontelephone households in terms of health insurance coverageand other variables known to be related to telephone status.     

Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults

The proportion of adults with only wireless telephones is growingrapidly. Using 2006 data from the National Center for HealthStatistics’ National Health Interview Survey, this articleis among the first to reveal that noncoverage of this populationcan result in nonnegligible bias for traditional random-digit-diallandline telephone surveys that do not call wireless telephonenumbers. In 2006 in the United States, 17 percent of low-incomeadults with household income below 200 percent of the federalpoverty thresholds, 25 percent of young adults aged 18–29years, and 32 percent of low-income young adults lived in householdswith only wireless telephones. Within each of these three subgroups,we compared wireless-only adults and adults with landline telephoneson demographic characteristics and 13 key indicators of healthstatus, health behaviors, health care service use, and healthcare access. Even after statistical adjustments that accountfor demographic differences between adults living in householdswith and without landlines, telephone surveys of landlines willunderestimate the prevalence of health behaviors, such as bingedrinking, smoking, and HIV testing. Obesity may be overestimatedand physical activity may be underestimated for low-income youngadults. No significant bias is predicted for other measuresof health status and health insurance coverage. Sample weightingprocedures that incorporate adjustments for multiple demographiccharacteristics are necessary to help attenuate coverage biasin traditional telephone surveys, but may not be sufficientfor behavioral risk factor surveys of low-income and young adults.     

Counting the uninsured: a comparison of national surveys

This Issue Brief reviews surveys that provide estimates of the uninsured population in the United States. It includes a discussion of why the estimates from the various surveys differ. It is important to understand the differences in the estimates of the uninsured population. The projected cost of implementing policy proposals depends on the estimates of the number of people affected by the proposals; for instance, the allocation of funding for the State Children's Health Insurance Program (S-CHIP) depends heavily on the available estimates. In addition, the estimated effectiveness of policy proposals to reduce the uninsured population will be accurate only if the correct count is known and the precise make-up of the uninsured population is understood. Currently, seven surveys can be used to make nationally representative estimates of the number of people without health insurance coverage. Some of the surveys collect health insurance information in the context of obtaining general information on health care, while other surveys are focused on other topics such as labor force participation and public assistance program participation. The most widely used survey that collects information on health insurance coverage is the Current Population Survey (CPS), conducted by the Census Bureau. The most recent estimates from the CPS suggests that 44.3 million Americans were uninsured in 1998. Besides the CPS, a number of other surveys collect information on the uninsured population. They include the Survey of Income and Program Participation (SIPP), Behavioral Risk Factor Surveillance System (BRFSS), Community Tracking Study (CTS), Medical Expenditure Panel Survey (MEPS), National Health Interview Survey (NHIS), and the National Survey of America's Families (NSAF). Estimates of the uninsured from these surveys range from 19 million to 44 million and vary depending on the time frame the survey covers. A number of states have started to question the validity of the uninsured estimates from the CPS, and other surveys, because of the small sample size in many states. As a result, some states have begun to conduct their own surveys to determine the number of uninsured residents. States that regularly conduct their own surveys include Florida, Massachusetts, Minnesota, New Mexico, Oregon, Vermont, and Wisconsin. Unfortunately, the various state surveys are not easily comparable. Research needs to continue to increase understanding of the differences among the surveys and to improve on methodologies to count the uninsured, as the future of public programs, such as S-CHIP and other state and local initiatives to expand health insurance coverage, depends on the accuracy of these estimates. Whatever survey is used, the results show that a substantial number of Americans do not have any health insurance coverage, and the number has been growing.     

Reaching a representative sample of college students: A comparative analysis

ABSTRACT

Objective: To explore the feasibility of a random-digit dial (RDD) cellular phone survey in order to reach a national and representative sample of college students. Methods: Demographic distributions from the 2011 National Young Adult Health Survey (NYAHS) were benchmarked against enrollment numbers from the Integrated Postsecondary Education System (IPEDS). The sample quality was compared with the Harvard School of Public Health College Alcohol Study (1993), National College Health Risk Behavior Survey (1995), and National College Health Assessment (2011). Results: Overall, the NYAHS performed as well, if not better, than previous college health surveys at reaching important demographic subgroups. Conclusions: Cellular phone RDD is one feasible approach for studies of the general population of college students that can potentially generate national estimates of health behavior.     

The Growth of Internet Research Methods and the Reluctant Sociologist*

Between 1999 and 2004 only one article appeared in the American Sociological Review, the American Journal of Sociology, or Social Forces using primary data collected with Web‐based research techniques. Since then there have been only a handful of studies published in these core sociology journals drawing on Web‐based surveys or other forms of Web‐based data. The use of Internet‐based data has become widespread in many academic fields, especially health research and education; Web‐based techniques are becoming routine in the practice and study of politics; and online commercial and market research has become a billion dollar industry. At the same time, the utility of random digit dialing surveys has eroded considerably owing to declining contact rates, increased use of technologies to screen unwanted telephone calls, and the replacement of landline telephones with cell phones. There is increasing evidence that Internet research can produce representative data. Although Web‐based surveys may overrepresent some populations, Internet usage in the general population is now well over 75 per cent and is especially strong among some hard to reach populations. Internet surveys have the potential to reduce measurement error, missing data, and respondent attrition. Sociologists must overcome their fear of participation in stigmatized Internet research and actively engage in the development of techniques and refinements that will increase the utility and validity of Internet‐based data collection.     

Mitofsky Waksberg: Learning From The Past

Warren Mitofsky and Joseph Waksberg were two of the most influentialresearchers in telephone surveys and survey research methods.Their development of the Mitofsky–Waksberg method forrandom digit dialing sampling revolutionized telephone samplingand surveys. Jointly and separately, Waksberg and Mitofsky wereinstrumental in the development of many innovative methods andapplications. This article reviews some of their contributionsand discusses how their approaches to these problems may bea model for addressing issues that are being encountered currentlyin the United States in cell phone research.     

Telephone sampling bias in surveying disability

Results of the California Disability Survey indicate that telephone interviewing is well suited for undertaking disability studies that provide (1) estimates of subgroups of the disabled population, including those that are statistically rare: (2) information on current and anticipated areas of policy concern; and (3) information for geographic areas important in rehabilitation program planning. Although these objectives necessitated a large sample size and a complex instrument, the costs and timeliness of telephone interviewing enabled the survey objectives to be reached. This paper discusses the advantages of telephone interviewing, as well as biases inherent in its use. The magnitude of the bias from the omission of nontelephone households is assessed, and the results confirm that the omission of nontelephone households introduces only minor biases into estimates for the total working-age population. A method of weighting is developed and illustrated.     

Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys

The high costs and largely unknown error properties of cellulartelephone interviews make screening for cell-only adults a potentiallyattractive option in dual frame RDD surveys. Screening out adultswith landline telephones from the cellular sample does not affectthe coverage properties of a dual frame survey, but it may affectother sources of error, especially nonresponse. In this study,data from a 2006 dual frame RDD survey conducted for the PewResearch Center, the Associated Press, and AOL are used to evaluatethe effects of implementing a cell-only screener on both thebias and variance of weighted survey estimates. The effect ofscreening appears to be minimal so long as an adjustment fortelephone service is included in the weighting method. Resultsof an attempt to correct for residual nonresponse due to inaccessibilityare also discussed.