Cell Phone Survey Feasibility in The U.S.: Sampling and Calling Cell Numbers Versus Landline Numbers

In 2004, we conducted a nationwide dual frame survey of landlineand cell phone numbers to evaluate the feasibility of includingcell phone numbers in a random digit dial telephone survey.Households with both landline and cell phones were eligiblefor selection in both samples. This article describes our designand data collection methods; compares the results from the twosamples (with an emphasis on operational characteristics); andpresents the outcomes of two experimental manipulations designedto improve the cell phone response rate.     

Telephone surveys under different frame constructions—evaluating differences in estimates,precision, and cost

Dual-frame, random-digit-dialing (DFRDD) surveys, using a frame of landline telephone numbers and a frame of cell phone numbers, have become the norm for telephone surveys. Two alternative frame constructions sample exclusively from a cell phone frame, referred to as the ‘single-frame cell,’ and sample from the listed landline frame and the cell phone frame, referred to as ‘listed/cell.’ The listed/cell frame construction differs from the DFRDD in that it does not select unlisted-landline frame members. We observe in our simulated data that the listed/cell frame construction results in estimates that more closely approximate the estimates from DFRDD compared with the single-frame cell, results in an improvement in precision compared with the DFRDD that is not as large as the improvement in the single-frame cell, and has lower cost than both the DFRDD and the single-frame cell. We conclude that a listed/cell frame construction might be a better solution than either a DFRDD or single-frame cell frame construction.     

Sampling Telephone Numbers and Adults, Interview Length, and Weighting in The California Health Interview Survey Cell Phone Pilot Study

This article describes several features included in a CaliforniaHealth Interview Survey cell phone pilot study that differ fromearlier cell phone surveys conducted in the United States. Onedifference is that the study used a screening design and onlyadults living in cell-only households were interviewed. Mostof the previous studies used dual frame designs. Another differencewas the development and implementation of a within-householdadult sampling procedure to cover adults when cell phones wereshared in the household. The study was also intended to determineif conducting a cell phone interview of the same scope and lengthas a regular telephone interview was feasible. Most previoussurveys were focused on cell phone topics and were shorter thanthe comparable landline surveys. We then explore some interestingproblems that arise in weighting a cell phone survey using ascreening design. We conclude with a discussion of planninga new cell phone survey in 2007 that incorporates findings fromthis study.     

Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys

The high costs and largely unknown error properties of cellulartelephone interviews make screening for cell-only adults a potentiallyattractive option in dual frame RDD surveys. Screening out adultswith landline telephones from the cellular sample does not affectthe coverage properties of a dual frame survey, but it may affectother sources of error, especially nonresponse. In this study,data from a 2006 dual frame RDD survey conducted for the PewResearch Center, the Associated Press, and AOL are used to evaluatethe effects of implementing a cell-only screener on both thebias and variance of weighted survey estimates. The effect ofscreening appears to be minimal so long as an adjustment fortelephone service is included in the weighting method. Resultsof an attempt to correct for residual nonresponse due to inaccessibilityare also discussed.     

What's Missing from National Landline RDD Surveys?: The Impact of the Growing Cell-Only Population

The number of cell phone only households has continued to grow– 12.8 percent of all households by the end of 2006, accordingto the National Health Interview Survey (NHIS). To assess theextent and nature of the potential bias in landline telephonesamples created by the absence of cell-only households, thePew Research Center conducted four independent dual frame studiesin 2006, three of which included at least 200 interviews withcell-only respondents. Despite the fact that there are significantdifferences between cell-only and landline respondents on manyimportant variables, across the four surveys with differentsubstantive content we find that including a cell-only samplewith a landline RDD sample produces general population estimatesthat are nearly identical to those from the landline samplealone. Yet, while the noncoverage problem is currently not damagingestimates for the entire population, we find evidence that itdoes create biased estimates on certain variables for youngadults, 25 percent of whom are cell-only according to the mostrecent government estimate.     

Mexican Immigrants,Labour Market Assimilation and the Current Population: The Sensitivity of Results Across Seemingly Equivalent Surveys

In this paper we compare estimates of immigrants’ labour supply assimilation profiles using the Current Population Survey Annual Demographic Files (March ADS) and the Current Population Survey Outgoing Rotation Groups (ORGs). We use a measure that is seemingly consistent across both surveys: usual weekly hours of work in the main job. This measure is not only important as it measures the intensive margin of labour supply, but it is also used to estimate hourly earnings in both surveys. Our results indicate that the two surveys produce dramatically different estimates of the change in average hours of work as immigrants’ years in the United States increase: estimates from the March ADS predict much steeper hour's assimilation profiles than do estimates obtained from the ORGs. These differences persist if we include controls for demographic characteristics, and only disappear once we control for occupation heterogeneity. We argue that these differences stem from two separate factors that differentiate the data. First, the ADS and ORG frame the usual “hours worked” question differently. Also, differences in the timing of the surveys may produce seasonality effects that differentially affect the composition of recent and earlier migrants, thereby changing assimilation profiles.     

Kernel density estimation on grouped data: the case of poverty assessment

Grouped data have been widely used to analyze the global income distribution because individual records from nationally representative household surveys are often unavailable. In this paper we evaluate the performance of nonparametric density smoothing techniques, in particular kernel density estimation, in estimating poverty from grouped data. Using Monte Carlo simulations, we show that kernel density estimation gives rise to nontrivial biases in estimated poverty levels that depend on the bandwidth, kernel, poverty indicator, size of the dataset, and data generating process. Furthermore, the empirical bias in the poverty headcount ratio critically depends on the poverty line. We also undertake a sensitivity analysis of global poverty estimates to changes in the bandwidth and show that they vary widely with it. A comparison of kernel density estimation with parametric estimation of the Lorenz curve, also applied to grouped data, suggests that the latter fares better and should be the preferred approach.     

Voters and Values in the 2004 Election

A poorly devised exit poll question undermined meaningful analysisof voters’ concerns in the 2004 presidential election.Twenty-two percent of voters picked "moral values" from a listof "issues" describing what mattered most in their vote, morethan selected any other item. Various commentators have misinterpretedthis single data point to conclude that moral values are anascendant political issue and to credit conservative Christiangroups with turning George W. Bush’s popular vote defeatin 2000 into his three million–vote margin of victoryin 2004. We suggest, rather, that while morals and values arecritical in informing political judgments, they represent personalcharacteristics and ill-defined policy preferences far morethan any discrete political issue. First by conflating moralsand values and then by further conflating characteristics andissues, the exit poll’s "issues" list distorted our understandingof the 2004 election. In this article, we examine the flawsin the 2004 National Election Pool exit poll’s "most importantissue" question and explore the presumed rising electoral importanceof moral values and the conservative Christians who overwhelminglyselected this item. Using national exit poll data from 1980through 2004 and other national surveys, we find that the moralvalues item on the issues list cannot properly be viewed asa discrete issue or set of closely related issues; that itsimportance to voters has not grown over time; and that whencontrolled for other variables, it ranks low on the issues listin predicting 2004 vote choices. The aggregated exit poll dataalso show that the voting behavior of conservative Christiansis relatively stable over time, and these voters were not primarilyresponsible for Bush’s improvement in 2004 over 2000.     

Citizenship Beyond Borders: A Cross‐National Study of Dual Citizenship*

As global integration increases, the implications for state boundaries and citizens’ identity grow more significant. Some scholars suggest that the recognition of dual citizenship reveals the extent to which cross‐national immigration requires states to formally recognize a multiplicity of national identities through dual citizenship ( Aleinikoff and Klusmeyer 2002 ; Castles and Davidson 2000 ; Falk 1994 ). We propose that scholars need to additionally consider citizenship identity as a source of national assimilation of the international community and postnational citizenship in world culture ( Brubaker 1992a ; Faist 2004 ; Soysal 1994 ; Turner 2001 ). We use logistic regression to evaluate this argument by examining factors that lead states to enact legislation recognizing dual citizenship. The resulting analysis suggests that the recognition of dual citizenship reflects national, ex‐colonial, and postnational cultural identities rather than the presence of cross‐national immigration.     

A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys

Valid and reliable public health data are becoming more difficultto obtain through random-digit dial (RDD) telephone surveys.As a result, researchers are evaluating different survey designs(i.e., sampling frame and survey mode combinations) as complementsor alternatives to RDD. Traditionally, mail surveys of the generalpublic have been limited due to a lack of a complete samplingframe of households. Recent advances in electronic record keeping,however, have allowed researchers to develop a sample from aframe of addresses (e.g., the U.S. Postal Service Delivery SequenceFile, which appears to provide coverage which rivals or possiblyexceeds that obtained through RDD sampling methods). To testthe use of this frame for surveying adults aged 18 years andolder across a wide geographic area, a pilot study was conductedas part of the 2005 Behavioral Risk Factor Surveillance System(BRFSS). The pilot compared use of a traditional, RDD telephonesurvey methodology to an approach using a mail version of thequestionnaire completed by a random sample of households drawnfrom an address-based frame. The findings indicate that themail survey approach can achieve higher response rates in low-response-ratestates (<40%) than RDD (particularly when two mailings aresent). Additionally, the address frame with mail survey designprovides access to cell phone only households and offers costsavings over the telephone approach. The resulting sample, however,significantly overrepresents non-Hispanic whites and peoplewith higher levels of education.     

USING DUAL FRAME DESIGNS TO REDUCE NONRESPONSE IN TELEPHONE SURVEYS

This article reports on the results of a series of experimentsdesigned to improve response rates for telephone surveys. Inthree surveys telephone households were selected using bothstandard random digit dialing (RDD) techniques and lists oftelephone numbers purchased from a commercial firm. In the RDDportions of the samples "cold contact" interviewing methodswere used; in the list frame portions advance letters were mailed,and the listed household name was used in the introduction.Experiments were designed to test the effects on response ratesof the advance letters and use of the listed household nameas a means of establishing rapport. The advance letters increasedresponse rates, but no difference could be attributed to theuse of names. The mixture of RDD and list sampling techniquesis also used to evaluate the effects of relative response rateson substantive findings. The cost consequences of these dualframe designs are assessed along a number of dimensions, andthe cost and error components of these designs are discussed.     

Popular Support for Democracy and its Institutions in Korea: The Dynamics and Sources of Regime Support and Institutional Trust

This article examines the contours and dynamics of popular support for democracy in South Korea, a country widely known as one of the most successful third-wave democracies in Asia. Analysis of the Korea Democracy Barometer and East Asia Barometer surveys conducted between 1996 and 2004 reveals that ordinary Koreans’ support for democracy has moved both downward and upward during the past eight years. Analysis of these surveys also reveals that the trajectories and sources of that movement vary considerably across different types of democratic support. Based on this finding that democratic authorization and exercise of power affect various manifestations of democratic support differently, we argue that to understand the dynamics of support for democracy accurately, one must distinguish between the realm of political performance and that of governmental performance.     

Requests, Blocking Moves, and Rational (Inter)action in Survey Introductions

We draw on conversation analytic methods and research to explicate the interactional phenomenon of requesting in general and the specific case of requesting participation in survey interviews. Recent work on survey participation has given much attention to leverage-saliency theory, but has not engaged how the key concepts of this theory are exhibited in the actual unfolding interaction of interviewers and potential respondents. We do so using digitally recorded and transcribed calls to recruit participation in the 2004 Wisconsin Longitudinal Study. We describe how potential respondents present interactional environments that are relatively discouraging or encouraging, and how, in response, interviewers may be relatively cautious or presumptive in their requesting actions. We consider how the ability of interviewers to tailor their behavior to their interactional environment can affect whether the introduction reaches the point at which a request to participate is made, the form that this request takes, and the sample person's response. Our analysis contributes to understanding how we might use insights from the analysis of interaction to increase cooperation with requests to participate in surveys.     

Under Two Flags: Dual Nationality in Latin America and Its Consequences for Naturalization in the United States

This article surveys the practice of dual nationality in the Western Hemisphere, particularly as it impacts the naturalization rates of immigrants in the United States. The article begins by looking at the extent and spread of dual nationality provisions and the pathways for its implementation. Next, the article turns to a discussion of the multiple (and at times conflicting) interests – of immigrants, sending states and receiving states – in dual nationality. While immigrants and sending countries are in general agreement on the positive benefits of dual nationality, commentators in receiving countries like the United States continue to express deep unease at the spread of dual nationality and its consequences for American citizenship. Are these concerns justified? Not according to U.S. naturalization rates. Data from 1965 to 1997 indicate that immigrants from countries recognizing dual nationality average higher naturalization rates in the United States than countries that do not.     

The State of Surveying Cell Phone Numbers in the United States: 2007 and Beyond

By the late 1970s, household telephone coverage grew to exceed90 percent in the United States, and by the mid-1980s telephonesurveying of the general public had become commonplace. Nevertheless,20 years later, the ability of researchers to reach representativesamples of the U.S. public via landline (wired) telephone surveysand gather reliable data is being seriously challenged for manyreasons, especially those related to cell phones and the growthof the "cell phone only" population. However, at present thereexists no widely accepted set of Cell Phone Surveying "bestpractices" for U.S. survey researchers to follow. Despite whatsome appear to believe, surveying persons reached on cell phonenumbers in the United States currently is a very complex undertakingif one wants to do it "right," i.e. to do it legally, ethically,and in ways that optimally allocate one's finite resources togather the highest quality data, and to analyze and interpretthose data accurately. This final "wrap-up" article in the specialissue provides a review of the empirical articles in the issuewith a focus on their practical implications for the decisionsthat researchers need to make regarding sampling, coverage,nonresponse, measurement, and weighting in surveys that includeinterviews with persons reached on cell phones. The articlealso highlights the practical implications of a number of legal,ethical, and other issues that relate to surveys in the UnitedStates that include cell phone numbers. Surveying the U.S. cellphone population is possible, if at a higher cost than surveyingtheir landline counterparts, and if with less precision thancurrently can be done surveying the landline population. Thenext five years should see a considerable growth in the methodologicaland statistical know-how that the survey community uses to plan,implement, and interpret cell phone surveys. There is a greatdeal that still must be learned.     

Lorenz Dominance and Non-welfaristic Redistribution

Our concern is for income inequalities that may result from non-welfaristic redistribution schemes. We show that for large classes of income functions Lorenz dominance results can be found in the comparison of two egalitarian equivalent mechanisms. Comparisons of different conditionally egalitarian mechanisms only yield poverty dominance results. In general, no egalitarian equivalent mechanism can be Lorenz dominated by a conditionally egalitarian mechanism. Our analysis stresses the need for accurate empirical estimates of the pre-tax income function and of the distributions of responsibility and compensation characteristics. We thank the Editor, Marc Fleurbaey and two anonymous referees, Geert Dhaene and seminar/conference participants at UAP-workshop (Namur, 2003), ‘Welfarist and non-welfarist approaches to public economics’ (Ghent, 2004), SED (Palma, 2004), SSC&W (Osaka, 2004) and IIPF (Milan, 2004) for helpful comments and suggestions. Financial support from the Federal Public Planning Service Science Policy, Interuniversity Attraction Poles Program – Belgian Science Policy [Contract No. P5/21] is gratefully acknowledged.     

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Recent experimental research has shown that respondents to forced-choicequestions endorse significantly more options than respondentsto check-all questions. This research has challenged the commonassumption that these two question formats can be used interchangeablybut has been limited to comparisons within a single survey mode.In this paper we use data from a 2004 random sample survey ofuniversity students to compare the forced-choice and check-allquestion formats across web self-administered and telephoneinterviewer-administered surveys as they are commonly used insurvey practice. We find that the within-mode question formateffects revealed by previous research and reaffirmed in thecurrent study appear to persist across modes as well; the telephoneforced-choice format produces higher endorsement than the webcheck-all format. These results provide further support forthe argument that the check-all and forced-choice question formatsdo not produce comparable results and are not interchangeableformats. Additional comparisons show that the forced-choiceformat performs similarly across telephone and web modes.     

Methodological challenges of designing a survey to capture young people’s (non-binary) affiliations in relationship to religion,sexuality and gender

ABSTRACT

Large-scale population studies surveying young people in relation to their worldviews have tended to frame their identities in a fixed and limited capacity while also treating the topics of religion/spirituality and sexuality/gender as discrete categories of scholarly analysis. We highlight the affordances and limitations of foregrounding fixed religious, sexual and gender-based identity categories in the process of collecting and analysing data related to the worldviews of young people. In this paper we argue the value of studying the complexities and intersections of these identities and worldviews together in one study. We do this through reference to the Australia’s Generation Z (AGZ) study: the first nationally representative sample focused on providing an evidence-based understanding of both the religious/spiritual/non-religious and sexuality/gender identities and worldviews of young Australians aged 13–18. We discuss how we built on existing surveys in designing the AGZ survey. We also demonstrate how this survey allowed for the incorporation of young people’s non-binary understandings of religion, sexuality and gender.     

Household Telephone Service and Usage Patterns in the United states in 2004: Implications for Telephone Samples

Changes in the U.S. telephone system, especially the rapid growthin the prevalence and use of cell phones, raise concerns aboutundercoverage error in random digit dial (RDD) telephone samples.A supplement to the Current Population Survey (CPS) was conductedin 2004 to examine telephone service and usage in U.S. households.This article explores the potential for biases in RDD surveysresulting from the increases in cell phones by presenting estimatesof the percentage of households with different types of telephoneservice, including the percentage of cell-only households, andgiving demographic profiles of households by type of telephoneservice. Logistic regression models examine variables that predictwhether households are without a telephone or only have cellphones. These predictors may be used for weighting adjustmentsto reduce undercoverage biases. We address some additional issues,including the wording of questions for measuring telephone service,that are relevant if telephone-sampling methods are revisedto include cell phones. The estimates from the CPS supplementare also used to help understand some of the new sampling andweighting problems associated with selecting samples from cellphone numbers.     

The Growth of Internet Research Methods and the Reluctant Sociologist*

Between 1999 and 2004 only one article appeared in the American Sociological Review, the American Journal of Sociology, or Social Forces using primary data collected with Web‐based research techniques. Since then there have been only a handful of studies published in these core sociology journals drawing on Web‐based surveys or other forms of Web‐based data. The use of Internet‐based data has become widespread in many academic fields, especially health research and education; Web‐based techniques are becoming routine in the practice and study of politics; and online commercial and market research has become a billion dollar industry. At the same time, the utility of random digit dialing surveys has eroded considerably owing to declining contact rates, increased use of technologies to screen unwanted telephone calls, and the replacement of landline telephones with cell phones. There is increasing evidence that Internet research can produce representative data. Although Web‐based surveys may overrepresent some populations, Internet usage in the general population is now well over 75 per cent and is especially strong among some hard to reach populations. Internet surveys have the potential to reduce measurement error, missing data, and respondent attrition. Sociologists must overcome their fear of participation in stigmatized Internet research and actively engage in the development of techniques and refinements that will increase the utility and validity of Internet‐based data collection.