Monitoring youth behavior in response to structural changes. Alternative approaches for measuring adolescent drinking

Random digit dial (RDD) telephone and self-administered school-based surveys of drinking and drinking and driving were compared for adolescents from two Northern California communities. The RDD and school-based surveys resulted in very different samples. The telephone sample contained a greater proportion of European Americans and a smaller proportion of Asian Americans and "other" ethnicities. Respondents to the telephone sample also tended to be older and of higher socioeconomic status (SES). The telephone survey captured very few school dropouts. Moreover, it resulted in lower self-reports of drinking and drinking and driving. Survey mode appeared to influence respondents equally, regardless of their gender, age, ethnicity, or SES.     

USING DUAL FRAME DESIGNS TO REDUCE NONRESPONSE IN TELEPHONE SURVEYS

This article reports on the results of a series of experimentsdesigned to improve response rates for telephone surveys. Inthree surveys telephone households were selected using bothstandard random digit dialing (RDD) techniques and lists oftelephone numbers purchased from a commercial firm. In the RDDportions of the samples "cold contact" interviewing methodswere used; in the list frame portions advance letters were mailed,and the listed household name was used in the introduction.Experiments were designed to test the effects on response ratesof the advance letters and use of the listed household nameas a means of establishing rapport. The advance letters increasedresponse rates, but no difference could be attributed to theuse of names. The mixture of RDD and list sampling techniquesis also used to evaluate the effects of relative response rateson substantive findings. The cost consequences of these dualframe designs are assessed along a number of dimensions, andthe cost and error components of these designs are discussed.     

Can Registration-Based Sampling Improve the Accuracy of Midterm Election Forecasts?

We compare the predictive accuracy of preelection polls usingtwo types of sampling frames, random digit dialing (RDD) andregistration-based sampling (RBS). The latter involves stratifiedrandom sampling from voter registration lists. In order to assessthe accuracy with which RDD and RBS predict election outcomes,we collaborated with the Washington Post, Quinnipiac, and CBSNews polls, which conducted parallel RDD and RBS surveys inMaryland, New York, Pennsylvania, and South Dakota prior tothe November 5, 2002, elections. The results suggest that inthe gubernatorial and congressional elections studied, RBS performedas well, if not better, than RDD, both in terms of forecastingaccuracy and cost.     

Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys

The high costs and largely unknown error properties of cellulartelephone interviews make screening for cell-only adults a potentiallyattractive option in dual frame RDD surveys. Screening out adultswith landline telephones from the cellular sample does not affectthe coverage properties of a dual frame survey, but it may affectother sources of error, especially nonresponse. In this study,data from a 2006 dual frame RDD survey conducted for the PewResearch Center, the Associated Press, and AOL are used to evaluatethe effects of implementing a cell-only screener on both thebias and variance of weighted survey estimates. The effect ofscreening appears to be minimal so long as an adjustment fortelephone service is included in the weighting method. Resultsof an attempt to correct for residual nonresponse due to inaccessibilityare also discussed.     

A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys

Valid and reliable public health data are becoming more difficultto obtain through random-digit dial (RDD) telephone surveys.As a result, researchers are evaluating different survey designs(i.e., sampling frame and survey mode combinations) as complementsor alternatives to RDD. Traditionally, mail surveys of the generalpublic have been limited due to a lack of a complete samplingframe of households. Recent advances in electronic record keeping,however, have allowed researchers to develop a sample from aframe of addresses (e.g., the U.S. Postal Service Delivery SequenceFile, which appears to provide coverage which rivals or possiblyexceeds that obtained through RDD sampling methods). To testthe use of this frame for surveying adults aged 18 years andolder across a wide geographic area, a pilot study was conductedas part of the 2005 Behavioral Risk Factor Surveillance System(BRFSS). The pilot compared use of a traditional, RDD telephonesurvey methodology to an approach using a mail version of thequestionnaire completed by a random sample of households drawnfrom an address-based frame. The findings indicate that themail survey approach can achieve higher response rates in low-response-ratestates (<40%) than RDD (particularly when two mailings aresent). Additionally, the address frame with mail survey designprovides access to cell phone only households and offers costsavings over the telephone approach. The resulting sample, however,significantly overrepresents non-Hispanic whites and peoplewith higher levels of education.     

Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

How Too Little Can Give You a Little Too Much: Determining the Number of Household Phone Lines in RDD Surveys

Accurately assessing the number of household phone lines inrandom-digit dialed (RDD) surveys for use in weighting has becomemore complicated over time. This study evaluates phone lineweighting by asking an in-depth battery of seven questions designedto measure the number of available land lines in the home moreprecisely. The results show that this weighting correction isnot so simple, and that inadequately parsing the type and useof household phone numbers produces an inaccurate weight. Failingexplicitly to remove irrelevant phone numbers, especially cellphones, produces a larger-than-appropriate weighting adjustment.Potentially more important on a practical level is that, whendone correctly, and even when "overdone," phone line weightinghad little impact across a diverse set of opinion and demographicvariables.     

SUBSTANCE USE ESTIMATES FROM RDD AND AREA PROBABILITY SAMPLES: IMPACT OF DIFFERENTIAL SCREENING METHODS AND UNIT NONRESPONSE

Survey mode effects linked to unit nonresponse and screeningprocedures were estimated in a study of self-reported to bacco,alcohol, and illicit drug use. Two samples of the same population were drawn: a random digit dial (RDD) sample with telephonescreening and a multistage area probability sample with in-personscreening. Respondents in both samples were interviewed by telephoneto control mode of communication effects. Households with outtelephones were excluded from both samples to control samplecoverage effects. Although results show substantially lowerresponse rates in the telephone survey due to screening refusals,the two survey procedures produced samples that were very similarin demographic characteristics and in substance use profiles.Differential screening operations and differential nonresponserates between the RDD and area probability designs did not appearto be a major source of mode effects in comparing telephoneand face-to-face surveys of substance use.     

Estimating Response to Follow-ups in Mail Surveys

A simple method for estimating the effectiveness of successivefollow-ups in mail surveys is presented.     

Estimating response to follow-ups in mail surveys.

A simple method for estimating the effectiveness of successive follow-ups in mail surveys is presented.     

EFFECTS OF INTERVIEW MODE ON SELF-REPORTED DRUG USE

The effects of interview mode on 18- to 34-year-olds' self-reportedtobacco, alcohol, marijuna, and cocaine use were investigated.RDD telephone and personal drug use surveys of the state ofNew Jersey were conducted in 1986–87. In the personalinterview, respondents recorded their drug use on self-administratedanswer sheets. Compared to the area probability sample in thepersonal interview, RDD yielded a sample of blacks higher inincome and education, and more likely to be married and currentlyemployed; white SES was also higher in the telephone sample,but to a lesser degree than for blacks. Controlling for demographiccharacteristics and RDD's exclusion of non-telephone households,the telephone survey yielded significantly lower estimates ofblacks' alcohol consumption, and lifetime and recent marijuanause. Whites' alcohol consumption was slightly lower by telephone;otherwise, estimates of whites' use of the four substances werenearly identical in the two modes. Sample coverage, respondentdemographic characteristics, and racial matching of interviewerand respondent did not account for the significant mode differences.Characteristics of the interview situation itself, such as provisionof privacy in the self-administered format, may have influencedtendencies toward socially desirable responding to a threateningtopic such as drug use.     

The Influence of Incremental Increases in Token Cash Incentives on Mail Survey Response: Is There an Optimal Amount?

The research reported is from a large-scale, national, mixed-modesurvey (N = 49,675 addresses), in which token incremental, noncontingentincentives used in the second (mail) stage of the survey weretested in an experimental design. The levels of the incentivestested ranged from $0 to $10, in one-dollar increments, withthe exception that there was no $9 condition. Furthermore, theexperiment was conducted using three types of household addressesthat originally were sampled in the first stage of the mixed-modesurvey via a random digit dial (RDD) frame: (1) households thatagreed to participate in the mail stage of the survey; (2) householdsthat were never contacted in the RDD stage despite numerouscontact attempts; and (3) households that had refused to participatewhen contacted in the RDD stage. It was determined that amongthose households that had previously agreed to participate,even those that received no cash incentive ($0) were more likelyto return completed surveys than the groups of previously noncontactedand previously refusing households that received $10. It wasalso found that the nature of the outcome of the prior contactwith the household mediated (that is, interacted with) the proportionaleffects of the incremental incentives, in ways only partiallyconsistent with Dillman’s social exchange theory. Thefindings, which are consistent with leverage-salience theory,suggest that if there is an optimal amount for an incentiveto use in a mail survey, then it is a variable amount best targetedto the individual household.     

Reaching a representative sample of college students: A comparative analysis

ABSTRACT

Objective: To explore the feasibility of a random-digit dial (RDD) cellular phone survey in order to reach a national and representative sample of college students. Methods: Demographic distributions from the 2011 National Young Adult Health Survey (NYAHS) were benchmarked against enrollment numbers from the Integrated Postsecondary Education System (IPEDS). The sample quality was compared with the Harvard School of Public Health College Alcohol Study (1993), National College Health Risk Behavior Survey (1995), and National College Health Assessment (2011). Results: Overall, the NYAHS performed as well, if not better, than previous college health surveys at reaching important demographic subgroups. Conclusions: Cellular phone RDD is one feasible approach for studies of the general population of college students that can potentially generate national estimates of health behavior.     

BIAS IN LIST-ASSISTED TELEPHONE SAMPLES

A number of researchers have suggested list-assisted samplingfor the selection of telephone households to overcome some ofthe operational difficulties associated with the Mitofsky-Waksbergmethods of random digit dialing (RDD). An advantage of a list-assistedmethod of RDD is that an equal probability systematic sampleof telephone numbers can be selected and the variances of estimatesfrom such a sample are usually lower than from a clustered designlike the Mitofsky-Waksberg method. The main disadvantage ofthe list-assisted method is that it excludes some householdsfrom the sample, thus creating a coverage bias in the estimates.This article describes research on the coverage bias for a particularmethod of list-assisted sampling. The two key determinants ofcoverage bias are the proportion of households that are noteligible for the sample and the differences in the characteristicsof the covered and not covered populations. The results showthat about 4 percent of all households are excluded in nationalsamples using this method of sampling. Furthermore, they showthat the differences between the covered and uncovered populationsare generally not large. The coverage bias resulting from theseconditions may often be small.     

THE EFFECTS OF EDUCATION ON FERTILITY: EVIDENCE FROM TAIWAN

This study investigates the impact of women's education on fertility. For identification, we use the 1968 compulsory education law change in Taiwan, which generated a regression discontinuity design (RDD) setting. We use the whole population of women from the 1980 and 2010 Population Censuses. Results of our RDD estimation using the exact date of birth suggest that the law change was effective in boosting women's education, but it did not have any impact on fertility. This is in stark contrast to most previous studies using only the birth year as the running variable or using it to construct instruments, which find that women's education depresses fertility. This study demonstrates that using a discrete running variable in RDD may generate a false discontinuity for an otherwise continuous regression function. (JEL J13, C21, I2)     

What's Missing from National Landline RDD Surveys?: The Impact of the Growing Cell-Only Population

The number of cell phone only households has continued to grow– 12.8 percent of all households by the end of 2006, accordingto the National Health Interview Survey (NHIS). To assess theextent and nature of the potential bias in landline telephonesamples created by the absence of cell-only households, thePew Research Center conducted four independent dual frame studiesin 2006, three of which included at least 200 interviews withcell-only respondents. Despite the fact that there are significantdifferences between cell-only and landline respondents on manyimportant variables, across the four surveys with differentsubstantive content we find that including a cell-only samplewith a landline RDD sample produces general population estimatesthat are nearly identical to those from the landline samplealone. Yet, while the noncoverage problem is currently not damagingestimates for the entire population, we find evidence that itdoes create biased estimates on certain variables for youngadults, 25 percent of whom are cell-only according to the mostrecent government estimate.     

Advance Letters as a Means of Improving Respondent Cooperation in Random Digit Dial Studies: A Multistate Experiment

Advance letters can potentially reduce the degree of nonresponsein random digit dial (RDD) surveys; however, they can also havea heterogeneous impact on subgroups, disproportionately raisingparticipation rates among certain segments of the populationand thereby having a detrimental effect on nonresponse bias.This is, in part, because advance letters can only be used inRDD surveys with the subset of respondents for whom an addresscan be identified. It may also be related to who in a householdsees the letter. We assess whether the use of advance letterscan improve the level of participation in the Behavioral RiskFactor Surveillance System (BRFSS) without introducing otherpotential data biases. The data reported here corroborate previousfindings, in terms of the positive impact that advance letterscan have on overall response rates (approximately a 6-percentage-pointgain). Moreover, the advance letters were cost-efficient inthat the cost of obtaining a fixed number of completed surveysusing advance letters was lower than the cost without letters.However, the positive impact of the advance letter on reducingnonresponse may have been offset to some extent in that theadvance letter may have biased the sample of those who completedthe interview toward older, white respondents and those of highersocioeconomic status, and away from younger, nonwhite individualsand persons with lower education and income levels. This lattergroup was underrepresented in the final sample and also lesslike to remember having received a letter.     

A model based approach for predicting annual poverty rates without expenditure data

The primus inter pares of the UN-approved Millennium Development Goals is to reduce poverty. The only internationally accepted method of estimating poverty requires a measurement of total consumption based on a time-consuming and resource-demanding measure of household expenditure in an integrated survey over 12 months. Rather than measuring poverty, say, only every fifth year, a model is presented to predict poverty based on a small set of household variables to be collected annually between two 12-monthly household surveys. Information obtained from these “light” surveys might then be used to predict poverty rates. The key question is whether the inaccuracy in these predictions is acceptable. It is recommended that these models be tested at a country level and if the test results are similar to those found here, that this approach be adopted.      

The measurement of unemployment using completed durations: evidence on the gender gap in unemployment in France

This paper examines how unemployment can be measured in a normative fashion - taking into account the mean and inequality of spell lengths - and how the extent of unemployment can be estimated from cross section data of the type found in labour force surveys. The issue is not straightforward since in these surveys completed durations of unemployed individuals are not observed yet they constitute the basis for calculating the kind of index that has been proposed to measure the extent of unemployment in a way that goes beyond the unemployment rate. The index proposed by Shorrocks has robust normative foundations and has an equivalent representation in terms of average complete duration and the density of completed durations. Building upon earlier work applied in the United States for estimating the average completed duration, we present a method that enables the index to be calculated based on an estimate of the density of completed durations. The approach is illustrated in the context of comparing male-female unemployment differences in France, where historically female unemployment has been higher than that of males.     

Estimation of International Migration in Post‐Soviet Republics

When annual migration data lack reliability, scholars apply alternative methods for estimating international migration. Yet, researchers note that alternative approaches have primarily been tested on developed countries, rather than developing countries that usually have dramatic migration shifts. I close this research gap. I use the example of 15 former Soviet republics to demonstrate several conclusions. First, I show that such alternative approaches as immigration‐by‐origin data of receiving countries do not result in reliable and valid estimates of post‐Soviet migration, given the large variation that exists in how former Soviet republics define “migrant”. Second, I demonstrate that population censuses, while a more superior alternative, fail to capture temporary migrants. In developing countries, the international emigration is mainly due to temporary (undocumented labour) migration. Third, I suggest that scholars and policy‐makers should apply household surveys as a possible alternative. However, while this method seems promising, given the limited use of household surveys in migration measurement in the post‐Soviet republics, future research by both scholars and applied researchers should explore the advantages and limitations of household surveys as an alternative source for estimation of migration. Finally, I outline methodological guidelines that researchers and scholars can advance on migration issues in the post‐Soviet region.