Must Respondent Selection Procedures for Telephone Surveys Be Invasive?

Two respondent selection procedures are compared in a combinationtelephone and mail survey involving a national probability sampleof over 2,500 telephone households. One procedure asks two householdcomposition questions in order to select a respondent randomly.The alternative asks no preliminary questions. Two hypothesesare proposed: (1) The alternative approach will result in fewerrefusals at the respondent selection phase because no obtrusivehousehold composition questions are asked, and (2) the integrityof the demographic profile of the resulting sample will notbe compromised. Both hypotheses are accepted.     

Experiments with incentives in telephone surveys

In an effort to counter the problem of noncooperation, survey organizations are offering incentives to respondents with increasing frequency, some at the outset of the survey, as has traditionally been done in mail surveys, and some only after the person has refused, in an attempt to convert the refusal. This article reports on a series of experiments carried out over a period of about 2 years with a monthly telephone survey, the Survey of Consumer Attitudes, in an effort to increase response rates or reduce interviewer effort. We report on experiments with prepaid versus promised incentives; advance letters; and advance letters with prepaid incentives; and we also report on the effects of incentives on response quality, sample composition, response bias, interviewer and respondent expectations, and costs.     

Finding Subgroups for Surveys

This article reports on an experiment comparing two methodsfor selecting national samples of blacks. The experiment wascarried out as part of the 1982 General Social Survey (GSS).One sample was selected using area probability sampling withthe selection probability for an area proportional to the sizeof its black population. The other sample was selected by samplingextra dwelling units in areas selected for the regular GSS.Although comparable in other respects, the two methods differedsharply in costs and statistical efficiency. The sample selectedfrom the special black frame proved less expensive to interviewand required fewer screening interviews in order to locate householdswith black members. The sample selected in the same areas asthe regular GSS sample exhibited a smaller "design effect"—probablybecause fewer respondents were selected from each sample area.The decision to use one method or the other to sample rare subgroupswould depend on several factors, including the degree to whichthe subgroup is segregated, the amount of savings possible throughpiggybacking, and the relationship between screening and interviewingcosts.     

Estimating the Nonresponse BiasDue to Refusals in Telephone Surveys

This paper attempts to aid the process of accumulating the necessaryinformation for making more informed judgments about the effectsof nonresponse under different conditions. Two measures, whichpermit quantifiable nonsubjective assessment of the effectsof nonresponse on sample estimates, are introduced and are usedto examine the effects of respondent refusals in a random-digit-dialedgeneral population telephone survey of over 1,200 householdsas the response rate isincreased from 74.5 percent to 86.8 percent.By applying these measures under a wide range of conditions,the adequacy of various response rates may be assessed and morerational decisions made about the costs and benefits of devotingextraordinary resources to minimizing nonresponse.     

FISCAL RULES AND GOVERNMENT BORROWING COSTS: INTERNATIONAL EVIDENCE

We find that the adoption of numerical fiscal rules reduces government borrowing costs in a sample of 101 advanced and developing countries for 1985–2010. We apply a variety of propensity score matching methods to address the self‐selection problem of policy adoption and find strong evidence that fiscal rules have large and significant treatment effects on lowering government borrowing costs in both international and domestic financial markets. The results are robust to changes in country sample and alternative estimation methodology, and are consistent with fiscal rules helping to build policy credibility by reducing the probability of default and the “risk premium” on government debt that compensates lenders for this possibility. (JEL E43, G12, H60)     

Effects and Costs of Tracing Strategies on Nonresponse Bias in a Survey of Workers with Low-Back Injury

Declining rates of participation are an increasing challengefor studies that involve telephone surveys. This study examinedthe costs of a telephone survey methodology that used increasinglyintensive tracing methods to track a pool of claimants who hadsustained occupational back injuries. It also compared the respondentsample to people who refused the survey and/or were not locatedor contacted. 3,181 claimants were drawn from a database maintainedby the Missouri Division of Workers’ Compensation (DWC)and 1,475 completed a telephone interview. The DWC databaseprovided data reflecting monetary and disability outcomes forall potential participants; telephone interviews provided additionalself-reported data. More intensive tracing strategies improvedthe representativeness of the sample, yielding more women andminority participants. Relative to less intensive techniques,advanced tracing efforts located people whose DWC records weremore similar to claimants not located. While the hands-on tracingefforts reduced the apparent bias of the respondent sample,costs increased as tracing strategies intensified – over$98,000 was spent to trace 1,027 claimants who were never locatedor contacted. The results suggest some guidelines that may facilitatedecision-making for researchers and funders who must balancethe trade-off between costs and nonresponse bias when planningsurveys.     

SUBSTANCE USE ESTIMATES FROM RDD AND AREA PROBABILITY SAMPLES: IMPACT OF DIFFERENTIAL SCREENING METHODS AND UNIT NONRESPONSE

Survey mode effects linked to unit nonresponse and screeningprocedures were estimated in a study of self-reported to bacco,alcohol, and illicit drug use. Two samples of the same population were drawn: a random digit dial (RDD) sample with telephonescreening and a multistage area probability sample with in-personscreening. Respondents in both samples were interviewed by telephoneto control mode of communication effects. Households with outtelephones were excluded from both samples to control samplecoverage effects. Although results show substantially lowerresponse rates in the telephone survey due to screening refusals,the two survey procedures produced samples that were very similarin demographic characteristics and in substance use profiles.Differential screening operations and differential nonresponserates between the RDD and area probability designs did not appearto be a major source of mode effects in comparing telephoneand face-to-face surveys of substance use.     

What respondents really expect from researchers

This article addresses the issue of falling response rates in telephone surveys. To better understand and maintain respondent goodwill, concepts of psychological contract and respondent expectations are introduced and explored. Results of the qualitative study show that respondent expectations are not only socially contingent but also ego-expressive, utilitarian, pleasurable, and epistemic by nature. Although results are reassuring in terms of commercialization of the psychological contract, they indicate some radical changes that are needed for the respondents to accept its continuation. The article discusses several practical and theoretical implications of such changes and suggests a series of corresponding propositions aimed at facilitating and inspiring future developments in this field.     

How unclear terms affect survey data

Although writing clear questions is accepted as a general goal in surveys, procedures to ensure that each key term is consistently understood are not routine. Researchers who do not adequately test respondent understanding of questions must assume that ambiguity will not have a large or systematic effect on their results. Seven questions that were drawn from questions used in national health surveys were subjected to special pretest procedures and found to contain one or more poorly defined terms. When the questions were revised to clarify the definition of key terms, significantly different estimates resulted. The implication is that unclear terms are likely to produce biased estimates. The results indicate that evaluation of survey questions to identify key terms that are not consistently understood and defining unclear terms are ways to reduce systematic error in survey measurement.     

Mental health citizen surveys: a comparison of two within household telephone sampling techniques.

Two within household telephone sampling techniques for collecting citizen survey data for community mental health center's needs assessments were compared on the basis of respondent demographic characteristics; response, completion, and refusal rates; and, administrative costs. Initially, all households were chosen via random digit dialing. Then, one technique allowed the surveyor to interview any adult member of a household who answered the phone. The other technique used was developed by Troldahl and Carter and required the use of a reference table procedure to determine the appropriate respondent. While more expensive, the Troldahl-Carter method produced significantly better representativeness of the population for a rural county. Factors relating to a decision regarding which technique to employ are discussed.     

Assessing the effect of changes in housing costs on food insecurity

In this paper, we examine whether changes in housing costs lead to changes in rates of food insecurity for economically vulnerable families. We use data on a national, longitudinal sample of young families with children merged with data on housing (rental) costs at the state, metropolitan statistical area, and county levels (2001–2003). Focusing on families near or at the poverty level and using household fixed effects, we demonstrate that increased housing costs over this time period have indeed increased rates of food insecurity. Our preferred results suggest that a $500 increase in yearly rental costs is associated with nearly a 3% increase in food-insecurity rates (10% relative increase). We show that our measure of rental costs seems to affect only the food insecurity of renters and not that of poor low-income home owners, suggesting the validity of our methods and robustness of the findings. We also look at selected subgroups (e.g., food stamp recipients, individuals receiving housing subsidies) and find few differences in the effects of housing cost increases on food insecurity.     

Preferences,rational choices and economic valuation: Some empirical tests

This study focuses on the respondent rationality hypothesis, usually assumed to be true in Discrete Choice Experiments. We examine lexicographic preferences, the influence of consistency, and the role of task complexity in the individual choice process. To this end, we carry out rationality tests in a survey on forest recreation. Results show that choice set orderings do not impact on choice probability. If a violation of continuity or consistency axioms does impact on choice probability, the Willingness-To-Pay estimators calculated using the total sample and the sub-samples of “irrational” respondents are not significantly different. This serves as a basis for discussing the traditional concept of rationality.     

Active parental consent in school-based research: how much is enough and how do we get it?

Active parental consent policies have been blamed for low participation rates and selection bias (i.e., loss of "high-risk" youths) in school-based studies. In this article, the authors describe active consent procedures that produced an overall active consent rate of 79% in a sample of more than 4,500 middle school students attending 29 schools in seven cities across the United States. Consent rates, however, varied considerably both within and between schools. To better understand factors associated with active parental consent rates, the authors examined district-level, school-level, and teacher-specific effects on consent rates.     

How does active parental consent influence the findings of drug-use surveys in schools?

This study examines the impact of passive and active parental consent procedures on the type of adolescents participating in a school-based survey examining substance use. Schools recruited from a random sample of metropolitan schools were assigned to passive or active parental consent condition. Results showed that participation rates in active consent schools were lower than in passive consent schools for junior students (60% vs. 80%) but not senior students. Although consent condition had limited impact on prevalence estimates among older students, among younger students estimates of cannabis use and ecstasy use were higher in the passive consent condition than the active consent condition. Active consent procedures introduce some degree of selection bias into studies of adolescents' substance use and may compromise the external validity of prevalence estimates produced, especially among younger students.     

Estimating minimum adequate foster care costs for children in the United States

Although foster care homes play a crucial role in providing stable placements to children who enter the child welfare system due to maltreatment, there is currently no federal minimum rate nor standard methodology to establish adequate rates to support foster parents to meet these children's needs. Therefore, it is important to establish a model to estimate the real costs associated with caring for children to serve as a foundation for states to set adequate reimbursement rates. The objectives of this study are to: use the methodology of a 2007 study to establish foster care minimum adequate rates for children (MARC) based on the child's age and geographical location in every state; update the MARC with cost of living adjustments to 2016; examine changes in gaps between the MARC and the current foster care rates; and identify states that have made increases to their reimbursement rates, relative to the MARC over time. Results found that all but four states provide lower foster care reimbursement rates than the adequate costs in 2016. This study recommends that, at the federal level, enhanced precision in operational definitions of care categories could increase consistency in the way that states reimburse foster families. Additionally, findings provide policy suggestions to establish a national methodology standard and increase foster care rates to the level that will meet children's needs. This study will enhance the scant body of literature found on establishing an economic model to estimate foster care costs.     

Respondent-Driven Sampling: An Assessment of Current Methodology

Respondent-Driven Sampling (RDS) employs a variant of a link-tracing network sampling strategy to collect data from hard-to-reach populations. By tracing the links in the underlying social network, the process exploits the social structure to expand the sample and reduce its dependence on the initial (convenience) sample.The current estimators of population averages make strong assumptions in order to treat the data as a probability sample. We evaluate three critical sensitivities of the estimators: to bias induced by the initial sample, to uncontrollable features of respondent behavior, and to the without-replacement structure of sampling.Our analysis indicates: (1) that the convenience sample of seeds can induce bias, and the number of sample waves typically used in RDS is likely insufficient for the type of nodal mixing required to obtain the reputed asymptotic unbiasedness; (2) that preferential referral behavior by respondents leads to bias; (3) that when a substantial fraction of the target population is sampled the current estimators can have substantial bias.This paper sounds a cautionary note for the users of RDS. While current RDS methodology is powerful and clever, the favorable statistical properties claimed for the current estimates are shown to be heavily dependent on often unrealistic assumptions. We recommend ways to improve the methodology.     

THE IMPORTANCE OF PERSISTENCE IN RESPONDENT SELECTION FOR PREELECTION SURVEYS

Different techniques for respondent selection can affect dataquality. These differences can result in variations in the distributionof partisans in preelection surveys, which in turn can havean effect on the distribution of candidate preference. Persistencein trying to interview designated respondents in telephone householdsincreased the number of Republicans in a 1984 sample, and thereforeReagan's margin over Mondale. Such differences in interviewingtechniques might account for some of the variations in nationalpreelection estimates of the outcome of the presidential election,and they suggest that caution be used in comparing marginalsfor party identification from different surveys employing differentrespondent selection techniques.     

Minimizing attrition in longitudinal studies of special populations: An integrated management approach

Whilst statistical techniques are available to estimate effects for missing subjects in longitudinal designs, minimizing subject attrition is a major goal in longitudinal research. This may be particularly difficult when attempting to follow individuals who have special characteristics or disabilities. The Colorado Treatment Outcome Study, a longitudinal study of individuals with serious and persistent mental illness (SPMI), developed an integrated management approach that resulted in average re-contact rates of 95% in each follow-up year. This integrated management approach emphasized the partnership of all key parties to the research, including the community, the respondents, the field interviewers, and management. Guided by a set of values that emphasized respect for the respondent, confidentiality and community relationships, procedures were developed that included an information system, office protocols, staff selection, supervision, staff training, and subject location and engagement techniques. While each element is explicated with attention to special populations, it is their coordination and integration that is critical for success.     

The Origins of Divergent Methodological Stances in Symbolic Interactionism*

Utilizing replies from a selected sample of contributors to the literature of symbolic interactionism, this paper examines the nature and extent of institutional ties among respondents holding similar views as to appropriate methodology in sociology. The specific institutional ties examined are those as: (1) fellow graduate students, (2) teachers and students, and (3) departmental colleagues. The methodological differences between the Chicago and Iowa schools, described in other studies, manifest themselves in the present sample, as do other views not neatly classifiable within either of these schools of symbolic interactionism. The more unconventional the conception of methodology held by a respondent, the more likely is the respondent to have been a fellow graduate student, a departmental colleague, and/or involved in a teacher-student relationship with other symbolic interactionists favoring the same conception. Respondents preferring the more orthodox methodological stances in modern sociology are less likely to have shared in such institutional ties.     

Bias decomposition and estimator performance in respondent-driven sampling

Respondent-Driven Sampling (RDS) is a method of network sampling that is used to sample hard-to-reach populations. The resultant sample is non-random, but different weighting methods can account for the over-sampling of (1) high-degree individuals and (2) homophilous groups that recruit members more effectively. While accounting for degree-bias is almost universally agreed upon, accounting for recruitment-bias has been debated as it can further increase estimate variance without substantially reducing bias. Simulation-based research has examined which weighting procedures perform best given underlying population network structures, group recruitment differences, and sampling processes. Yet, in the field, analysts do not have a priori knowledge of the network they are sampling. We show that the RDS sample data itself can determine whether a degree-based estimator is sufficient. Formulas derived from the decomposition of a ‘dual-component’ estimator can approximate the ‘recruitment component’ (RC) and ‘degree component’ (DC) of a sample’s bias. Simulations show that RC and DC values can predict the performance of different classes of estimators. Samples with extreme ‘RC’ values, a consequence of network homophily and differential recruitment, are better served by a classical estimator. The use of sample data to improve estimator selection is a promising innovation for RDS, as the population network features that should guide estimator selection are typically unknown.