CHARITIES, NO; LOTTERIES, NO; CASH, YES: MAIN EFFECTS AND INTERACTIONS IN A CANADIAN INCENTIVES EXPERIMENT

The effects on response rates to mail surveys of prepaymentcash incentives, charitable donations, and the chance to wina lottery prize are examined in a true experimental design.A survey on environmental issues was conducted during the fallof 1994 among 3,088 households drawn from the Grand River watershedregion of southwestern Ontario, Canada. The 20-cell incentivesexperiment crossed three levels of prepaid monetary cash incentive($2, $5, and $10) with a control group (no cash incentive),plus the offer to make a charitable donation ranging from $2to $10 on behalf of the respondent, or participation in a lotteryfor prizes of $200. The overall response rate to the survey,with three follow-ups, was 70.7 percent with the effects ofthe prepaid cash rewards demonstrated, but not those for thelottery or charitable donations. Logistic regression analysisof response rate supports a main effects model for cash incentivesbut does not support either of two alternative models for maineffects of the lottery and charitable donations or interactions.Further analysis demonstrates a pattern of earlier responseas a result of prepaid incentives. The findings are discussedin terms of their cost-benefit implications and with respectto conceptualizations of respondents' motives for completingmail questionnaires.     

Do Advance Letters Improve Preelection Forecast Accuracy?

The survey methodology literature has debated whether advanceletters to potential survey respondents will reduce nonresponsebias and thereby improve the accuracy of preelection forecasts.This research note analyzes the results of experiments conductedin Maryland, New York, and Pennsylvania in which advance letterswere sent to a random sample of potential survey respondentsto 2002 preelection surveys. We find a significant increasein the overall response rate, although notably less than inpast studies. However, the advance letters did not improve therepresentativeness of survey respondents or the accuracy ofthe election forecasts.     

The Influence of Advance Letters on Response in Telephone Surveys: A Meta-Analysis

Recently, the leading position of telephone surveys as the majormode of data collection has been challenged. Telephone surveyssuffer from a growing nonresponse, partly due to the generalnonresponse trend for all surveys and partly due to changesin society and technology influencing contactability and willingnessto answer. One way to counteract the increasing nonresponseis the use of an advance letter. In mail and face-to-face surveys,advance letters have been proven effective. Based on the proveneffectiveness in face-to-face and mail surveys, survey handbooksadvise the use of advance letters in telephone surveys. Thisstudy reviews the evidence for this advice and presents a quantitativesummary of empirical studies on the effectiveness of advanceletters in raising the response rate for telephone surveys.The major conclusion is that advance letters are also an effectivetool in telephone surveys, with an average increase in responserate (RR1) from 58 percent (no letter) to 66 percent (advanceletter), and an average increase in cooperation rate (COOP1)from 64 percent (no letter) to 75 percent (advance letter).     

Contracts, Individual Revenue and Performance

In some jobs individual workers have control over revenue, effort and productivity. These jobs include professional firms for law, medicine and consulting. They include personal services in areas from hair styling to taxi driving. The firm offers contracts that allow for a sharing of risks and rewards. These incentives include a split of output between the firm and worker and employee ownership. For U.S. real estate agents, a choice is available between splitting revenue with the firm or retaining 100?% above a fixed prepaid minimum. These are equity and sequential debt contracts. Under the sequential debt contract, effort increases but output per hour declines. Separately, agents increase effort and productivity if offered ownership in the firm, effectively a claim on others?? performance.     

The Cost-Effectiveness of Alternative Advance Mailings in a Telephone Survey

Advance letters are being used increasingly to combat the declinein response rates, but their effectiveness depends partly ontheir being read. In the face of the post-9/11 anthrax mailingscare, the Office for Survey Research (OSR) converted to usingadvance postcards rather than advance letters. Subsequently,the OSR conducted an experiment to test the effectiveness ofsending letters, postcards, or nothing to directory-listed numbersselected in a large random digit dial (RDD) telephone survey.In terms of response rates, the results clearly indicate thatsending advance letters is more effective than sending postcards,which, in turn, is more effective than sending nothing. A detailedcost-effectiveness evaluation indicates that sending lettersactually saves money, despite the increased costs associatedwith the mailings, compared to sending postcards; sending postcardsis also more cost-effective than not sending anything.     

Advance Letters as a Means of Improving Respondent Cooperation in Random Digit Dial Studies: A Multistate Experiment

Advance letters can potentially reduce the degree of nonresponsein random digit dial (RDD) surveys; however, they can also havea heterogeneous impact on subgroups, disproportionately raisingparticipation rates among certain segments of the populationand thereby having a detrimental effect on nonresponse bias.This is, in part, because advance letters can only be used inRDD surveys with the subset of respondents for whom an addresscan be identified. It may also be related to who in a householdsees the letter. We assess whether the use of advance letterscan improve the level of participation in the Behavioral RiskFactor Surveillance System (BRFSS) without introducing otherpotential data biases. The data reported here corroborate previousfindings, in terms of the positive impact that advance letterscan have on overall response rates (approximately a 6-percentage-pointgain). Moreover, the advance letters were cost-efficient inthat the cost of obtaining a fixed number of completed surveysusing advance letters was lower than the cost without letters.However, the positive impact of the advance letter on reducingnonresponse may have been offset to some extent in that theadvance letter may have biased the sample of those who completedthe interview toward older, white respondents and those of highersocioeconomic status, and away from younger, nonwhite individualsand persons with lower education and income levels. This lattergroup was underrepresented in the final sample and also lesslike to remember having received a letter.     

THE EFFECT OF MONETARY INCENTIVES AND FOLLOW-UP MAILINGS ON THE RESPONSE RATE AND RESPONSE QUALITY IN MAIL SURVEYS

The joint and comparative effects of the use of monetary incentivesand follow-up mailings were examined in a mail survey of suburbanWashington, DC cable television subscribers. Four experimentalgroups received monetary incentives enclosed with the firstmailing only ($0.25, $0.50, $1.00, or $2.00) and three follow-upmailings. These groups were compared with each other and againsta control group that did not receive an incentive. The resultsindicated that the response rate from the first mailing increasedsignificantly as the incentive amount increased from zero to$0.25, and from $0.25 to $1.00. Four mailings without an incentiveproduced a higher response rate than a single mailing with anincentive, but a combination of follow-up mailings and a $1.00or $2.00 incentive produced a significantly higher responserate than an equivalent number of mailings without an incentive. There was some evidence of intertreatment response bias. Largermonetary incentives tended to produce: (1) a greater degreeof effort expended in completing the questionnaires, as measuredby the number of short answers and comments provided, and thenumber of words written, and (2) comments that were more favorabletoward the survey sponsor.     

Gender attitudes in the Arab region – The role of framing and priming effects

Most evidence on survey response effects is based in the Western world. We use data from two randomized experiments built into a nation-wide representative household survey in Tunisia to analyze the effects of framing and priming on responses to gender attitudes in the Arab context. Our first experiment shows that questions on attitudes towards decision-making power when framed in an equality frame reduce responses in favor of gender inequality. In our second experiment we find that responses to attitudes towards domestic violence are susceptible to an audio primer. Oral statistical information about the incidence of domestic violence in Tunisia increases disapproval of domestic violence among the male subsample further, but does not affect women. In terms of impact heterogeneity, we find mixed results for treatment interventions interacting with the gender of the interviewer and the interviewer’s perceived religiosity.     

Moral hazard in marriage: the use of domestic labor as an incentive device

This paper argues that some women in developing countries use domestic labor as a tool to incentivize husbands. A theoretical model is derived based on the traditions of rural Malawi, where men often supplement farm income with wage labor. As wage labor is not observed by the wife, this creates moral hazard: husbands may not make enough effort to bring home wages. The model predicts that women overcome this by using domestic labor as an incentive device: they increase their domestic labor and reduce their leisure in response to good consumption outcomes, but only if they cannot rely on divorce threat as an alternative source of incentives. This prediction is confirmed using survey data from Malawi. Identification is based on the fact that Malawi’s kinship traditions exogenously determine women’s accessibility to divorce. Where divorce is not an option, women make inefficient labor choices in order to provide incentives.     

TMR: An Automated System for Managing Revenue

Abstract

The Total Medical Record (TMR) system used by the Division of University Health Services at Duke University is an integrated medical and management information system. One module supports the financial needs of the division. To facilitate managerial performance, TMR automatically prints insurance claims, refund statements, revenue summary data, service utilization, itemized visit statements, monthly patient bills, dunning letters, and has a host of unique features that aid management with controlling the private fees generated by the University Health Services. In addition, the servicing of prepaid programs is accommodated, and the splitting of charges between prepaid and fee-for-service programs occurs automatically.     

The influence of interviewer characteristics on support for democracy and political engagement in Sub-Saharan Africa*

This study investigates how an interviewer’s characteristics affect how respondents answer survey questions about democracy and political engagement. I analyze data from the 2008 Afrobarometer surveys, in which 810 interviewers surveyed 27,713 respondents across 20 countries in sub-Saharan Africa. Using these data, I study how interviewer education, age, and gender affect two outcomes: (1) response distributions to attitudinal and behavioral survey questions and (2) the likelihood of respondents saying ‘don’t know’ to a survey question. The analysis also investigates how the respondent’s perception of who sponsored the survey (NGO, private sector, government) affects attitudes. The results show that these interviewer characteristics affect the quality of survey data on political attitudes and behaviors. In the discussion, I consider the implications for research based on public opinion data about democracy and political engagement.     

USING DUAL FRAME DESIGNS TO REDUCE NONRESPONSE IN TELEPHONE SURVEYS

This article reports on the results of a series of experimentsdesigned to improve response rates for telephone surveys. Inthree surveys telephone households were selected using bothstandard random digit dialing (RDD) techniques and lists oftelephone numbers purchased from a commercial firm. In the RDDportions of the samples "cold contact" interviewing methodswere used; in the list frame portions advance letters were mailed,and the listed household name was used in the introduction.Experiments were designed to test the effects on response ratesof the advance letters and use of the listed household nameas a means of establishing rapport. The advance letters increasedresponse rates, but no difference could be attributed to theuse of names. The mixture of RDD and list sampling techniquesis also used to evaluate the effects of relative response rateson substantive findings. The cost consequences of these dualframe designs are assessed along a number of dimensions, andthe cost and error components of these designs are discussed.     

Variation within households in consent to link survey data to administrative records: evidence from the UK Millennium Cohort Study

This study expands our knowledge of consent in linking survey and administrative data by studying respondents’ behaviour when consenting to link their own records and when consenting to link those of their children. It develops and tests a number of hypothesised mechanisms of consent, some of which were not explored in the past. The hypotheses cover: parental pride, privacy concerns, loyalty to the survey, pre-existing relations with the agency holding the data, and interviewer effects. The study uses data from the longitudinal Millennium Cohort Study to analyse the correlates of consent in multiple domains (i.e. linkage of education, health and economic records). The findings show that respondent’s behaviour vary depending on the consent domain and on the person within the household for whom consent is sought. In particular, the cohort member’s cognitive skills and the main respondent’s privacy concerns have differential effects on consent. On the other hand, loyalty to the survey proxied by the longitudinal response history has a significant and strong impact on consent irrespective of the outcome. The findings also show that interviewers account for a large proportion of variations in consent even after controlling for the characteristics of the interviewer’s assignment area. In total, it is possible to conclude that the significant impact of some of the correlates will lead to sample bias which needs to be accounted for when working with linked survey and administrative data.     

Incentives for Increasing Return Rates: Magnitude Levels, Response Bias, amd Format

The use of monetary incentives has been shown to significantlyincrease response rate. However, previous investigations haverarely investigated the effects of incentives greater than $1,compared different magnitudes of incentives, or investigatedresponse bias due to incentives. The current study also investigatedthe utility of an Answer Check. Results suggest that monetaryincentives increase response rate, larger incentives do notnecessarily further increase survey response, incentives donot appear to bias responses, and the Answer Check does notfacilitate response rate.     

Debiasing or rebiasing? Moderating the illusion of delayed incentives

This paper studies corrective strategies for the illusion of delayed incentives (Soman, 1998), the phenomena that money-for-effort transactions that are unattractive in the present appear attractive when they are in the future. This illusion occurs because future effort is discounted more heavily than future monetary outcomes. In this research, we show that this bias of differential discounting can be corrected by asking consumers to perform effort at the time of decision-making. We further outline three explanations for why this intervention attenuates the illusion of delayed incentives, and discuss whether these explanations constitute a debiasing effect or a rebiasing effect (Larrick, 2004). We report the results of two laboratory experiments and discuss theoretical and practical implications of our findings.     

The Consequences ofLarge Monetary Incentivesin Mail Surveys of Elites

Research concerning incentives and responses to mail questionnairesusually deals with the effects on response rates of paymentsof less than a dollar and with mass respondents. This articlereports the results of an international mail survey of eliterespondents using incentives ranging from 0 to 50 dollars. Thestudy found that larger incentives have a slight positive effecton the response rate and a substantial effect on response quality.The implications of these findings for cost-effective researchare discussed.     

Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes

Traditional statistical analyses of interviewer effects on surveydata do not examine whether these effects change over a fieldperiod. However, the nature of the survey interview is dynamic.Interviewers’ behaviors and perceptions may evolve asthey gain experience, thus potentially affecting data quality.This paper looks at how interview length and interviewer evaluationsof respondents change over interviewers’ workloads. Multilevelmodels with random interviewer effects are used to account forthe clustering of cases within interviewers and individual interviewercharacteristics in the 1984, 1988, and 2000 National ElectionStudies. The 1984 and 1988 NES released sample in four replicates,minimizing the confound between order in an interviewers’workload and sample composition. We find that over the courseof the studies, both measures change significantly. Interviewerprior survey experience also was significantly negatively relatedto the length of the interview. These findings have implicationsfor interviewer training prior to and during studies, as wellas suggesting future research to reveal why these behaviorsand perceptions change.     

Helpless in Finance: The Cost of Helping Effort Among Bank Employees

Theory suggests that individual performance pay increases effort but may reduce the incentive to help co-workers. In an original survey of finance industry employees subject to individual performance pay, we demonstrate that those workers who report they do not help co-workers earn significantly more. This result is particularly strong for those workers with the strongest individual performance pay incentives. Moreover, when those workers report that their coworkers help them, they also earn significantly more. These dual results are consistent with a strong incentive to free-ride on the helping effort of others in the face of individual performance pay.

Experiments in Producing Nonresponse Bias

While nonresponse rates in household surveys are increasingin most industrialized nations, the increasing rates do notalways produce nonresponse bias in survey estimates. The linkagebetween nonresponse rates and nonresponse bias arises from thepresence of a covariance between response propensity and thesurvey variables of interest. To understand the covariance term,researchers must think about the common influences on responsepropensity and the survey variable. Three variables appear tobe especially relevant in this regard: interest in the surveytopic, reactions to the survey sponsor, and the use of incentives.A set of randomized experiments tests whether those likely tobe interested in the stated survey topic participate at higherrates and whether nonresponse bias on estimates involving variablescentral to the survey topic is affected by this. The experimentsalso test whether incentives disproportionately increase theparticipation of those less interested in the topic. The experimentsshow mixed results in support of these key hypotheses.     

The effect of interviewer evaluation of respondents’ attitudes and understanding on future survey participation in a panel study

This study explored whether the interviewers’ judgments of respondents’ performances in an initial survey can predict survey participation for a second survey among the same group of respondents in a panel study. Specifically, this study examined two interviewer variables: the interviewers’ judgments of (1) the respondents’ understanding of survey questions and (2) the respondents’ attitudes toward the survey. The analyses revealed that respondents who were rated to have an excellent understanding of the survey questions and friendly attitudes were more likely to respond to the second interview request than those with a poor understanding and unfriendly attitudes, after controlling for the characteristics of the respondents. Of these two, the judgment of the respondents’ attitudes was a stronger predictor. However, interviewer’s judgments are not correlated with the primary survey outcome or the data quality. This suggests limited value of the interviewer observation.