UNDERSTANDING MAIL SURVEY RESPONSE BEHAVIOR A META-ANALYSIS

A meta-analysis of prior studies of techniques designed to inducemail survey response rates was conducted. Research encompassing184 effects (study outcomes) in 115 studies (articles) for 17predictors of response rate was examined. The average effectsize across all manipulations was r=.065, indicating an averageincrease of about 6.5 percent in response rates for manipulations.Effect sizes for specific predictors and two potential moderatorsof effects were examined. Results indicated that repeated contactsin the form of preliminary notification and follow-ups, appeals,inclusion of a return envelope, postage, and monetary incentives,were effective in increasing survey response rates. Significanteffect sizes for the predictors ranged from an increase in responseof 2 percent to 31 percent. Implications of the results forthe conduct of mail surveys and future research on mail surveyresponse behavior are discussed.     

LARGE MONETARY INCENTIVES AND THEIR EFFECT ON MAIL SURVEY RESPONSE RATES

The effect of various large monetary incentives was examinedin a mail survey of owners of small construction subcontractingcompanies. The sample was randomly divided into seven groups,each of which received either a different monetary amount enclosedwith the first mailing ($1 cash, $5 cash, $5 check, $10 check,$20 check, or $40 check) or an offer of a $50 check to be sentonce the questionnaire was returned. These groups were comparedagainst each other and against a control group that did notreceive an incentive. Three follow-up mailings were sent tothe nonrespondents in each group. The results indicate that$1 significantly increased the response rate over the no incentivecontrol condition regardless of the number of mailings. Theresponse rate also increased significantly as the incentiveamount increased from $1 to $5 and from $5 to $20. There wasa nonsignificant difference in response rate between the $5check and $5 cash groups. The promise of $50 did not resultin a significantly higher response rate than the control group.Discussion includes a cost-benefit analysis of the incentives,as well as a section about the increased "favorability" of respondentstoward the survey sponsor within the large incentive groups.     

THE EFFECT OF MONETARY INCENTIVES AND FOLLOW-UP MAILINGS ON THE RESPONSE RATE AND RESPONSE QUALITY IN MAIL SURVEYS

The joint and comparative effects of the use of monetary incentivesand follow-up mailings were examined in a mail survey of suburbanWashington, DC cable television subscribers. Four experimentalgroups received monetary incentives enclosed with the firstmailing only ($0.25, $0.50, $1.00, or $2.00) and three follow-upmailings. These groups were compared with each other and againsta control group that did not receive an incentive. The resultsindicated that the response rate from the first mailing increasedsignificantly as the incentive amount increased from zero to$0.25, and from $0.25 to $1.00. Four mailings without an incentiveproduced a higher response rate than a single mailing with anincentive, but a combination of follow-up mailings and a $1.00or $2.00 incentive produced a significantly higher responserate than an equivalent number of mailings without an incentive. There was some evidence of intertreatment response bias. Largermonetary incentives tended to produce: (1) a greater degreeof effort expended in completing the questionnaires, as measuredby the number of short answers and comments provided, and thenumber of words written, and (2) comments that were more favorabletoward the survey sponsor.     

RESPONSE RATES AND QUALITY OF DATA: A RE-EXAMINATION OF THE MAIL QUESTIONNAIRE*

Traditionnellement, dans les sciences sociales on a été préoccupé par les techniques destinées à accoître les taux de réponse aux questionnaires distribués par la poste. Ce n'est que rarement qu'on a essayé de déterminer si des taux de réponse élevés avait une relation avec la qualité des données obtenues. Si on considère les taux de réponse d'envinon 60 pour cent obtenus dans les enquêtes faites par entrevues par les centres de recherche commerciaux, on doit réexaminer le problème de l'utilisation des questionnaires en voyés par la poste comme une alternative, même si on ne prend pas en considération la grande différence de coût par sujet entre les entrevues et les questionnaires envoyés par la poste. Dans la recherche discutée ici, le même questionnaire a été envoyé par la poste, dans des conditions différentes, à trois échantillons tirés au hasard. Le taux de réponse pour les trois Cchantillons variaient de 33 à 48 pour cent, mais les données obtenues dans chaque échantillon étaient pratiquement équivalentes. Pour controler ce resultat, un questionnaire semblable a été envoyéà un échantillon tiré d'une autre population. Le taux de réponse à ce questionnaire était juste inférieur à 90 pour cent, mais les données obtenues des premiers et des derniers sujets étaient généralement équivalentes. Traditionally, social scientists have been preoccupied with techniques for increasing response rates to mail questionnaires. Only rarely have there been attempts to ascertain whether high response rates are related to the quality of data obtained. Given current completion rates of about 60 per cent for interviews by commercial survey research centres, the issue of using mail questionnaires as an alternative data-gathering technique requires re-examination, even apart from any consideration of the wide differential between interviews and mail questionnaires in cost per respondent. In the current research, the same questionnaire was mailed under varying conditions to three independent random samples. The response rates for the three samples varied from 33 to 48 percent, but the data obtained from each sample were basically equivalent. As a further check on this finding, an overlapping questionnaire was sent to a sample from another population. The response rate to this questionnaire was just under 90 per cent, but the data from earlier and later respondents were generally equivalent.     

EFFECTS OF QUESTIONNAIRE LENGTH, RESPONDENT-FRIENDLY DESIGN, AND A DIFFICULT QUESTION ON RESPONSE RATES FOR OCCUPANT-ADDRESSED CENSUS MAIL SURVEYS

An experimental study of alternatives to the current U.S. decennialcensus questionnaire shows that shortening the questionnaireand respondent-friendly questionnaire design improve response,whereas asking a potentially difficult and/or objectionablequestion, that is, social security number, lowers response.This national study of 17,000 household addresses also demonstratesthat relatively high mail survey response can be achieved withoutaddressing correspondence to individual names of residents.     

EFFECTS OF A PREPAID NONMONETARY INCENTIVE ON RESPONSE RATES AND RESPONSE QUALITY IN A FACE-TO-FACE SURVEY

We conducted a randomized experiment on a face-to-face interviewsurvey in order to test the effects on response rates of a prepaidnonmonetary incentive. Results showed a statistically significantincrease in response rates, mostly through reduction in refusalrates, in the half sample that received the incentive (a gift-typeballpoint pen) as compared with a no incentive control group.The effect appears to be due to greater cooperation from incentiverecipients at the initial visit by an interviewer. Unexpectedly,the incentive group also showed a significantly higher rateof sample ineligibility, possibly due to easier identificationof vacant residences or nonexistent addresses. In addition,evidence suggests greater response completeness among respondingincentive recipients early in the interview, with no evidenceof increased measurement error due to the incentive.     

PROMISED CONTRIBUTIONS TO CHARITY AND MAIL SURVEY RESPONSES: REPLICATION WITH EXTENSION

The efficacy of four types of incentives for stimulating mailsurvey response rates was examined against a no-incentive controlgroup. Two of the incentives were enclosed personal cash rewards(25ø and $1), and two were promised rewards. Of the latter,one was an impersonal reward, the promise of a $1 contributionto a charity of the respondent's choice, and one a personalreward, the opportunity to win a $200 cash prize. As hypothesized,the personal cash rewards generated response rates that weresignificantly higher than that of the control group. Also ashypothesized, the charity-incentive group produced a responserate that was not significantly different from the control group.Discontinuation of the use of charity incentives should be seriouslyconsidered by future researchers. In contrast, the cash prizeincentive yielded a significantly greater response than thecontrol group, as expected, and was on par with the 25øgroup. Additional investigation of the use of cash prize incentivesis recommended.     

PHYSICIAN RESPONSE TO A MAILED SURVEY AN EXPERIMENT IN TIMING OF PAYMENT

This article reports the results of an experiment in obtainingphysician response to a mailed questionnaire. Each physicianwas eligible for a payment of $20. A randomly selected halfreceived the payment with their initial questionnaire and coverletter; half were told they would receive their payment afterthey completed and returned the questionnaire. The same mailand telephone followup procedures were used for both groups.Overall, prepayment had significant positive effects on responserates. This paper examines these effects in terms of responserates for various specialties, field efficiencies, cost, andrepresentativeness of the sample.     

MAIL SURVEYS FOR ELECTION FORECASTING?: AN EVALUATION OF THE COLUMBUS DISPATCH POLL

Because of slow turnaround time and typically low response rates,mail surveys have generally been considered of little valuein election forecasting. However, statewide mail surveys conductedby the Columbus Dispatch newspaper since 1980 have made remarkablyaccurate forecasts of Ohio election outcomes. In comparisonto statewide surveys by two other organizations employing conventionaltelephone interview methods, the mail surveys were consistentlymore accurate and were generally less susceptible to sourcesof inaccuracy such as high rolloff and low publicity. The mailsurvey's advantage is attributable at least in part to largersample sizes, sampling and response procedures that yieldedmore representative samples of voters, lack of the need to allocateundecided respondents, and superior questionnaire design. Thesefindings suggest that mail surveys not only may be viable alternativesto telephone surveys but may actually be superior to them undersome conditions. Further-more, these results demonstrate thatsurveys with low response rates are not necessarily low in validity.     

SURVEY QUESTIONS ABOUT THE FREQUENCY OF VAGUELY DEFINED EVENTS: THE EFFECTS OF RESPONSE ALTERNATIVES

Experimental research has shown that the choice of responsealternatives can influence responses to questions about thefrequency of vaguely defined target events (e.g., feeling annoyed);the set of response alternatives is treated as information thatcontributes to the interpretation of the question. In a seriesof split ballot experiments we investigate whether such effectsoccur in a large-scale survey context for sets of response alternativesthat might be used interchangeably by survey researchers. Thepredicted response shifts were found in our field experiments.Those presented with response alternatives discriminating atlow frequencies reported fewer of the target episodes than thosepresented with higher-frequency response alternatives. However,the size of the observed shifts varied from zero to 13 percentand depended on a number of characteristics of the survey, forexample, the presence of "priming" questions and the orientationof the response scales. Response alternatives are not neutraland therefore must be carefully considered when constructinga survey question.     

RESPONSE LATENCY VERSUS CERTAINTY AS INDEXES OF THE STRENGTH OF VOTING INTENTIONS IN A CATI SURVEY

This CATI study compares two measures of attitude strength,accessibility and certainty, for the purpose of predicting discrepanciesbetween voting intentions and voting behavior. Accessibilitywas indexed by response latency to a voting intention questionand certainty by a question about the finality of the votingintention. Logistic regression showed response latency to bea better predictor of discrepancies between voting intentionsand voting behavior than certainty. The theoretical implicationsof this finding as well as the practical implications of a methodfor measuring response latencies in CATI election studies arediscussed.     

"DON'T KNOW" BOXES IN FACTUAL QUESTIONS IN A MAIL QUESTIONNAIRE: EFFECTS ON LEVEL AND QUALITY OF RESPONSE

This article summarizes the findings of an experiment to evaluatethe effect of "Don't Know" (DK) boxes in a mail questionnaireconsisting of factual questions. In a split-panel test, halfof the respondents received a questionnaire with DK boxes andthe other half received a questionnaire without DK boxes. Theoverall response rates were the same for the two versions. Thequestionnaire version without DK boxes obtained an appreciablyhigher rate of substantive responses for many items. However,there were few large substantive changes in response distributionsbetween the two versions, and there was no appreciable differencein the response error rates.     

THE EFFECTIVENESS OF FAMILY THERAPIES FOR SELECTED BEHAVIORAL DISORDERS OF CHILDHOOD

This article reviews the scientific evidence for the effectiveness of family-based aproaches in the treatment of selected childhood behavioral disorders. Although limitations certainly exist, family interventions have consitently improved child and, in some cases, parent functioning in families with children presenting with conduct disorder (CD) and autism. Parents and other family members also directly benefit from child-focused interventions, gaining in knowledge, child management skills, and attitudinal improvements. Longh-term follow-ups indicate that CD and autistic children achieved lasting gains. Similarly, the research on attention-deficit/hyperactivity disorder (ADHD) indicates that parent training improves child noncompliance and aggression yet does not consistently affect core symptoms of ADHD. There is no evidence that adding short-term family interventions improves ADHD child functioning beyond improvements from the use of psychostimulant medications. Some tentative support for family involvement in the treatment of childhood anxieties and fears is reviewed, but clear conclusions await future investigations. Finally, several methodological limitations and needed areas of research are discussed.     

RESPONSE LATENCY AS A SIGNAL TO QUESTION PROBLEMS IN SURVEY RESEARCH

This research explores the potential utility of response latencyas an index of question problems in survey research. The timerespondents took to answer three types of bad questions wascompared to the time they took to answer the repaired versionsof the questions. Questions containing a superfluous negativeand double-barreled questions took longer to answer than nearlyidentical questions without these problems. Repaired versionsof questions soliciting frequency estimates, however, took longerto answer than their problematic counter parts. The resultsare discussed in the context of a model of question answering,and their implications for survey methodology are explored.     

OPTIMAL CALL SCHEDULING FOR A TELEPHONE SURVEY

Contacting a suitable respondent at home and securing the interviewis a significant component of the typical household telephonesurvey. An optimal calling schedule minimizes the number ofcallbacks required and thus conserves resources. This articlepresents an analysis of time-of-day and day-of-week effectson the probabilities of obtaining an answered outcome and aninterviewed outcome in a large national telephone survey ofadult males. The data presented indicate that, at least forthe survey population included in this study, the chances ofobtaining an answer and conducting an interview on the firstcall are much better on weekday evenings and on weekends thanthey are during weekday daytime hours. Moreover, there is someevidence to indicate that this finding also applies to secondcalls made to first-call no-answers, at least with regard tocontacting efforts. Time-of-day and/or day-of-week effects onfirst call outcomes appear to be generally consistent with respectto both contacting and interviewing, although Sunday has a decidedlyhigher interview response rate for answered calls than do theother calling periods analyzed.     

ENRICHMENT OUTCOME RESEARCH:A META-ANALYSIS OF PREMARITAL, MARITAL AND FAMILY INTERVENTIONS

The findings from a meta-analysis of 85 studies of premarital, marital, and family enrichment, representing 3,886 couples or families are presented. Metaanalysis is a method for statistically aggregating and evaluating empirical findings. Findings from the study are discussed in terms of overall enrichment effectiveness as well as salient program, subject, design, measurement, and analysis characteristics. The enrichment studies yielded an average effect size of .44, which indicates that the average person who participates in enrichment is better off following intervention than 67% of those who do not. The most powerful factors related to outcome were measurement variables rather than those related to program content, structure, leadership, or participant characteristics. Implications of these findings are discussed.     

THE ECONOMICS OF JOB SEARCH: A SURVEY

Part I of this survey presents a comprehensive and inte- grated analysis of job search. The labor market is characterized by incomplete information and search is conducted in an optimal manner. A variety of job search models are studied in this microeconomic setting. The paper contains a number of new results. Part 11 of the survey [to appear in the September 1976 issue of Economic Inquiry] addresses the empirical and policy implications of search theory.     

A COMPARISON OF THE VIABILITY OF MAIL SURVEYS IN JAPAN AND THE UNITED STATES

A comparison of response rates and patterns to a mail surveyconducted simultaneously on random samples drawn from telephonelistings in Seattle, Washington, and Kobe, Japan, is presented.When undeliverable surveys (those for which the intended respondenthad moved or passed away) are subtracted from the original samples,a comparison of the differences in the response rate percentagesfor the two countries is found to be statistically insignificant.Adherence to the theoretical foundation of the Total DesignMethod is claimed to be the principal reason for success inachieving a higher response rate than predicted by Japanesescholars to the mail survey in Japan. These results show thatthere is potential for adapting survey research techniques developedin the United States to non-Occidental settings.     

THE EFFICACY AND EFFECTIVENESS OF MARITAL AND FAMILY THERAPY: A PERSPECTIVE FROM META-ANALYSIS

This article reviews the major findings from a multiproject meta-analysis of the effects of marital and family therapy (MFT). Across 163 randomized trials, MFT demonstrates moderate, statistically significant, and often clinically significant effects. No orientation is yet demonstrably superior to any other, nor is MFT superior to individual therapy. Cost effectiveness information is scant in these 163 studies, but supportive. Randomized experiments yield very different answers from nonrandomized experimental studies of the effects of MFT, calling into question whether we should mix the two in reviews. We have also found several new differences in the ways that marital therapy (MT) and family therapy (FT) studies are conducted, making them harder to compare. Finally, important questions still exist about whether any psychotherapy, including MFT, yet has sufficient information about how well research generalizes to everyday clinical practice.