Effects of Filter Questions in Public Opinion Surveys

Extending previous work, the authors find that the wording ofa filter question can make a significant difference in the percentageof "don't know" (DK) responses elicited by an item, especiallywith topics that are more abstract or less familiar to surveyrespondents. They also find, however, that the content of anitem can have a substantial, independent effect on DK or "noopinion" responses, regardless of how the filter question isworded. In general, it appears that the less familiar the issueor topic, the greater the increase in DK responses producedby adding a filter. Even more important, the analysis showsthat filtering can in some instances dramatically affect theconclusions a pollster would draw about the distribution ofpublic opinion on an issue. Indeed, such effects may occur moreoften than has previously been suspected, though the circumstancesunder which they emerge remain elusive. The authors suggestthat such effects may become amenable to analysis by probingrespondents about "what they had in mind" as they answered thequestion.     

Effects of Question Wording and Format on Political Attitude Consistency

Numerous studies in the past decade have suggested that theAmerican voter has become increasingly sophisticated, ideologically,and that this is due largely to the separate or combined impactof the growing educational attainments of the U.S. public andthe greater salience of politics in the 1960s and 1970s. Thisstudy, however, hypothesizes that much of the change might bedue to a simple methodological artifact—changes in questionwording and format—and reports data from a recent nationalexperiment which strongly supports such an interpretation. Thegeneralizations derived from The American Voter may, therefore,not be quite as "time-bound" as the current literature wouldlead us to believe.     

Mass and Elite Foreign Policy Opinions

Since the Vietnam War, public preferences on foreign policyissues have played a greater role in the American politicalprocess. Yet comparatively little is known about the mannerin which attitudes on international affairs are formed and howthey are related to other political cognitions. Using data fromtwo large studies of foreign policy opinion conducted by theChicago Council on Foreign Relations in 1974 and 1978, thisresearch examines the question of how mass and elite opiniondiffers in a number of foreign policy domains and looks at whetherthese distinctions have changed across years. In general, thereare patterns of difference in the foreign policy beliefs ofthe elites and the mass public, the most striking being in their"internationalist" view of the U.S. role in world affairs, withsmaller distinctions in the domains of human rights, detente,world problems, Americanism, and international organizations.     

THE ANSWERING MACHINE GENERATION: WHO ARE THEY AND WHAT PROBLEM DO THEY POSE FOR SURVEY RESEARCH?

Increased incidence of telephone answering machines and theuse of such devices to screen calls pose a potential threatto the representativeness of samples in telephone surveys. Usingdata from nine statewide surveys, this analysis examines theextent to which answering machines are used to screen callsand the demographic characteristics associated with answeringmachine use and call screening. Results show that at most twoto three percent of households use answering machines to screencalls, and that such screening is more likely to take placein households with higher family incomes, outside rural areas,and which include individuals who are younger and have higherlevels of education. While call screening does not presentlythreaten the representativeness of samples in telephone surveys,the increased incidence of answering machines together withthe increased percentage of households indicating these devicesare sometimes used to screen calls demonstrate that the potentialbias from this source is growing.     

The Importance of Replicating a Failure to Replicate: Order Effects on Abortion Items

A failure to replicate a previously demonstrated order effecton responses to questions about abortion led to a second experimentwhich was designed to test the hypothesis that the failure wasdue to a change in the question context which had immediatelypreceded the abortion items in the original experiments by Schuman,et al. Unlike the results of the first experiment, the resultsof this second experiment showed that the order effect reportedby Schuman and his associates does replicate. In addition, thedata indicate that context may condition the magnitude of thisorder effect. The authors also suggest that the effect may bedue to the implicit contrast between the reasons given for abortionby the items when they are asked in one order rather than theother.     

Pseudo-Opinions on Public Affairs

This article reports on the often suspected but rarely researchedtendency of survey respondents to give opinions on topics towhich they have given little or no thought. The findings, basedon a question about a fictitious public affairs issue, do showthat the magnitude of the problem is substantial. But the dataalso demonstrate that this phenomenon does not represent simplerandom error, reflecting instead basic social-psychologicaldispositions which can be elicited, unwittingly, in the contextof the interview.     

What Must My Interest in Politics Be If I Just Told You "I Don't Know"?

How much people think they follow what's going on in governmentand public affairs depends upon the context in which they areasked the question. If asked immediately after difficult questionsabout what they know of their congressman's record, they aremuch less likely to think they pay attention to public affairsthan if they are asked, first, how interested they are in suchmatters. More important, data from two independent experimentsshow that this context effect cannot be eliminated, or significantlyreduced, by interposing a buffer of questions on unrelated topicsbetween the items that are known to affect one another. Theauthors discuss the psychological significance of these findingsand their implications for survey research.     

Effects of Presenting One Versus Two Sides of an Issue in Survey Questions

Data from a split-ballot experiment show that offering respondentsan alternative position on an issue not only affects the marginals,it also influences whether respondents will give an opinionat all. Furthermore, the study demonstrates that these formeffects can occur despite the use of filter questions whichtheoretically screen out those who tend to be most susceptibleto such effects: the less educated or uninformed. The analysisdoes provide evidence, however, that less educated respondentsare indeed more affected by differences in question format andthat they are much more likely to "acquiesce" to one-sided agree/disagreeforms. In discussing the results the authors develop an information-processingmodel of question form effects and a methodological strategyto generate further research on a much-needed theory of thesurvey instrument.