Response Order Effects in Dichotomous Categorical Questions Presented Orally: The Impact of Question and Respondent Attributes

Using data from 548 experiments in telephone surveys conductedby the Gallup Organization, we explored how attributes of questionsand respondents moderate response order effects in dichotomouscategorical questions. These effects were predominantly recencyeffects and occurred most in questions that were more difficultto comprehend (especially among respondents with the least education),with response choices that were more difficult to comprehend(because they were complete sentences instead of words or phrasesand because they were not mutually exclusive), and that wereasked after many prior questions. Recency effects were alsomore common in questions that explicitly or implicitly encouragedrespondents to wait until they had heard all the answer choicesbefore formulating a judgment than in questions that inducedrespondents to begin formulating a judgment before all the answerchoices had been read (especially among the least educated respondents).A study of interviewer behavior revealed patterns of pausingbetween and within sentences that help to explain why some typesof questions are especially prone to recency effects and othersare not.