The Impact of Question and Respondent Characteristics on Comprehension and Mapping Difficulties

Behavior coding is one technique researchers use to detect problemsin survey questions, but it has been primarily explored as apractical tool rather than a source of insight into the theoreticalunderstanding of the cognitive processes by which respondentsanswer survey questions. The latter is the focus of the currentinvestigation. Using data from a large study in which face-to-faceinterviews were taped and extensive behavior coding was done,we tested whether sets of respondent behavior codes could beused to distinguish respondent difficulties with comprehensionof the question from difficulties associated with mapping ajudgment onto the response format provided, and whether characteristicsof the survey questions and respondents could be used to predictwhen and for whom such difficulties would occur. Sets of behaviorcodes were identified that reflected comprehension and mappingdifficulties, and these two types of difficulties were associatedwith different question and respondent characteristics. Thisevidence suggests that behavior coding shows promise as a toolfor researchers studying the cognitive processes involved inanswering survey questions.