"DON'T KNOW" BOXES IN FACTUAL QUESTIONS IN A MAIL QUESTIONNAIRE: EFFECTS ON LEVEL AND QUALITY OF RESPONSE

This article summarizes the findings of an experiment to evaluatethe effect of "Don't Know" (DK) boxes in a mail questionnaireconsisting of factual questions. In a split-panel test, halfof the respondents received a questionnaire with DK boxes andthe other half received a questionnaire without DK boxes. Theoverall response rates were the same for the two versions. Thequestionnaire version without DK boxes obtained an appreciablyhigher rate of substantive responses for many items. However,there were few large substantive changes in response distributionsbetween the two versions, and there was no appreciable differencein the response error rates.     

"Don't Know": Item Ambiguity or Respondent Uncertainty?

What to do with "Don't Know" responses in survey data is dependentupon a prior understanding of their causes. A proximity analysisof the "Don't Know" responses alone yields the same scale asdoes a scalogram analysis of the definite responses alone indicatingresponse uncertainty as one source of the response. The qualityof the scales is improved by eliminating ambiguous items, indicatingstimulus ambiguity as another source. The data used are responsesto items on attitude toward abortion obtained in an island-widesurvey of Taiwan.     

DO OPEN-ENDED QUESTIONS MEASURE "SALIENT" ISSUES?

Closed-ended questions dominate most interview schedules. Yetthe almost exclusive use of this form did not arise becauseopen-ended questions, its major competitor, proved to be weakindicators of public opinion. Instead, responses from open-endedquestions proved more difficult and expensive to code and analyzethan those from closed-ended questions. Although such practicalconcerns are important, the real task of survey researchersis to measure public opinion accurately. Using an experimentaldesign, this article tests whether open-ended questions measurethe important concerns of respondents—one of the long-claimedadvantages of this format. The results, on balance, show thatopen-ended comments reflect such concerns, suggesting that pollstersmay want to include more of these questions in their surveysof public opinion.     

The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions

This article reports results from 14 experimental comparisonsdesigned to test 7 hypotheses about the effects of two typesof nonverbal languages (symbols and graphics) on responses toself-administered questionnaires. The experiments were includedin a survey of 1,042 university students. Significant differenceswere observed for most comparisons, providing support for allseven hypotheses. These results support the view that respondents’answers to questions in self-administered surveys are influencedby more than words. Thus, the visual presentation of questionsmust be taken into consideration when designing such surveysand, especially, when comparing results across surveys in whichthe visual presentation of questions is varied.     

TAXES AND EXPENDITURES IN THE U.S.: PUBLIC OPINION SURVEYS AND INCIDENCE ANALYSIS COMPARED

The usefulness of surveys for eliciting fiscal opinions of voters is investigated. Using data from the U.S. Advisory Commission in Intergovernmental Relations annual surveys of attitude toward government and taxes, the survey responses are related to economic and political factors and the survey results compared to those predicted by incidence analysis. The results from this analysis and a review of previous studies of this type suggest that there is often no consistent economic foundation for answers to fiscal survey questions. It also appears that some standard survey questions are too general to produce precise estimates of the factors influencing responses.     

Question Understanding Aid (QUAID): A Web Facility that Tests Question Comprehensibility

When respondents do not understand the meaning of a survey question,they will not supply valid and reliable answers. Survey methodologistsshould therefore benefit from computer tools and other analyticalschemes that help them identify problems with questions withrespect to comprehension difficulty. We developed a Web facilitycalled Question Understanding Aid (QUAID; www.psyc.memphis.edu/quaid.html)that assists survey methodologists in identifying problems withthe wording, syntax, and semantics of questions on questionnaires.The survey methodologist enters the question into the Web facility,along with any context information and answer alternatives thataccompany the question. QUAID quickly returns a list of potentialproblems with question comprehension, including unfamiliar technicalterms, vague or imprecise relative terms, vague or ambiguousnoun phrases, complex syntax, and working memory overload. Thisarticle describes QUAID and some empirical studies that haveassessed the validity and utility of QUAID’s critiquesof questions. The output of QUAID was compared with the judgmentsof experts in language, discourse, and cognition during thedevelopment of the tool. In one evaluation, expert survey methodologistscritiqued and revised problematic questions, whereas in a secondevaluation survey methodologists evaluated the quality of originalproblematic questions, questions revised with the assistanceof QUAID, and questions revised without QUAID. In a third evaluation,eye-tracking data were collected while respondents read questionson a computer screen and answered questions aloud. Respondentshad a tendency to give up processing difficult questions tooearly (called an early exit), which potentially threatens thevalidity of the respondents’ answers. Survey methodologistsare encouraged to use QUAID and further evaluate its validityand utility.     

IMPROVEMENT OF THE QUALITY OF RESPONSES TO FACTUAL SURVEY QUESTIONS BY INTERVIEWER TRAINING

In this study we analyze the effects of interviewer trainingon the quality of responses. Data from a field experiment reveala number of significant differences between trained and untrainedinterviewers in terms of nonresponse and the amount of informationobtained. For some questions, an interaction effect occurredbetween interviewer training, the use of tape recorders, andthe kind of responses. The effects of training appear to bedependent on the structure of the questions. Since such effectsoccur primarily with questions that assume a great deal of intervieweractivity, it is suggested that they are indeed due to the applicationof the techniques acquired by training, namely giving instructions,probing, and feedback. A brief analysis of the interviewer-respondentinteraction supports this interpretation.     

A theory of recovery

1. Mental illness generally involves a loss of self--the meaning and purpose that once defined life are gone. 2. People respond to loss of self by asking "why." 3. Whether or not people find answers to "why" questions, they need to move on to asking, "How do I proceed?" 4. Action based on answers to "how" questions provides new meaning and purpose.     

Trust in surveys and games – A methodological contribution on the influence of money and location

This paper explores methods to study trust. In a variety of settings, answers to survey questions and choices in a trust game are obtained from student sample pools. Some subjects are approached by mail and execute their task at home whereas others participate in classroom experiments. No differences between the results obtained by these methods are observed. Furthermore, one additional group plays the trust game with purely hypothetical payments, and another receives random lottery payments. This changes trust behavior dramatically, whereas trustworthiness is unaffected. Subjects without any financial incentives exhibit less trust and their trust choices are significantly correlated with survey trust answers. There is no such correlation for the corresponding choices with real payments.     

Grade-induced beliefs about undergraduate generalist social work practice competency

Standardized criterion-referenced achievement testing of undergraduate generalist social work knowledge based on correct answers to specific questions is compared to several norm-referenced measures of student learning, including student self-reports and the instructor-imposed cumulative grade point average (GPA). Two hypotheses are tested, namely, (a) norm-referenced measures systematically overstate content knowledge, and (b) student perception or belief about the ascribed meaning of the GPA inflates self-rated attainments. The implications of additional confirming evidence for the hypotheses are explored with a view to limiting the number of social workers with undergraduate degrees who are permitted to enter the field each year overconfident about their knowledge and practice capabilities.     

Presentation matters: how mode effects in item non-response depend on the presentation of response options

Comparability of data across modes is an important issue in survey research. In this paper we discuss item non-response to attitudinal questions in telephone and web surveys. We present results from a survey experiment conducted in Italy and in Spain that compares different presentations of response options in an online setting with a benchmark telephone survey. In line with earlier studies we find that (A), the share of non substantial answers in the online survey depends on how the response option is presented. Comparing different presentations in an online survey to the standard approach of telephone surveys using propensity score matching, we find that (B), the share and pattern of non substantial answers, is most similar across the two modes in the online survey when the it is captured in an instruction on each screen for the online survey. Our findings are of particular relevance for the design of multiple mode or mix-mode surveys of attitudinal questions using online and telephone modes.     

Husbands’ and wives’ time spent on housework: A comparison of measures

This study compares a series of estimates of the time spent on housework from survey responses and time‐use estimates from the Experience Sampling Method (ESM) obtained from husbands and wives in the Sloan 500 Family Study. These include estimates from husband's and wife's answers to questions about own time and spouse's time on household tasks, and time‐use estimates from the ESM. The three ESM estimates include primary activity only, primary plus secondary activity, and primary and secondary activity plus time spent thinking about household tasks. We find that estimates of hours spent on housework differ substantially and significantly across various measures, as does the absolute size of the gap between hours spent by husbands and wives. Share of housework done by husbands differs somewhat less.     

EDUCATED GUESSES: THE PROCESS OF ANSWERING FACTUAL KNOWLEDGE QUESTIONS IN SURVEYS

Responses to autobiographical questions are known to representmore than simply retrieval of information from memory; inference,cuing, and "availability" all play a role. Using responses toitems in four different surveys, we find that respondent motivationand ability, together with contextual cues, help determine howsurvey respondents answer knowledge questions about the worldaround us. Thus, we extend the domain of factual items for whichthe role of inferential processes is recognized, and we specifymore precisely the kinds of factors that respondents use inanswering such questions. We also find suggestive evidence thatattitudes influence answers to information questions, thus extendingas well the kinds of factors seen as likely to affect reportsabout facts.     

The Effect of Question Framing and Response Options on the Relationship between Racial Attitudes and Beliefs about Genes as Causes of Behavior

Prior research suggests that the attribution of individual and group differences to genetic causes is correlated with prejudiced attitudes toward minority groups. Our study suggests that these findings may be due to the wording of the questions and to the choice of response options. Using a series of vignettes in an online survey, we find a relationship between racial attitudes and genetic attributions when respondents are asked to make causal attributions of differences between racial groups. However, when they are asked to make causal attributions for characteristics shown by individuals, no such relationship is found. The response scale used appears to make less, if any, difference in the results. These findings indicate that the way questions about genetic causation of behavior are framed makes a significant contribution to the answers obtained because it significantly changes the meaning of the questions. We argue that such framing needs to be carefully attended to, not only in posing research questions but also in discourse about genetics more generally.     

Color, Labels, and Interpretive Heuristics for Response Scales

We carried out two experiments to investigate how the shadingof the options in a response scale affected the answers to thesurvey questions. The experiments were embedded in two web surveys,and they varied whether the two ends of the scale were representedby shades of the same or different hues. The experiments alsovaried the numerical labels for the scale points and examinedresponses to both unipolar scales (assessing frequency) andbipolar scales (assessing favorability). We predicted that theuse of different hues would affect how respondents viewed thelow end of the scale, making responses to that end seem moreextreme than when the two ends were shades of the same hue.This hypothesis was based on the notion that respondents usevarious interpretive heuristics in assigning meaning to thevisual features of survey questions. One such cue is visualsimilarity. When two options are similar in appearance, respondentswill see them as conceptually closer than when they are dissimilarin appearance. The results were generally consistent with thisprediction. When the end points of the scale were shaded indifferent hues, the responses tended to shift toward the highend of the scale, as compared to scales in which both ends ofthe scale were shaded in the same hue. Though noticeable, thisshift was less extreme than the similar shift produced whennegative numbers were used to label one end of the scale; moreover,the effect of color was eliminated when each scale point hada verbal label. These findings suggest that respondents havedifficulty using scales and pay attention even to incidentalfeatures of the response scales in interpreting the scale points.     

POWER AND SOLIDARITY IN “INTERRUPTION”: A CRITIQUE OF THE SANTA BARBARA SCHOOL CONCEPTION AND ITS APPLICATION BY ORCUTT AND HARVEY (1985)

The simplistic conception of interruption employed in conversation analysis and survey research reports of male monopoly must be elaborated to include other social meanings in addition to violation of a speaker's rights if it is to account for conversational order or deviance. With no reason to suppose that the narrow, technical sense of interruption and the implacably negative evaluation conversation analysts make about interruption are generally shared throughout the rest of American culture, answers to questions in surveys about interruption cannot be assumed to refer to the category termed interruption in conversation analysis. Dissensus about coding instances of the category are reported and other situated meanings of interruption suggested.     

Qualifications to Generalized Absolutes: "Approval of Hitting" Questions on the GSS

An analysis of "approval of hitting" questions on the GeneralSocial Surveys finds evidence of correlated error. Respondentsfrequently ignore the absolute phrasing of questions. As a result,they often contradict themselves by approving of specific usesof hitting after having rejected any use of such force. Thesecontradictors tend to have lower education and less supportfor punitive responses on other items.     

Professional boundaries. A matter of therapeutic integrity

1. Establishing and maintaining boundaries creates a relational space in which the patient and nurse are able to explore treatment issues within the safety of the therapeutic relationship. 2. The therapeutic relationship in founded on the basis of trust, respect, and appropriate use of power. Understanding the power differential within the nurse patient relationship is critical to the establishment and maintenance of professional boundaries. 3. Therapeutic integrity is rooted in knowledge, experience, and clinical judgment. As such, there are not always clearly demarcated answers to the tough questions posed by the dilemmas in practice. 4. Questions arising from practice challenge nurses to reflect on the meaning of boundaries in their personal and professional lives.     

EFFECTS OF QUESTIONNAIRE DESIGN ON THE QUALITY OF SURVEY DATA

The importance of format, graphic layout, and question routinginstructions on the quality of survey data has been acknowledgedfor self-administered survey instruments, but the significanceof these factors for questionnaires administered by interviewershas not been recognized equally. This paper examines two studiesin which interviewers used different questionnaire designs toadminister the same set of survey questions to randomly selectedadult respondents in housing units that made up area probabilitysamples of the Detroit metropolitan area. The paper presentsempirical evidence of specific effects (questions skipped inerror, unprobed answers) directly attributable to the choiceof questionnaire design. The analysis shows that question nairedesign choices can either help or hurt the quality of data collectedby interviewers. Furthermore, the behaviors of experienced andinexperienced interviewers are affected in similar ways. Inother words, interviewing experience does not compensate forformat deficits in the design of survey instruments.     

Vocal signs of confidence

This study examined whether two paralinguistic variables, vocal loudness and response latency, were associated with confidence in answers to trivia questions. Audience presence and size were manipulated and subjects' assertiveness was measured. Subjects verbally responded to trivia questions by indicating their choice and how confident they were in each answer. Tapes of these responses were later analyzed for latency of response and loudness of speech. As expected, the more confident individuals were in their answers, the faster and louder they responded. Assertive subjects spoke louder. The presence of an audience had no effects on vocal responding. Apparently, the confidence of a speaker can be inferred from the speed and loudness of the speaker's responses.This study was a Master's thesis project conducted by the second author under the supervision of the first author.