Effects of Personal Salutations in E-mail Invitations to Participate in a Web Survey

Effects of personalizing survey invitations on response rateshave been extensively researched in the realm of mail surveys.Commonly, it is found that response rates increase when personalizationis applied. Recently, efforts have been made to investigatewhether these findings extend to the field of Web surveys thatuse e-mail invitations. Using data from a Web experiment conductedamong first-year university students, personalization of e-mailinvitations is shown to have significantly increased the responserate by 7.8 percentage points. From the theory that personalizationdecreases the level of anonymity and perceived privacy, differencesin responses to sensitive questions were expected. However,no effects of personalization on the responses to a questionprobing for respondents’ sexual behavior were found.     

Gender attitudes in the Arab region – The role of framing and priming effects

Most evidence on survey response effects is based in the Western world. We use data from two randomized experiments built into a nation-wide representative household survey in Tunisia to analyze the effects of framing and priming on responses to gender attitudes in the Arab context. Our first experiment shows that questions on attitudes towards decision-making power when framed in an equality frame reduce responses in favor of gender inequality. In our second experiment we find that responses to attitudes towards domestic violence are susceptible to an audio primer. Oral statistical information about the incidence of domestic violence in Tunisia increases disapproval of domestic violence among the male subsample further, but does not affect women. In terms of impact heterogeneity, we find mixed results for treatment interventions interacting with the gender of the interviewer and the interviewer’s perceived religiosity.     

Response Effects in the Electronic Survey

This report examines the electronic survey as a research tool.In an electronic survey, respondents use a text processing programto self-administer a computer-based questionnaire. As more peoplehave access to computers, electronic surveys may become widespread.The electronic survey can reduce processing costs because itautomates the transformation of raw data into computer-readableform. It can combine advantages of interviews (e.g., prompts,complex branching) with those of paper mail surveys (e.g., standardization,anonymity). An important issue is how the electronic surveyaffects the responses of people who use it. We conducted anexperimental sample survey on health attitudes, behaviors, andpersonal traits using two forms of administration: electronicand paper mail. Closed-end responses in the electronic surveywere less socially desirable and tended to be more extreme thanwere responses in the paper survey. Open-ended responses thatcould be edited by respondents were relatively long and disclosing.These findings are consistent with other research on computer-mediatedcommunication, raising general issues about using computersto collect self-report data.     

Context Effects on Survey Responses to Questions About Abortion

The proportion of favorable responses to a general questionabout abortion was significantly smaller in an NORC survey thanwere responses to the same item in an SRC survey. We hypothesizedthat questionnaire context was the main source of the difference–inone survey the general item followed a more specific questionabout abortion–and carried out two split-ballot questionorder experiments that provide strong support for the hypothesis.The context effect seems to arise from the generality of theabortion question; the specific question itself showed no changedue to context. Furthermore, there is some evidence that ambivalencetoward the issue of abortion is a factor in the sensitivityof the general question to context. The context effect in theseexperiments is also notable for producing contrast rather thanconsistency between responses to adjacent questions on the sameissue. Implications of the findings for the study of socialchange are emphasized.     

The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions

This article reports results from 14 experimental comparisonsdesigned to test 7 hypotheses about the effects of two typesof nonverbal languages (symbols and graphics) on responses toself-administered questionnaires. The experiments were includedin a survey of 1,042 university students. Significant differenceswere observed for most comparisons, providing support for allseven hypotheses. These results support the view that respondents’answers to questions in self-administered surveys are influencedby more than words. Thus, the visual presentation of questionsmust be taken into consideration when designing such surveysand, especially, when comparing results across surveys in whichthe visual presentation of questions is varied.     

Comparing Check-All and Forced-Choice Question Formats in Web Surveys

For survey researchers, it is common practice to use the check-allquestion format in Web and mail surveys but to convert to theforced-choice question format in telephone surveys. The assumptionunderlying this practice is that respondents will answer thetwo formats similarly. In this research note we report resultsfrom 16 experimental comparisons in two Web surveys and a papersurvey conducted in 2002 and 2003 that test whether the check-alland forced-choice formats produce similar results. In all 16comparisons, we find that the two question formats do not performsimilarly; respondents endorse more options and take longerto answer in the forced-choice format than in the check-allformat. These findings suggest that the forced-choice questionformat encourages deeper processing of response options and,as such, is preferable to the check-all format, which may encouragea weak satisficing response strategy. Additional analyses showthat neither acquiescence bias nor item nonresponse seem topose substantial problems for use of the forced-choice questionformat in Web surveys.     

Comparison of web and mail surveys in collecting illicit drug use data: a randomized experiment

This randomized experiment examined survey mode effects for self-reporting illicit drug use by comparing prevalence estimates between a Web-based survey and a mail-based survey. A random sample of 7,000 traditional-aged undergraduate students attending a large public university in the United States was selected to participate in the spring of 2001. Students were randomly assigned to self-administer a survey via the Web or U.S. mail. The Web survey produced a significantly higher response rate than the U.S. mail survey. The prevalence estimates of illicit drug use (lifetime and past year) did not differ significantly between the two survey modes. The findings provide preliminary evidence that Web and U.S. mail surveys produce similar results regarding illicit drug use among undergraduate students. Although additional research is needed involving more diverse samples, these findings bode well for using Web surveys in college-based research.     

SURVEY QUESTIONS ABOUT THE FREQUENCY OF VAGUELY DEFINED EVENTS: THE EFFECTS OF RESPONSE ALTERNATIVES

Experimental research has shown that the choice of responsealternatives can influence responses to questions about thefrequency of vaguely defined target events (e.g., feeling annoyed);the set of response alternatives is treated as information thatcontributes to the interpretation of the question. In a seriesof split ballot experiments we investigate whether such effectsoccur in a large-scale survey context for sets of response alternativesthat might be used interchangeably by survey researchers. Thepredicted response shifts were found in our field experiments.Those presented with response alternatives discriminating atlow frequencies reported fewer of the target episodes than thosepresented with higher-frequency response alternatives. However,the size of the observed shifts varied from zero to 13 percentand depended on a number of characteristics of the survey, forexample, the presence of "priming" questions and the orientationof the response scales. Response alternatives are not neutraland therefore must be carefully considered when constructinga survey question.     

Counting the uninsured: a comparison of national surveys

This Issue Brief reviews surveys that provide estimates of the uninsured population in the United States. It includes a discussion of why the estimates from the various surveys differ. It is important to understand the differences in the estimates of the uninsured population. The projected cost of implementing policy proposals depends on the estimates of the number of people affected by the proposals; for instance, the allocation of funding for the State Children's Health Insurance Program (S-CHIP) depends heavily on the available estimates. In addition, the estimated effectiveness of policy proposals to reduce the uninsured population will be accurate only if the correct count is known and the precise make-up of the uninsured population is understood. Currently, seven surveys can be used to make nationally representative estimates of the number of people without health insurance coverage. Some of the surveys collect health insurance information in the context of obtaining general information on health care, while other surveys are focused on other topics such as labor force participation and public assistance program participation. The most widely used survey that collects information on health insurance coverage is the Current Population Survey (CPS), conducted by the Census Bureau. The most recent estimates from the CPS suggests that 44.3 million Americans were uninsured in 1998. Besides the CPS, a number of other surveys collect information on the uninsured population. They include the Survey of Income and Program Participation (SIPP), Behavioral Risk Factor Surveillance System (BRFSS), Community Tracking Study (CTS), Medical Expenditure Panel Survey (MEPS), National Health Interview Survey (NHIS), and the National Survey of America's Families (NSAF). Estimates of the uninsured from these surveys range from 19 million to 44 million and vary depending on the time frame the survey covers. A number of states have started to question the validity of the uninsured estimates from the CPS, and other surveys, because of the small sample size in many states. As a result, some states have begun to conduct their own surveys to determine the number of uninsured residents. States that regularly conduct their own surveys include Florida, Massachusetts, Minnesota, New Mexico, Oregon, Vermont, and Wisconsin. Unfortunately, the various state surveys are not easily comparable. Research needs to continue to increase understanding of the differences among the surveys and to improve on methodologies to count the uninsured, as the future of public programs, such as S-CHIP and other state and local initiatives to expand health insurance coverage, depends on the accuracy of these estimates. Whatever survey is used, the results show that a substantial number of Americans do not have any health insurance coverage, and the number has been growing.     

Measuring expectations of inflation: Effects of survey mode,wording, and opportunities to revise

Several national surveys aim to elicit consumers’ inflation expectations. Median expectations tend to track objective inflation estimates over time, although responses display large dispersion. Medians also tend to differ between surveys, possibly reflecting survey design differences. Using a nationally representative Dutch sample, we evaluate the importance of three survey design features in explaining observed differences: mode (face-to-face vs. web), question wording (‘prices in general’ vs. ‘inflation’), and the explicit opportunity to revise responses. We examine effects on item non-responses, revisions, reported inflation expectations and their deviation from the CPI inflation rate. We discuss implications of our findings for survey design.     

Web survey design and administration.

Many claims are being made about the advantages of conducting surveys on the Web. However, there has been little research on the effects of format or design on the levels of unit and item response or on data quality. In a study conducted at the University of Michigan, a number of experiments were added to a survey of the student population to assess the impact of design features on resulting data quality. A sample of 1,602 students was sent an e-mail invitation to participate in a Web survey on attitudes toward affirmative action. Three experiments on design approaches were added to the survey application. One experiment varied whether respondents were reminded of their progress through the instrument. In a second experiment, one version presented several related items on one screen, while the other version presented one question per screen. In a third experiment, for one series of questions a random half of the sample clicked radio buttons to indicate their answers, while the other half entered a numeric response in a box. This article discusses the overall implementation and outcome of the survey, and it describes the results of the imbedded design experiments.     

Exploring the Socioeconomic and Immigration Health Contexts of Asian Americans

Despite an overall picture of good health for Asian Americans in the aggregate, growing evidence shows that specific subgroups—particularly those with high proportions of immigrants—have diverse patterns relative to socioeconomic position and health. Using data from the largest U.S. state health survey, this study explores the contributions of socioeconomic and immigration factors to health status among Asian American subgroups. Results indicate that level of education, income, and English language ability were significant predictors of health status for the Chinese, Korean, and Vietnamese subgroups. Varying patterns in the results by subgroup are discussed in the context of immigration policy.     

Peeking into the past to plan the future: Assessing bird species richness in a neotropical city

Urbanization has been considered as a major threat to biodiversity, making its ecology of increasing interest. Many urban ecology studies have been developed in a short time-scale, measuring real-time patterns. However, long-term studies are imperative to understand the responses of some species to the urbanization process. In this study, we aimed to assess the information provided by a recent one-year citywide bird survey when compared to a published ‘historical’ bird list of a neotropical city (Xalapa, Mexico) that compiles information of the past three decades, mainly from urban greenspaces. Specifically, we contrasted species richness values and assessed differences in species composition between both lists. We recorded 51 species in the one-year citywide survey, representing ~15 % of those reported in the historical list. Nonetheless, the upper-bound confidence interval of the citywide survey richness prediction represented ~66 %. Most of the species recorded in the one-year citywide survey are insectivores and granivores, a pattern that agrees with previous findings that underline the importance of insectivores as part of urban bird communities in the Neotropics. Although we used robust methods to compare our one-year citywide bird list and the historical list for the city of Xalapa, we acknowledge the limitations of comparing them. However, our results shed some light on the kind and type of information that one-year citywide surveys can provide and the importance of long-term studies for comprehending the processes involved in biodiversity changes within urban areas over time. Undoubtedly, establishing long-term citywide surveys sampling birds and other biodiversity groups will allow us to better understand the response of biodiversity to urbanization over time.     

Presentation matters: how mode effects in item non-response depend on the presentation of response options

Comparability of data across modes is an important issue in survey research. In this paper we discuss item non-response to attitudinal questions in telephone and web surveys. We present results from a survey experiment conducted in Italy and in Spain that compares different presentations of response options in an online setting with a benchmark telephone survey. In line with earlier studies we find that (A), the share of non substantial answers in the online survey depends on how the response option is presented. Comparing different presentations in an online survey to the standard approach of telephone surveys using propensity score matching, we find that (B), the share and pattern of non substantial answers, is most similar across the two modes in the online survey when the it is captured in an instruction on each screen for the online survey. Our findings are of particular relevance for the design of multiple mode or mix-mode surveys of attitudinal questions using online and telephone modes.     

Health System Reform and the Elderly: The Case of Israel

Abstract

Following the implementation of the National Health Insurance Law in January 1995, major changes have taken place in the provision of health services by Israel's sick funds and the way these services are funded. This article presents the principal changes that have occurred and assesses their impact on those aged 65 and older, using findings of a survey conducted one year after the law's enactment. While the findings show that the elderly's situation has improved in some areas, they also raise concerns that the elderly are having difficulty navigating the reformed health system and that they are not benefiting from the new competition among the sick funds.     

Mental health need, awareness, and use of counseling services among international graduate students

OBJECTIVE AND PARTICIPANTS: The authors examined the prevalence of mental health needs in international graduate students, their knowledge of mental health services, and their use of on-campus and off-campus counseling services. METHODS: All registered graduate students in the Spring 2004 semester received an e-mail invitation to participate in a Web survey. Of the 3,121 completed surveys, 551 completed surveys were from international graduate students. RESULTS: Approximately 44% of international graduate students responded that they had had an emotional or stress-related problem that significantly affected their well-being or academic performance within the past year. International students who reported a more functional relationship with their advisors were less likely to report having an emotional or stress-related problem in the past year and using counseling services. International students who reported higher financial confidence were also less likely to use counseling services. CONCLUSIONS: There is an unmet mental health need among international graduate students. Special mental health outreach efforts should be directed at international graduate students, with particular attention on the relationship between students and their advisors and on adequate financial support for students.     

(How) Can we value health, safety and the environment?

The assumptions that underpin the conventional economic model of ‘rational agents’ tend to be substantially violated by data from surveys that try to elicit people’s values for health, safety and environmental goods. Psychological research suggests that there may be a large affective component in people’s responses to such surveys, with the result that those data are not amenable to the ‘logic’ of economic rationality. This raises questions both about the way we model human judgment and decision processes, and also about the use of survey data to guide public policy in these and other areas.     

Color, Labels, and Interpretive Heuristics for Response Scales

We carried out two experiments to investigate how the shadingof the options in a response scale affected the answers to thesurvey questions. The experiments were embedded in two web surveys,and they varied whether the two ends of the scale were representedby shades of the same or different hues. The experiments alsovaried the numerical labels for the scale points and examinedresponses to both unipolar scales (assessing frequency) andbipolar scales (assessing favorability). We predicted that theuse of different hues would affect how respondents viewed thelow end of the scale, making responses to that end seem moreextreme than when the two ends were shades of the same hue.This hypothesis was based on the notion that respondents usevarious interpretive heuristics in assigning meaning to thevisual features of survey questions. One such cue is visualsimilarity. When two options are similar in appearance, respondentswill see them as conceptually closer than when they are dissimilarin appearance. The results were generally consistent with thisprediction. When the end points of the scale were shaded indifferent hues, the responses tended to shift toward the highend of the scale, as compared to scales in which both ends ofthe scale were shaded in the same hue. Though noticeable, thisshift was less extreme than the similar shift produced whennegative numbers were used to label one end of the scale; moreover,the effect of color was eliminated when each scale point hada verbal label. These findings suggest that respondents havedifficulty using scales and pay attention even to incidentalfeatures of the response scales in interpreting the scale points.     

Analyzing free-text survey responses: An accessible strategy for developing patient-centered programs and program evaluation

Despite widespread availability of yoga in the Veterans Health Administration (VA), it remains unclear how to best evaluate yoga programs. This is particularly problematic for programs aimed at veterans with mental health concerns, as evaluation typically focuses narrowly upon mental health symptom severity, even though program participants may have other health-related priorities. We analyzed responses to free-text questions on 237 surveys completed by veterans with mental health concerns enrolled in a yoga program at six VA clinics in Louisiana to characterize veteran participants’ experiences with yoga. Qualitative analysis resulted in 15 domains reflecting veterans’ individual health-related values and priorities. We use results to illustrate the potential for analysis of free-text responses to reveal valuable insights into patient experiences, demonstrating how these data can inform patient-centered program evaluation. The approach we present is more accessible to those responsible for decision-making about local programs than conventional methods of analyzing qualitive evaluation data.     

CROSS‐NATIONAL MORAL BELIEFS: The Influence of National Religious Context

International surveys have documented wide variation in religious beliefs and practices across nations, but does this variation in the national religious context make a difference? Building on existing theory, we explain why religion should have both micro‐ and macro‐level effects on morality not sanctioned by the state and why the effects of religion differ from other forms of culture. Using two international surveys and hierarchical linear modeling techniques we sort out the effects of national context and personal beliefs on morality with and without legal underpinnings. We find that national religious context, the respondent's age, and religious beliefs and practices are the most consistent predictors of the sexual morality index. For morality sanctioned by the state, however, the effects for personal beliefs and practices are attenuated, and the effects of the national religious context are no longer significant.