Effects of Personal Salutations in E-mail Invitations to Participate in a Web Survey

Effects of personalizing survey invitations on response rateshave been extensively researched in the realm of mail surveys.Commonly, it is found that response rates increase when personalizationis applied. Recently, efforts have been made to investigatewhether these findings extend to the field of Web surveys thatuse e-mail invitations. Using data from a Web experiment conductedamong first-year university students, personalization of e-mailinvitations is shown to have significantly increased the responserate by 7.8 percentage points. From the theory that personalizationdecreases the level of anonymity and perceived privacy, differencesin responses to sensitive questions were expected. However,no effects of personalization on the responses to a questionprobing for respondents’ sexual behavior were found.     

An experiment to test the feasibility and quality of a web-based questionnaire of teachers

The use of web-based surveys to gather information from teachers has become increasingly common primarily based on the premise that they can reduce costs. Yet, relatively, little is known about the quality or cost effectiveness of web-based versus mail surveys for teachers. To study the efficacy of web-based teacher surveys, the author randomly assigned a nationally representative sample of 877 elementary school teachers to a paper or web survey mode. The cost savings from the web administration were not enough to offset the loss of sample due to missing/inaccurate email addresses and the lower response rates that resulted from the web survey.     

Email subject lines and response rates to invitations to participate in a web survey and a face-to-face interview: the sound of silence

This paper investigates the relationship between blank and non-blank email subject lines on levels of response to a solicitation to participate in an interview, and on participation in a web survey. Email use has grown substantially in recent years, presenting significant opportunity to the empiricist seeking research respondents. However, response to emails may be low because growth in the sheer volume of messages that individuals receive per day has led to a sense of ‘email overload’, and faced with the challenge of personal email management, many recipients choose to ignore some messages, or do not read them all fully. Drawing on information gap theory, we expected that sending an invitation with a blank subject line would induce a sense of curiosity in recipients that would improve email response and willingness to participate in research studies. However, findings from research with two samples with different propensities to participate in research (academics and business owners) revealed that an email invitation with a blank subject line does not increase overall response rates to a web survey and a face-to-face interview over either an informative subject line or a provocative subject line, but that it does prompt a greater number of active refusals. Based on this finding, recommendations for researchers are outlined.     

Mixed-Mode Contacts In Web Surveys: Paper is Not Necessarily Better

This paper investigates the impact of paper and email contactson web survey response rates. We use six combinations of paperand email prenotifications and reminders to test the impactof mixed-mode contacts. In addition, we use two survey samplesthat differ in their relationship with the sponsoring institutionto test if the impact of contact mode is conditional on relationshipbetween respondents and the survey researchers. Contrary toprevious research, we find little differences in response ratesacross experimental groups.     

Personalization of Mail Surveys for General Public and Populations with a Group Identity: Results from Nine Studies*

Abstract The effect of personalization on mail survey response rates was examined in nine studies that included 17 comparisons under several research conditions. A study of this variable across multiple experiments in five agricultural experiment stations was undertaken because of conflicting results from previous research and from concern that the effectiveness of personalization might have decreased over time. Results show that, while response to general public surveys appeared to increase modestly across all treatment groups, there was no positive effect for populations in which a group identity (e.g., Dear Oregon Gardner or Dear ATV Owner) is employed to address respondents in cover letters. Personalization appears to remain useful for improving response in surveys of the general public.     

Investigating recruitment and completion mode biases in online and door to door electronic surveys

Electronically assisted survey techniques offer several advantages over traditional survey techniques. However, they can also potentially introduce biases, such as coverage biases and measurement error. The current study compares the relative merits of two survey distribution and completion modes: email recruitment with internet completion; and door to door recruitment with either tablet or internet completion. Presentation mode is held constant so that we are able to separate the impacts of recruitment mode and completion mode on responses. Recruitment mode appeared to influence both response rates and which socio-demographic groups were represented. However, the difference between the two recruitment modes was relatively small. Completion mode appeared to have little or no impact on responses although it did influence completion times. The email distributed survey performed better with regard to time costs and the number of respondents obtained. Differences between the two survey modes appeared to be largely due to recruitment method rather than completion mode.     

A comparison of email networks and off-line social networks: A study of a medium-sized bank

Recently there has been a surge in the availability of online data concerning the connections between people, and these online data are now widely used to map the social structure of communities. There has been little research, however, on how these new types of relational data correspond to classical measures of social networks. To fill this gap, we contrast the structure of an email network with the underlying friendship, communication, and advice seeking networks. Our study is an explorative case study of a bank, and our data contains emails among employees and a survey of the ego networks of the employees. Through calculating correlations with QAP standard errors and estimating exponential random graph (ERG) models, we find that although the email network is related to the survey-based social networks, email networks are also significantly different: while off-line social networks are strongly shaped by gender, tenure, and hierarchical boundaries, the role of these boundaries are much weaker in the email network.     

Evaluating the Quality of Online Survey Data Collected in 2018 in the USA: Univariate,Bivariate, and Multivariate Analyses

Collecting survey data in Japan and the USA is difficult for many reasons, including the difficulty of creating complete sampling frames and low response rates. Online surveys, which collect data through online access panels and recruit members using nonprobability sampling, is becoming a popular alternative to traditional survey techniques, both in market and social science research. However, sociological researchers are hesitant to adopt online survey data collection techniques as it is unclear how representative such data are. This article examines the quality of online survey data by comparing online surveys with the two datasets collected using probability sampling, the American Community Survey and the General Social Survey. We conducted univariate, bivariate, and multivariate analyses and investigated whether similarities improve from the univariate analyses to bivariate relationships, and to bivariate relationships with controls. The results show that similarities improved from univariate to bivariate and multivariate analyses, but the results of bivariate and multivariate analyses did not differ. We concluded that data collected via volunteer‐based access panels yielded similar results to the benchmark data, especially when the relationships among variables are examined. We argue that automatically discrediting the data collected via volunteer‐based access panels is not good practice. With careful considerations of data quality, online survey data with nonprobability sampling could advance sociological research by giving researchers more freedom to ask questions at a relatively low cost, with a quick turnaround time, and more opportunities to conduct cross‐national research.     

Reliability and validity of egocentered network data collected via web: A meta-analysis of multilevel multitrait multimethod studies

Our goal in this article is to assess reliability and validity of egocentered network data collected through web surveys using multilevel confirmatory factor analysis under the multitrait multimethod approach.In this study, we analyze a questionnaire of social support of Ph.D. students in three European countries. The traits used are the frequency of social contact questions. The methods used are web survey design variants.We consider egocentered network data as hierarchical; therefore, a multilevel analysis is required. Within and between-ego reliabilities and validities are defined and interpreted.Afterwards, we proceed to a meta-analysis of the results of the three countries where within and between-ego validities and reliabilities are predicted from survey design variables which have to do with question order (by questions or by alters), response category labels (end labels or all labels) and lay-out of the questionnaire (graphical display or plain text). Results show that question order by questions, all-labeled response categories and a graphical display lay-out with images lead to a better data quality. Our basic approach consisting on multilevel and meta-analysis can be applied to evaluate the quality of any type of egocentered network questionnaire, regardless of the data collection mode.     

Does the online collection of ego-centered network data reduce data quality? An experimental comparison

We analyze whether differences in kind and quality of ego-centered network data are related to whether the data are collected online or offline. We report the results of two studies. In the first study respondents could choose between filling out ego-centered data through a web questionnaire and being probed about their network in a personalized interview. The second study used a design in which respondents were allocated at random to either online or offline data collection. Our results show that the data quality suffers from the online data collection and the findings indicate that this is the consequence of the respondents answering “mechanically”. We conclude that network researchers should avoid to simply copy traditional network items into a web questionnaire. More research is needed about how new design elements specific for web questionnaires can motivate respondents to fill out network questions properly.     

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Recent experimental research has shown that respondents to forced-choicequestions endorse significantly more options than respondentsto check-all questions. This research has challenged the commonassumption that these two question formats can be used interchangeablybut has been limited to comparisons within a single survey mode.In this paper we use data from a 2004 random sample survey ofuniversity students to compare the forced-choice and check-allquestion formats across web self-administered and telephoneinterviewer-administered surveys as they are commonly used insurvey practice. We find that the within-mode question formateffects revealed by previous research and reaffirmed in thecurrent study appear to persist across modes as well; the telephoneforced-choice format produces higher endorsement than the webcheck-all format. These results provide further support forthe argument that the check-all and forced-choice question formatsdo not produce comparable results and are not interchangeableformats. Additional comparisons show that the forced-choiceformat performs similarly across telephone and web modes.     

Investigating the impact of violations of the “left and top means first” heuristic on response behavior and data quality

ABSTRACT

Web surveys are an established data collection mode that use written language to provide information. The written language is accompanied by visual elements, such as presentation formats and shapes. However, research has shown that visual elements influence response behavior because respondents sometimes use interpretive heuristics to make sense of the visual elements. One such heuristic is the ‘left and top means first’ (LTMF) heuristic, which suggests that respondents tend to believe that a response scale consistently runs from left to right or from top to bottom. We conducted a web survey experiment to investigate how violations of the LTMF heuristic affect response behavior and data quality. For this purpose, a random half of respondents received response options that followed a consistent order and the other half received response options that followed an inconsistent order. The results reveal significantly different response distributions between the two groups. We also found that inconsistently ordered response options significantly increase response times and decrease data quality in terms of criterion validity. We, therefore, recommend using options that follow the design strategies of the LTMF heuristic.     

NEW MEDIA'S POTENTIAL FOR PERSONALIZATION

Despite the growing maturity of new, interactive media, rhetoric about its possibilities and potentialities that abounded in its earliest days still endures. The growth of detailed and empirical work, which has sought to populate the digital landscape with grounded research calling into question this rhetoric, has not stopped new media debate from continuing to be shaped, in part, by the language of the potential. This paper is concerned with one aspect of new media's potential, personalization. It focuses on new media's proclaimed capacity to be adapted to meet the needs and desires of individual users. This is a trope that runs through much humanities and social sciences literature on new media and ICTs, yet despite recognition of the possibility of personalization offered by networked media technologies, there is very little grounded, empirical work on this subject in these fields. In order to address this absence, this paper compares an attempt to personalize new media web content on a two-year research endeavour entitled Project @pple with the rhetoric about the potentiality for personalization that new technologies offers. It aims to contribute to understandings of personalization by detailing the issues that arise when attempting to implement it. The argument of the paper is that the difficulties encountered on Project @pple suggest that, in real-life situations, characterized as they are by constraints and complexities, it is not always a straightforward process for personalization to cease to be potential and to become actual.     

Unions and social media: Prospects for gender inclusion

The emergence of web 2.0 technologies led to optimistic predictions that social media (SM) might alter traditional gendered patterns of member participation in trade unions. Greene, Hogan, and Grieco and others suggested that the forms of communication and engagement these technologies offered to unions and their members had the potential to foster gender inclusion and contribute to union diversity, arguably central to effective representation. This article reports on a survey of union members’ experiences with and perceptions of their union's SM services, to identify whether there is a gendered dimension to members’ use. The findings indicate that for most union members regardless of gender, more traditional communication channels such as face‐to‐face contact and email remain the preferred means of communication. However, the findings also show that women are just as likely as men, if not more so, to engage with union SM. Given that historically, women largely participated less in union activities than their male counterparts, this broad parity of use by women supports the conclusion that SM has substantial potential to improve women's participation in unions.     

Möglichkeiten und Probleme des Einsatzes postalischer Befragungen

The research focuses on the strategies in mail surveys that lead to high response rates and on the social characteristics of the respondents that are recruited into the survey through reminders. Empirical basis are surveys in the East German cities of Chemnitz, Dresden and Leipzig with an original sample size of ca 2000 persons per city and up to three reminders. The response rate varies between 68 and 70 percent and surpasses the usual response rates in face to face and telephone surveys done in German cities. In contrast to common assumptions in the literature it has no effect on the response rate whether stamps are used on the address and return envelope, neither do original versus scanned signatures make a difference and whether financial incentives are used in the late reminders. Through the reminders especially respondents with low education and of little political interest are recruited into the survey.     

SurveyMotion: what can we learn from sensor data about respondents’ completion and response behavior in mobile web surveys?

Participation in web surveys via smartphones increased continuously in recent years. The reasons for this increase are a growing proportion of smartphone owners and an increase in mobile Internet access. However, research has shown that smartphone respondents are frequently distracted and/or multitasking, which might affect completion and response behavior in a negative way. We propose ‘SurveyMotion (SMotion)’, a JavaScript-based tool for mobile devices that can gather information about respondents’ motions during web survey completion by using sensor data. Specifically, we collect data about the total acceleration (TA) of smartphones. We conducted a lab experiment and varied the form of survey completion (e.g. standing or walking). Furthermore, we employed questions with different response formats (e.g. radio buttons and sliders) and measured response times. The results reveal that SMotion detects higher TAs of smartphones for respondents with comparatively higher motion levels. In addition, respondents’ motion level affects response times and the quality of responses given. The SMotion tool promotes the exploration of how respondents complete mobile web surveys and could be employed to understand how future mobile web surveys are completed.     

The effect of data collection mode on smoking attitudes and behavior in young African American and Latina women. Face-to-face interview versus self-administered questionnaires

Evaluating smoking prevention and cessation programs requires valid data collection. This study examined two survey modes--face-to-face (FTF) interview and self-administered questionnaire (SAQ)--comparing response rates, sample characteristics, data quality, and response effects. From two family planning clinics, 601 female Latina and African American clients ages 12 to 21 were recruited and randomized to either group. Results reveal that neither mode is superior to the other. The SAQ may therefore be preferable for this population, despite its higher rate of incompletes, because it yields results similar to the FTF yet is more cost effective and less disruptive to clinic routines.     

INFLUENCE OF AN INVITATION TO ANSWER BY TELEPHONE ON RESPONSE TO CENSUS QUESTIONNAIRES

An experimental test was made of whether completion rates fora census questionnaire could be improved by offering the optionof calling a toll-free number and providing the requested informationto an interviewer as an alternative to mailing it back. Conductedon a national probability sample of house holds, five treatmentpanels were created to examine the effect of delivering, indifferent ways, the invitation to respond by telephone or mail.Offering such invitations did not improve completion rates exceptwhen included with a follow-up letter that did not also includea replacement questionnaire.     

Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

Problems Relating to Declining Response Rates to Social Survey Research in Japan: Trends After 2000

Abstract: Sociologists in Japan recognize that the circumstances under which social survey research is conducted have been deteriorating and have become critical since 2000. Social survey research is understood as a large-scale survey that mainly draws samples by a random sampling method, using a questionnaire as a measurement tool. The crises of social survey research are summarized as following: (1) the increasing difficulty of conducting accurate sampling and (2) the decline in response rates to drawn samples (planned samples). This paper focuses on the decline in response rates to social survyes by examining recent trends in relation to social surveys conducted by the Japanese government and local government (prefectures). In summary, there has been a landslidedecline of the response rate between 2005 and 2006 in surveys conducted by Japanese government. This decline was thought to mainly come from the declining of response rates among the elderly and women, groups that had previously had high response rates. This seems to be the result of the growing level of the crime involving the abuse of the personal information. However, there did not appear to be a serious decline in the response rate for surveys conducted by local governments. Given that it is not easy to restore response rates, it is important to develop research methods and techniques to cope with the decline. Further, informing respondents of the purpose of the research and its benefit to may help raise response rates.