RESPONSE RATES AND QUALITY OF DATA: A RE-EXAMINATION OF THE MAIL QUESTIONNAIRE*

Traditionnellement, dans les sciences sociales on a été préoccupé par les techniques destinées à accoître les taux de réponse aux questionnaires distribués par la poste. Ce n'est que rarement qu'on a essayé de déterminer si des taux de réponse élevés avait une relation avec la qualité des données obtenues. Si on considère les taux de réponse d'envinon 60 pour cent obtenus dans les enquêtes faites par entrevues par les centres de recherche commerciaux, on doit réexaminer le problème de l'utilisation des questionnaires en voyés par la poste comme une alternative, même si on ne prend pas en considération la grande différence de coût par sujet entre les entrevues et les questionnaires envoyés par la poste. Dans la recherche discutée ici, le même questionnaire a été envoyé par la poste, dans des conditions différentes, à trois échantillons tirés au hasard. Le taux de réponse pour les trois Cchantillons variaient de 33 à 48 pour cent, mais les données obtenues dans chaque échantillon étaient pratiquement équivalentes. Pour controler ce resultat, un questionnaire semblable a été envoyéà un échantillon tiré d'une autre population. Le taux de réponse à ce questionnaire était juste inférieur à 90 pour cent, mais les données obtenues des premiers et des derniers sujets étaient généralement équivalentes. Traditionally, social scientists have been preoccupied with techniques for increasing response rates to mail questionnaires. Only rarely have there been attempts to ascertain whether high response rates are related to the quality of data obtained. Given current completion rates of about 60 per cent for interviews by commercial survey research centres, the issue of using mail questionnaires as an alternative data-gathering technique requires re-examination, even apart from any consideration of the wide differential between interviews and mail questionnaires in cost per respondent. In the current research, the same questionnaire was mailed under varying conditions to three independent random samples. The response rates for the three samples varied from 33 to 48 percent, but the data obtained from each sample were basically equivalent. As a further check on this finding, an overlapping questionnaire was sent to a sample from another population. The response rate to this questionnaire was just under 90 per cent, but the data from earlier and later respondents were generally equivalent.     

Child abuse and neglect re-reports: Combining and comparing data from two national sources

This study compared child-level estimates of child maltreatment re-report and recurrence in two national sets of data on child maltreatment: state administrative data submitted to the National Child Abuse and Neglect Data System (NCANDS) and caseworker interviews from the National Survey of Child and Adolescent Well-Being (NSCAW). Maltreatment data from NCANDS and NSCAW were merged for 2230 children that had intersecting information from both sets. The percentage of child cases with at least one re-report of abuse or neglect over the study period differed depending on the data set. The NCANDS re-report estimate was 32.3% (95% CI 26.5%, 38.6%) whereas the estimate based on NSCAW caseworker interviews was 22.9% (95% CI 17.6%, 29.2%). More than a quarter of the children with observations from the union of the two data sets had a re-report identified by one source but not the other (set difference). Most often, the set difference in re-reports appeared in NCANDS, but was not reported by an NSCAW caseworker. When the set difference from NCANDS was added to the re-reports by NSCAW caseworkers, the resulting union of re-reports increased the point estimate in the NSCAW–NCANDS intersection to 40.9% (95% CI 34.3%, 47.8%). Restricting the comparisons to only substantiated re-reports (recurrence) narrowed the differences in absolute terms but the set difference in recurrence rates was proportionally similar. Potential explanations for non-intersecting re-reports and recurrence between the two data sets were examined. Findings illuminate methodological challenges that may arise when child maltreatment re-report and recurrence data from administrative and survey sources are merged, and the value of the union of state-level administrative data with national survey data for studies of safety and well-being of children reported for maltreatment.     

Do required fields in online surveys in the social sciences impair reliability?

ABSTRACT

The present study is the first to examine empirically whether required fields in online surveys impair reliability and response pattern, as participants forced to respond to all items may provide arbitrary answers. Two hundred and thirteen participants completed a survey consisting of six questionnaires testing personal and social issues and perceptions. They were randomly assigned to one of two versions of the survey: optional-fields (N = 104) or required-fields (N = 109). Comparison of the Cronbach’s alpha of the two versions revealed identical reliability values for all questionnaires, save for somatization, where a minor difference was found. Confirmatory factor analysis showed no difference in the factor structure of the two versions, and no differences were found by Bayesian t-test and Levene’s test for equality of variances. The findings suggest that required fields do not impair reliability or change the response pattern, and therefore can be used in online surveys to prevent missing data.     

Responding to remediation and gatekeeping challenges in supervision

Thirty faculty in randomly selected Commission on Accreditation for Marriage and Family Therapy Education (COAMFTE) programs representing four geographical regions of the United States were asked to respond to a survey that included seven brief vignettes depicting gatekeeping and remediation challenges supervisors may face when working with therapists-in-training. Research participants were asked to select from among a range of 17 response options and were also asked to provide a rationale for the selection of their responses. The response rate for the mailed survey was 34%. Telephone interviews were conducted with three respondents who had provided especially detailed responses to the open-ended portions of the mailed survey and who also volunteered to be questioned regarding how supervision decisions are made. Results support the conclusion that COAMFTE faculty take their gatekeeping function seriously as they balance commitments to multiple stakeholders. When presented with hypothetical "bare-bones" vignettes, the supervisors in our study consistently recommended talking with the student in order to more fully understand the context of the student's performance problem before deciding how to proceed. Supervisors recommended a variety of remediation efforts, but reserved the most severe consequences, such as probation, dismissal, and filing an ethics complaint, for the vignette involving dishonesty and lack of personal integrity.     

The Ladder 1959

ABSTRACT

In June 1958, the lesbian publication The Ladder distributed a four-page mailed questionnaire to its 500 subscribers. The 157 returned questionnaires constitute the earliest study of lesbianism, and the results were published in The Ladder in the September 1959 issue. This survey is reprinted in the Journal of Lesbian Studies with permission of the founding editors.     

Relationship of work hours with selected health behaviors and academic progress among a college student cohort

Approximately 57% of college students work while attending school. Health risks related to working while in college have not been widely studied. OBJECTIVE: The authors' purpose in this study was to determine associations between hours worked, binge drinking, sleep habits, and academic performance among a college student cohort. PARTICIPANTS AND METHODS: The authors randomly selected a sample of 1,700 undergraduates from a southeastern US university and mailed to them a survey requesting a variety of self-reported health behaviors and hours worked. A total of 903 completed questionnaires were received, indicating a response rate of 57.3%. RESULTS: Binge drinking, less sleep, and lower academic performance were significantly associated with working 20 or more hours per week. Those variables were not associated with working fewer than 20 hours per week. CONCLUSIONS: Although administrators and others in higher education are aware of the impact of economics on a school's ability to operate, they may not be aware of the impact on students' health.     

Parental substance misuse and child care social work: research in a city social work department in England

This research set out to establish evidence about the scale and impact of and response to parental substance misuse in child care social work teams in a city social services department in England. The article draws on some aspects of the data collected in the research, which includes: a snapshot survey of all child care social work caseloads in the city; group interviews with practitioners and parents in recovery; individual interviews with parents using a pilot project that focused on parental substance misuse. The research provides evidence of parental substance misuse as a key factor that needs greater consideration within child care social work assessments and as an issue to target in developing preventative responses to child welfare concerns. Child care social workers are shown to need specialist support in undertaking this task to best effect. Copyright © 2004 John Wiley & Sons, Ltd.     

Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias

The last 50 years have seen a gradual replacement of face-to-faceinterviewing with telephone interviewing as the dominant modeof survey data collection in the United States. But some ofthe most expensive and large-scale nationally funded, long-termsurvey research projects involving national area-probabilitysamples and long questionnaires retain face-to-face interviewingas their mode. In this article, we propose two ways in whichshifting such surveys to random digit dialing (RDD) telephoneinterviewing might affect the quality of data acquired, andwe test these hypotheses using data from three national modeexperiments. Random digit dialing telephone respondents weremore likely to satisfice (as evidenced by no-opinion responding,nondifferentiation, and acquiescence), to be less cooperativeand engaged in the interview, and were more likely to expressdissatisfaction with the length of the interview than were face-to-facerespondents, despite the fact that the telephone interviewswere completed more quickly than the face-to-face interviews.Telephone respondents were also more suspicious about the interviewprocess and more likely to present themselves in socially desirableways than were face-to-face respondents. These findings shedlight on the nature of the survey response process, on the costsand benefits associated with particular survey modes, and onthe nature of social interaction generally.     

Relationship of Work Hours With Selected Health Behaviors and Academic Progress Among a College Student Cohort

Approximately 57% of college students work while attending school. Health risks related to working while in college have not been widely studied. Objective: The authors' purpose in this study was to determine associations between hours worked, binge drinking, sleep habits, and academic performance among a college student cohort. Participants and Methods: The authors randomly selected a sample of 1,700 undergraduates from a southeastern US university and mailed to them a survey requesting a variety of self-reported health behaviors and hours worked. A total of 903 completed questionnaires were received, indicating a response rate of 57.3%. Results: Binge drinking, less sleep, and lower academic performance were significantly associated with working 20 or more hours per week. Those variables were not associated with working fewer than 20 hours per week. Conclusions: Although administrators and others in higher education are aware of the impact of economics on a school's ability to operate, they may not be aware of the impact on students' health.     

Learning pattern of social work students: a longitudinal study

Research on learning patterns of social work students is scare. This longitudinal study addresses this issue by employing mixed methods of quantitative and qualitative study design to understand the learning pattern of students throughout their undergraduate social work studies. Six rounds of quantitative data collected by structured questionnaires and four rounds of qualitative data collected by focus group interviews have been conducted at an interval of every 6 months to track the learning pattern and to evaluate the continuous impacts of social work education on students. Results indicate that although students’ evaluation on the curriculum was positive and with significant difference in mean scores, paired sample t-tests found no significant difference between time 1 and 6 of all outcome variables. With regard to students’ change in attitude, acquisition of knowledge and skills, results indicate a ‘U-shape’ trend and a fluctuating learning pattern with a rebound in different outcome variables toward the end of the three-year study. Qualitative data echoes the findings and reveals that students go through the confused freshman stage, to disillusioned in practicum and finally with enriching experience upon graduation. The paper discusses meanings and implications of student learning pattern to social work education.     

Diglossia and identity in Northeast Thailand: Linguistic,social, and political hierarchy

The paper explores diglossic relations between Central Thai and phasa isan, a variety officially known as a dialect of Thai, but linguistically close to Lao. Phasa isan is spoken by almost one‐third of Thailand's population but its speakers in the Northeast are often stigmatized as uneducated and backward. We conducted field research mainly among university students in Ubon Ratchathani, a northeastern border province, by drawing upon data from survey questionnaires, reflective essays, interviews, and field observations. The findings suggest a transitional diglossic relationship in which Central Thai is the High and phasa isan the Low variety. These relationships are discussed in terms of nationalism, social hierarchy, and language maintenance and shift.     

Interviewing Older Adults: A Comparison of Telephone and Face-to-Face Modalities

This article examines telephone interviewing of older adultsand compares it with face-to-face interviews. Specifically,the following issues are examined in several national surveys:(1) differences in age distributions between the samples ofadults reached in both modes; (2) explanations for potentialdifferences in age distributions; (3) differences between thetwo modes in demographic characteristics in the adults reached,in interview process and in response quality, and how thesemode differences vary by age of the respondents. Telephone surveystend to underrepresent older persons, and older persons whodo participate in a telephone survey are disproportion atelywell educated. Implications of the lower response rate amongolder persons are softened by the fact that reponse distributionsacross a range of questions show little difference by interviewmode between older persons and persons of other age groups.     

Investigating validity and effectiveness of cognitive interviewing as a pretesting method for non-English questionnaires: Findings from Korean cognitive interviews

This study aims to empirically investigate whether cognitive interviewing is effective as a pretesting method for detecting problems in translated survey questionnaires as in the English language source questionnaire. The validity of non-English cognitive interviews is of concern among researchers because the cognitive interviewing technique was developed based on the communicative norms of Western cultures where directness and openness in expressing one’s opinion is encouraged. However, different communicative norms are present in Asian languages and cultures. To date, little research has been conducted on this topic. Using qualitative and quantitative analysis, we compared survey question problems identified through the cognitive interviews conducted in English and in Korean and found that Korean cognitive interviewing was indeed effective for detecting problematic survey questions.     

The Relationship Between Work-to-Family Conflict and Family-to-Work Conflict: A Longitudinal Study

The relationship between work-to-family conflict and family-to-work conflict was tested using a longitudinal research design with 234 dual-earner couples caring for both children and aging parents. Two waves of mailed survey data were collected. The mediating effects of role-related satisfaction were hypothesized to link these two forms of work-family conflict. The analytical steps for determining mediating effects followed suggestions by Baron and Kenny (1986). The results failed to show significant mediating effects of role-related satisfaction between the two forms of conflict over time for both husbands and wives, contrary to suggestions by Frone, Yardley, and Markel (1997). Instead, positive, direct relationships between the two types of work-family conflict were found. Limitations and implications are discussed.     

Enhancing response rates to a smoking survey for enlisted U.S. Navy women.

This study examined the effectiveness of a stepped approach for increasing response rates to a mailed follow-up smoking survey among newly enlisted women in the navy. The effect of the stepped approach on response rates and on the characteristics of respondents was evaluated. Also, costs were estimated for each of the steps to determine their relative benefits. Results showed that the stepped approach was effective in more than doubling the survey response rate among smokers. Reluctant respondents did not differ from on-time respondents in terms of demographics or baseline smoking, although nonrespondents were less educated and heavier smokers than on-time and reluctant respondents. Strategies documented here could well apply to survey efforts with other hard-to-reach populations.     

Desegregation in Higher Education: An Examination of Traditionally Black and White Institutions*

Using the data collection techniques of historical analysis, secondary analysis of EEO-6 records, mailed questionnaires, and personal interviews, race relations among trustees, administrators, and faculty in the southeastern United States were examined during the late 1970s and early 1980s. The conclusion that blacks at traditionally white institutions (TWI) (1) had more tenuous job status, (2) experienced less job satisfaction, (3) experienced greater degrees of institutional alienation, and (4) more often stated a need for affirmative action programs than did their white colleagues at the same institution will surprise no one. Similar patterns and perceptions were found for whites at traditionally black institutions (TBI) where whites were in the “minority.” Although whites at TBIs remained members of the societal majority group as traditionally defined in the sociology of intergroup relations, their position and perceptions within specific TBIs were remarkably similar to the experience of black professionals at TWIs. Some implications for policy are drawn.     

Pensions,poverty and wellbeing in later life: Comparative research from South Africa and Brazil

This paper draws on two linked studies of social policy and wellbeing in later life. The studies make comparisons between distinct groups of older people at the national and sub-national levels, as well as over time. The paper reflects on some of the main challenges for operationalising this complex design, as well as for interpreting findings and identifies lessons for other studies. The first study, conducted in 2002, included a questionnaire survey of around 2000 households containing at least one older person in South Africa and Brazil, supplemented by a set of in-depth qualitative interviews. Intriguingly, these countries had remarkably similar pension programmes, providing the majority of older people a reliable payment of roughly US$3 a day. This offered the prospect of exploring the effects of similar interventions in distinct developmental and cultural settings. In both countries, we found that these pensions had a substantial impact on the prevalence and depth of poverty in the study households, and were usually shared between older people and other family members. The second survey took place in 2008/9 and involved revisiting the households included in the 2002 survey, along with a separate set of in-depth interviews. This provided an opportunity for dynamic analysis of economic and wellbeing effects, against a backdrop of increased divergence in the wider national settings. Among other things, this revealed high and increasing levels of life satisfaction across all the study groups, although the extent to which this was directly related to generous pension provision cannot be ascertained.     

The silence of numbers: A split-sample experiment concerning respondents' linguistic sensitivity in Danish survey studies

This article presents, for the first time in English, findings from a Danish research project on language comprehension and memory as generators of measurement problems in sociological and other social-science survey studies. Utilizing survey questionnaires as instruments of measurements, the article deals with the linguistic sensitivity of Danish adults and the measurement problems it entails. The article uncovers the nature and extent of statistically significant response differences due to moderate linguistic changes. The article problematizes the trustworthiness of Danish social-science survey studies and provides evidence that respondents' answers to survey questions depend to a marked extent on the latter's wording. The empirical basis is a split-sample experiment in which 1900 respondents completed two versions of the 'same' questionnaire.     

Sex Differences in Initiation of and Adjustment to Divorce

Men and women for this study were randomly chosen from courthouse divorce files. After initial contact by mail and by phone, those who agreed to participate were mailed questionnaires developed to assess adjustment to divorce and to obtain information about the nature of the divorce. Forty-four percent of those who received questionnaires completed and returned the assessment; the total sample consisted of 133 subjects. A sex difference in decision to separate was found, with both men and women reporting that the decision had been the woman's. Women showed better adjustment than men on a divorce adjustment scale. In addition, women reported significantly fewer suicidal feelings than males. However, males had significantly lower factor scores on the Profile of Mood States for tension and on the Semantic Differential for stability. Other Profile of Mood States and Semantic Differential measures did not reveal significant sex differences.