How unclear terms affect survey data

Although writing clear questions is accepted as a general goal in surveys, procedures to ensure that each key term is consistently understood are not routine. Researchers who do not adequately test respondent understanding of questions must assume that ambiguity will not have a large or systematic effect on their results. Seven questions that were drawn from questions used in national health surveys were subjected to special pretest procedures and found to contain one or more poorly defined terms. When the questions were revised to clarify the definition of key terms, significantly different estimates resulted. The implication is that unclear terms are likely to produce biased estimates. The results indicate that evaluation of survey questions to identify key terms that are not consistently understood and defining unclear terms are ways to reduce systematic error in survey measurement.     

The impact of child obesity on active parental consent in school-based survey research on healthy eating and physical activity

Previous studies have shown that active consent procedures result in sampling bias in surveys dealing with adolescent risk behaviors such as cigarette smoking and illicit drug use. To examine sampling bias from active consent procedures when the survey topic pertains to childhood obesity and associated health behaviors, the authors pair data obtained from both active and passive consent procedures. The authors find that parents of children who are overweight or at risk for being overweight are significantly less likely to give active consent. In addition, parents of children enrolled in lower grades are more reluctant to consent to participate.     

Increasing the effectiveness of time-use survey with qualitative methods: The analysis of time–space interaction

Time-use surveys have been rich data sources in many countries for a long time. Turkey was among the countries that realized the potential of time-use surveys quite late and completed the first national survey in 2006. Despite its importance for a wide range of issues and applications, the first survey has flaws in design, which reduce its effectiveness and reliability. This is mostly due to disregarding cultural factors while tracking the methodology of European examples. This study aims to propose more appropriate methods of gathering time-use data in the Turkish context through a field survey in Ankara, the capital city. A mixed methodology that combines quantitative and qualitative methods effectively was applied and used to enrich data. The influence of space use was stressed and leisure activities were utilized to exemplify the use and benefits of mixed methods.     

The relative frequency of offensive and defensive gun uses: results from a national survey

Some controversy exists about the relative frequency of criminal and self-defense gun use in the United States. Using data from a national random-digit-dial telephone survey of over 1,900 adults conducted in 1996, we find that criminal gun use is far more common than self-defense gun use. This result is consistent with findings from other private surveys and the National Crime Victimization Surveys. In this survey, all reported cases of criminal gun use and many cases of self-defense gun use appear to be socially undesirable. There are many instances of gun use, often for intimidation, that are not reported to the police and may not appear in official crime statistics.     

Evaluating the Quality of Online Survey Data Collected in 2018 in the USA: Univariate,Bivariate, and Multivariate Analyses

Collecting survey data in Japan and the USA is difficult for many reasons, including the difficulty of creating complete sampling frames and low response rates. Online surveys, which collect data through online access panels and recruit members using nonprobability sampling, is becoming a popular alternative to traditional survey techniques, both in market and social science research. However, sociological researchers are hesitant to adopt online survey data collection techniques as it is unclear how representative such data are. This article examines the quality of online survey data by comparing online surveys with the two datasets collected using probability sampling, the American Community Survey and the General Social Survey. We conducted univariate, bivariate, and multivariate analyses and investigated whether similarities improve from the univariate analyses to bivariate relationships, and to bivariate relationships with controls. The results show that similarities improved from univariate to bivariate and multivariate analyses, but the results of bivariate and multivariate analyses did not differ. We concluded that data collected via volunteer‐based access panels yielded similar results to the benchmark data, especially when the relationships among variables are examined. We argue that automatically discrediting the data collected via volunteer‐based access panels is not good practice. With careful considerations of data quality, online survey data with nonprobability sampling could advance sociological research by giving researchers more freedom to ask questions at a relatively low cost, with a quick turnaround time, and more opportunities to conduct cross‐national research.     

Consequences of reducing nonresponse in a national telephone survey

Critics of public opinion polls often claim that methodological shortcuts taken to collect timely data produce biased results. This study compares two random digit dial national telephone surveys that used identical questionnaires but very different levels of effort: a "Standard" survey conducted over a 5-day period that used a sample of adults who were home when the interviewer called, and a "Rigorous" survey conducted over an 8-week period that used random selection from among all adult household members. Response rates, computed according to AAPOR guidelines, were 60.6 percent for the Rigorous and 36.0 percent for the Standard study. Nonetheless, the two surveys produced similar results. Across 91 comparisons, no difference exceeded 9 percentage points, and the average difference was about 2 percentage points. Most of the statistically significant differences were among demographic items. Very few significant differences were found on attention to media and engagement in politics, social trust and connectedness, and most social and political attitudes, including even those toward surveys.     

Using cognitive interviews to refine translated survey questions: an example from a cross-national crime survey

Lately, social researchers have become increasingly interested and involved in comparative and cross-national research. In part this reflects a growing realisation that key social, economic and political processes are not merely ‘domestic’ but are common across countries and cultures. Although numerous good quality data sets now exist (e.g. the European Social Survey or International Social Survey Programme), in many instances researchers need to design their own bespoke comparative surveys. Such surveys require careful design if they are to produce meaningful data. Herein we introduce, outline and report on one technique which may usefully be employed in order to design better survey items for use in cross-national surveys.     

The role of consumer and business sentiment in forecasting telecommunications traffic

The objective of this study is to investigate the relevance of consumer and business sentiment surveys in forecasting international telecommunications traffic. The components of telecommunications data used in the study relate to total telephone traffic outgoing from Australia and its social and business telephone sub-categories. Three sentiment surveys are considered: one consumer survey conducted by the University of Melbourne, and two business surveys, each conducted by a business association and major bank. The above data series are available on a quarterly basis from the first quarter of 1973. The approach employed consists of the application of transfer function modelling techniques to the traffic and various indexes measuring consumer and business sentiment. A major finding of this study is that sentiment series are correlated with international telephone traffic, and that in all but one case, the relationship can be modelled by some form of transfer function. The forecasts of the estimated models satisfactorily incorporate directional swings in the traffic.     

A Comparison of Survey Techniques on Sensitive Sexual Behavior in Italy

This article compares two national surveys carried out through the most commonly used procedures in Italy: CATI (computer-assisted telephone interviews) and SAQ-FI (self-answered questionnaires following interviews). Both surveys ask two identical questions concerning sensitive sexual behavior: early age at first intercourse and same-sex attraction. The SAQ-FI survey had both unit non-response and item non-response rates much lower than the CATI survey. Moreover, in the CATI survey, the groups with highest item non-response rates were also the groups with the lowest proportions of early intercourse and homosexual attraction. In addition, a differential analysis of the respondents produced diverse results for the two surveys. This is especially true of results by gender for same-sex attraction: Such behavior is more common among men (3.1%) than women (2.9%), according to the CATI survey, whereas the opposite is true of the SAQ-FI survey (6.1% of men vs. 7.7% women). In Italy at the beginning of the 21st century, CATI surveys reveal a lower level of early intercourse and same-sex attraction than SAQ-FI surveys. This article argues that the CATI survey underestimates the true level of these sensitive sexual behaviors in the Italian population.     

Characterizing urban butterfly populations: the case for purposive point-count surveys

Developing effective butterfly monitoring strategies is key to understanding how butterflies interact with urban environments, and, in turn, to developing local conservation practices. We investigated two urban habitat types (public gardens and restored/reconstructed prairies) and compared three survey methods (Pollard transects, purposive point counts, and random point counts) to determine which was most productive for detecting butterflies and assessing family diversity. We conducted 66 butterfly surveys by using each method (198 total) from May through September in 2015 and 2016 at six sites (three public gardens and three prairies) in Ames, Ankeny and Des Moines, Iowa. All survey methods were used on 11 sampling dates at each site. Overall, we observed 2,227 butterflies representing 38 species: 1,076 in public gardens and 1,151 in prairie areas. We used a smaller data set standardized for survey effort, including 1,361 of these sightings, to compare survey methods and habitat types. Although there were no significant differences in number of butterfly sightings between the two habitats, more sightings (798) were documented by using purposive point counts when compared to Pollard transects (297) or random point counts (266) (for both comparisons, p?<?0.0001). Occupancy modeling also indicated that purposive point counts were most effective in detecting certain species of butterflies, most notably those within the Pieridae (whites, sulphurs) and Papilionidae (swallowtails). We conclude that public gardens and restored/reconstructed prairies in urban settings can provide important butterfly habitat, and that purposive point-count surveys are most effective for detecting butterflies in these relatively small-scale landscape features.

     

Applying national survey results for strategic planning and program improvement: The National Diabetes Education Program

Since the 1970s, the federal government has spearheaded major national education programs to reduce the burden of chronic diseases in the United States. These prevention and disease management programs communicate critical information to the public, those affected by the disease, and health care providers. The National Diabetes Education Program (NDEP), the leading federal program on diabetes sponsored by the National Institutes of Health (NIH) and the Centers for Disease Control and Prevention (CDC), uses primary and secondary quantitative data and qualitative audience research to guide program planning and evaluation. Since 2006, the NDEP has filled the gaps in existing quantitative data sources by conducting its own population-based survey, the NDEP National Diabetes Survey (NNDS). The NNDS is conducted every 2–3 years and tracks changes in knowledge, attitudes and practice indicators in key target audiences. This article describes how the NDEP has used the NNDS as a key component of its evaluation framework and how it applies the survey results for strategic planning and program improvement. The NDEP's use of the NNDS illustrates how a program evaluation framework that includes periodic population-based surveys can serve as an evaluation model for similar national health education programs.     

The Healthy Young Men's Study: Sampling Methods to Recruit a Random Cohort of Young Men Who Have Sex With Men

Recruiting a scientifically sound cohort of young men who have sex with men (YMSM) is an enduring research challenge. The few cohort studies that have been conducted to date on YMSM have relied on nonprobability sampling methods to construct their cohorts. While these studies have provided valuable information about HIV risk behaviors among YMSM, their generalizability to broader YMSM populations is limited. In this article, the authors describe a venue-based sampling methodology used to recruit a large and diverse cohort of YMSM from public venues in Los Angeles County, California. Venue-based sampling is a multistage probability sampling design that uses standard outreach techniques and standard survey methods to systematically enumerate, sample, and survey hard-to-reach populations. The study design allowed the authors to estimate individual, familial, and interpersonal psychosocial factors associated with HIV risk and health-seeking behaviors for a cohort of YMSM with known properties. Study participants completed an extensive baseline survey and over a 2-year period completed four follow-up surveys at 6-month intervals. The baseline survey was administered in both English and Spanish.     

Editorial: Speaking in Tongues—Diversity in Mixed Methods Research

Pre‐testing is a standard tool to increase appropriateness and to capture the audience, yet rarely is this technique employed for population surveys. This study aimed to examine parental views on the content and methodology of a population child health survey. Forty‐eight Australian families of children aged 0–12 years were interviewed about the content and proposed data collection methods of the draft Victorian Population Survey of Child Health and Wellbeing. Concerns with instructions and items of several commonly used child health measures were identified. Parents preferred face‐to‐face methods rather than telephone interviews due to survey legitimacy; however, 72% of parents indicated that they would participate in a telephone survey. This study provided new findings about respondent views on child health measures and the conditions under which CATI versus face‐to‐face methods for child health data collection methods would be acceptable.     

MAIL SURVEYS FOR ELECTION FORECASTING?: AN EVALUATION OF THE COLUMBUS DISPATCH POLL

Because of slow turnaround time and typically low response rates,mail surveys have generally been considered of little valuein election forecasting. However, statewide mail surveys conductedby the Columbus Dispatch newspaper since 1980 have made remarkablyaccurate forecasts of Ohio election outcomes. In comparisonto statewide surveys by two other organizations employing conventionaltelephone interview methods, the mail surveys were consistentlymore accurate and were generally less susceptible to sourcesof inaccuracy such as high rolloff and low publicity. The mailsurvey's advantage is attributable at least in part to largersample sizes, sampling and response procedures that yieldedmore representative samples of voters, lack of the need to allocateundecided respondents, and superior questionnaire design. Thesefindings suggest that mail surveys not only may be viable alternativesto telephone surveys but may actually be superior to them undersome conditions. Further-more, these results demonstrate thatsurveys with low response rates are not necessarily low in validity.     

Demystifying the use of mail survey in developing countries: a case of Nepal

ABSTRACT

The mail survey is a popular data collection tool in developed countries, but little is known about its use and efficacy in developing countries. Grounded on social exchange and diffusion of innovations theories, this study sought to ascertain whether the mail survey works in Nepal – one of the developing countries, by looking at how long respondents take to respond to mail surveys, and what the determinants are of the response time. A mail survey was conducted in June–September 2012 among 863 village animal health workers. Analysis of the data derived from 500 usable surveys shows that if properly planned and executed, the mail survey will perfectly work in developing countries as well. However, response period greatly varied by respondent’s home region, program sponsor, income groups, but not by gender, age, and education. The paper concludes by outlining suggestions to improve the mail survey.     

Effects of screening questionnaires on response in a two-phase postal survey

For surveys targeting specific population groups, the two-phase postal approach (screener followed by a topical survey sent to eligible households) has been demonstrated to be more effective at identifying population domains of interest than random digit dial telephone methods considering cost, coverage, and response. An important question is how best to motivate screener response from eligible households. In 2011, we conducted a large-scale field test to empirically test a number of methods for motivating response. We fielded screening surveys that varied content-influencing relevance, and also switched screener questionnaires for following up nonrespondents to the initial postal survey – an approach we have labeled responsive tailoring. In another experiment, we tested the effect of asking for first names in the screener questionnaire. In this article, we describe the effects of these experimental treatments on response to both the screener and the topical survey.     

From student to citizen: a survey of students from the People's Republic of China (PRC) in Australia

This article is a survey of students from the People's Republic of China (PRC) who arrived in Australia after 1986 and were still there in 1992. Students from the PRC began to enter Australia in 1972 when China and Australia established diplomatic relations. The numbers were insignificant until 1986 when Australia launched its education export policy.
The article provides statistics and analysis on the motivation of these students, their education and family backgrounds and their present conditions and aspirations. The article also addresses issues such as Australia's education export and imigration policies, the dilemma between political and economic refugees in terms of human rights and the impact of the 1989 Tiananmen Square massacre on PRC students in Australia at the time.
The survey results suggest that the majority of the former PRC students have settled successfully in Australia, though not without considerable difficulties and emotional cost. The question of distinction between political and economic refugees is acknowledged as a very thorny one, and it is suggested that immigration policy based on national interest, and clearly stated, would be more convincing, less hypocritical and therefore in the long run more effective.     

Comparing the effectiveness of respondent-driven sampling and quota sampling in migration research

We evaluate the effectiveness in empirical migration research of the respondent-driven sampling (RDS) and the quota sampling with regard to four criteria: quality of the data; sociometric diversity of the sample; geographic coverage of the sample; and cost-effectiveness. We review two surveys of ex-USSR migrants, conducted simultaneously in the Warsaw metropolitan area, each undertaken with the use of one of the two sampling methods. We find that although the RDS was considerably costlier, it enabled faster data collection and posed a lower risk of underestimating the cost of the survey. We also find that the RDS generated higher-quality data, and that it was more successful in surveying short-term circular migrants. This comparative evaluation contributes to the literature on social research methods in that it links the sampling methods with the quality and character of the data, and helps assess the usefulness and applicability of RDS in empirical migration studies.