Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey

Declining contact and cooperation rates in random digit dial(RDD) national telephone surveys raise serious concerns aboutthe validity of estimates drawn from such research. While researchin the 1990s indicated that nonresponse bias was relativelysmall, response rates have continued to fall since then. Thecurrent study replicates a 1997 methodological experiment thatcompared results from a "Standard" 5-day survey employing thePew Research Center’s usual methodology with results froma "Rigorous" survey conducted over a much longer field periodand achieving a significantly higher response rate. As withthe 1997 study, there is little to suggest that unit nonresponsewithin the range of response rates obtained seriously threatensthe quality of survey estimates. In 77 out of 84 comparableitems, the two surveys yielded results that were statisticallyindistinguishable. While the "Rigorous" study respondents tendedto be somewhat less politically engaged, they did not reportconsistently different behaviors or attitudes on other kindsof questions. With respect to sample composition, the Standardsurvey was closely aligned with estimates from the U.S. Censusand other large government surveys on most variables. We extendour analysis of nonresponse to include comparisons with thehardest-to-reach respondents and with respondents who terminatedthe interview prior to completion.     

STABILITY AND CHANGE IN THE U.S. PUBLIC'S KNOWLEDGE OF POLITICS

The U.S. public's current knowledge about politics is comparedwith levels of knowlege in the 1940s and 1950s. Fourteen questionsasked by Gallup on various surveys from 1945 to 1957 were includedon a larger survey of political knowledge conducted by telephonein 1989 with a randomly selected sample of 610 adult U.S. residents.On 8 of the 14 items, the percentage answering correctly in1989 was higher than in the earlier surveys (by 4–15 points).One item showed an increase of 1 percent, two were down 1 percent,and three others declined by 5 percent, 9 percent, and 10 percent.When level of education is controlled, however, levels of knowledgeappear to have declined for most of the items. A reanalysisof some of the original Gallup data is used to estimate theeffectiveness of schools in transmitting political informationin 1989 compared with the earlier years.     

Changes in Americans' Policy Preferences, 1935-1979

Using responses to 3,315 survey questions asked of nationalsamples, we examine how policy preferences of Americans havechanged over the last 45 years. The data indicate that therehas been considerable stability in public opinion: responsesto half the 613 repeated policy items show no significant changeat all; approximately half the detectable changes were lessthan 10 percentage points; and rarely did preferences fluctuatesignificantly back and forth within a short time period. Foreignpolicy changes were no larger or more frequent than domestic,but they did tend to occur more abruptly. When opinion shiftsoccurred, they were not random or capricious; they were usuallyrelated to important changes in citizens' social and economicenvironments. Rapid shifts generally coincided with major eventsin international affairs or the economy.     

Problems Relating to Declining Response Rates to Social Survey Research in Japan: Trends After 2000

Abstract: Sociologists in Japan recognize that the circumstances under which social survey research is conducted have been deteriorating and have become critical since 2000. Social survey research is understood as a large-scale survey that mainly draws samples by a random sampling method, using a questionnaire as a measurement tool. The crises of social survey research are summarized as following: (1) the increasing difficulty of conducting accurate sampling and (2) the decline in response rates to drawn samples (planned samples). This paper focuses on the decline in response rates to social survyes by examining recent trends in relation to social surveys conducted by the Japanese government and local government (prefectures). In summary, there has been a landslidedecline of the response rate between 2005 and 2006 in surveys conducted by Japanese government. This decline was thought to mainly come from the declining of response rates among the elderly and women, groups that had previously had high response rates. This seems to be the result of the growing level of the crime involving the abuse of the personal information. However, there did not appear to be a serious decline in the response rate for surveys conducted by local governments. Given that it is not easy to restore response rates, it is important to develop research methods and techniques to cope with the decline. Further, informing respondents of the purpose of the research and its benefit to may help raise response rates.     

The effect of personalization on response rates and data quality in web surveys

Personalizing correspondence has often shown to significantly increase survey response rates in mail surveys. This study experimentally tests whether personalization of email invitations acts correspondingly to web survey response rates. Also, it is investigated whether personalization influences data quality. The results of the study, using a large student sample, show that personalization significantly increases the web survey response rate by 8.6 percentage points. The data quality does not appear to be affected in any major way by personalizing the email invitations. However, the analyses do show that respondents of the personalization condition tend to respond with more social desirability bias to sensitive questions. Therefore, it is concluded that personalization has positive effects on the survey response rate, but one should carefully consider whether or not to personalize when a survey on sensitive topics is conducted.     

A comparative study on the relationship between social networking site use and social capital among Australian and Korean youth

Social network sites (SNS) have become increasingly pervasive in the daily lives of adolescents. This study explores the relationship between SNS use and perceived online social capital among adolescents using survey data from Australia and Korea. We conducted a face-to-face survey of adolescents (12–15-year-olds) in major cities in Australia (N?=?401) and Korea (N?=?644) in 2013. There was no significant difference in time spent on SNS between adolescents in the two countries; however, significant differences in the way adolescents use SNS were found. Australian adolescents tended to use SNS for group activities, whereas Koreans used it for social monitoring. There was a positive relationship between SNS use and online social capital in both countries. However, the types of social capital that were found to have a strong relationship to SNS use were different. Among Australian adolescents, SNS activities had a higher correlation with bonding social capital compared to bridging capital, whereas the result was contrasting among Korean adolescents. The impact of SNS use on online social capital differed between the two countries, showing that Australian adolescents reporting low SNS use gained online social capital if they used SNS more intensely. Finally, the impact of SNS use and culture on perceived online social capital was examined by conducting hierarchical multiple regressions. Interpersonal communication and group activities emerged as significant predictors of online social capital.     

Möglichkeiten und Probleme des Einsatzes postalischer Befragungen

The research focuses on the strategies in mail surveys that lead to high response rates and on the social characteristics of the respondents that are recruited into the survey through reminders. Empirical basis are surveys in the East German cities of Chemnitz, Dresden and Leipzig with an original sample size of ca 2000 persons per city and up to three reminders. The response rate varies between 68 and 70 percent and surpasses the usual response rates in face to face and telephone surveys done in German cities. In contrast to common assumptions in the literature it has no effect on the response rate whether stamps are used on the address and return envelope, neither do original versus scanned signatures make a difference and whether financial incentives are used in the late reminders. Through the reminders especially respondents with low education and of little political interest are recruited into the survey.     

Assessing the Reliability and Validity of the Jump$tart Survey of Financial Literacy

Financial education represents an area of popular interest, owing largely to the Jump$tart surveys of financial literacy. However, while the surveys represent indicators of financial knowledge among high school seniors, these measures have not been statistically validated. This article describes an assessment of the surveys reliability (internal consistency), and validity. It reports a moderately high degree of consistency overall, however, discloses low to moderate internal consistencies among subscales. It also finds significant response differences to one quarter of comparable items between surveys. The researcher observes challenges to affirming the surveys validity and offers statistics suggesting social bias among survey items. He calls for further research into measures of financial literacy.The author appreciates the guidance and support of Dr. Duane Giannangelo, Dr. Jean Steiz, Dr. Lou Franceschini, and Dr. Steven Ross.     

Job transfers: a neglected aspect of migration in Malaysia

"This article examines the incidence of [job-related] transfers in Malaysia. The retrospective migration data from the Malaysia Family Life Survey [conducted in 1976-1977] are used to demonstrate that transfers comprise 18 percent of all migration in the country and that there has been a rise in the incidence of transfers over a 35 year period. Factors underlying this trend are outlined. Furthermore, significant differences in age, educational attainment and other characteristics between transferees and other types of migrants are identified and their implications discussed."     

Assessment of Cigarette Smoking and Smoking Susceptibility among Youth: Telephone Computer-Assisted Self-Interviews versus Computer-Assisted Telephone Interviews

This experimental study assesses the effect of two survey methods,telephone audio computer-assisted self-interviewing (T-ACASI)and computer-assisted telephone interviewing (CATI), on self-reportsof smoking behavior and smoking susceptibility among adolescents12–17 years of age in California. In T-ACASI, participantslisten to prerecorded, computer-controlled questions and respondby pressing the keypad on a touch-tone telephone. In CATI, interviewersadminister the questions and enter responses into a computer.Prior research suggests that youth may be more likely to reportsensitive behaviors in a self-administered survey like T-ACASIcompared to an interviewer-administered survey like CATI, dueto greater perceived confidentiality. Logistic regression analyseswere conducted on unweighted data, controlling for demographicdifferences. Adjusted estimates of current smoking (past 30days) were significantly greater in T-ACASI (8.3 percent) thanCATI (4.5 percent). Smoking susceptibility (i.e., lack of afirm commitment not to smoke among those who have never smoked)was also greater in T-ACASI (45.0 percent) than CATI (34.9 percent).In both surveys, social desirability response bias was negativelyassociated with smoking, which suggests that response bias wasproblematic for both modes. Many respondents reported that aparent was present during the interview (59.4 percent in CATI;42.0 percent in T-ACASI). In both surveys, parental presencewas negatively associated with smoking, which suggests thatthis factor could also contribute to underreporting. Applicationof sample weights to the data eliminated the survey mode effects;however, the CATI current smoking estimate (9.3 percent) fromthis study was significantly less than an estimate (14.2 percent)obtained from a self-administered, school-based survey conductedthe same year on California adolescents.     

RACE-OF-INTERVIEWER EFFECTS IN A PREELECTION POLL VIRGINIA 1989

All published preelection surveys of the 1989 Virginia gubernatorialcontest overestimated the vote share of the black candidateand eventual victor, Douglas Wilder. We offer a "social desirability"interpretation of the polls' inaccuracies and hypothesize thatclaiming support for Wilder was the socially desirable responsefor some whites, especially when the interviewer was black.We show a race-of-interviewer effect on the vote intention ofwhite respondents of 8–11 percentage points in a preelectionsurvey of Virginia voters. The effects were greatest among whiteDemocrats and among whites who were more uncertain of theirvote intention. We discuss the implications of these findingsfor race-of-interviewer research and for improving the accuracyof preelection forecasts in contests with black and white opposingcandidates.     

Demystifying the use of mail survey in developing countries: a case of Nepal

ABSTRACT

The mail survey is a popular data collection tool in developed countries, but little is known about its use and efficacy in developing countries. Grounded on social exchange and diffusion of innovations theories, this study sought to ascertain whether the mail survey works in Nepal – one of the developing countries, by looking at how long respondents take to respond to mail surveys, and what the determinants are of the response time. A mail survey was conducted in June–September 2012 among 863 village animal health workers. Analysis of the data derived from 500 usable surveys shows that if properly planned and executed, the mail survey will perfectly work in developing countries as well. However, response period greatly varied by respondent’s home region, program sponsor, income groups, but not by gender, age, and education. The paper concludes by outlining suggestions to improve the mail survey.     

Inheritances and the distribution of wealth or whatever happened to the great inheritance boom?

Using data from the Survey of Consumer Finances (SCF), we found that on average over the period from 1989 to 2007, about one fifth of American households at a given point of time reported a wealth transfer and these accounted for quite a sizeable figure, about a quarter of their net worth. Over the lifetime, about 30 percent of households could expect to receive a wealth transfer and these would account for close to 40 % of their net worth near time of death. However, there is little evidence of an inheritance “boom.” In fact, from 1989 to 2007, the share of households reporting a wealth transfer fell by 2.5 percentage points, a time trend statistically significant at the one percent level. The average value of inheritances received among all households did increase but at a slow pace, by 10 %; the time trend is not statistically significant. Wealth transfers as a proportion of current net worth fell sharply over this period, from 29 to 19 %, though the time trend once again is not statistically significant. We also found that inheritances and other wealth transfers tend to be equalizing in terms of the distribution of household wealth, though a number of caveats apply to this result.     

Attendance problems and disciplinary procedures in Nebraska schools

STUDY PURPOSE: School principals were surveyed in rural and urban Nebraska schools to compare policies and procedures on school attendance, and to contrast the use of disciplinary procedures for attendance, violence and substance abuse. METHODS: A survey was sent to a 50 percent random sample of Nebraska schools. Respondent school addresses in metropolitan statistical areas (MSA) areas were categorized as urban, and non-MSA respondents were classified as rural area schools. RESULTS: Of the 680 surveys mailed, 464 (68.2 percent) completed surveys were returned. Overall, 86.2 percent of respondents were from rural schools and 13.8 percent from urban schools. A list of disciplinary problems and procedures was reported for a first-time offense by rural and urban schools. Both an "occasional attendance problem" and a "chronic attendance problem" were most frequently addressed through parent contact in both urban and rural areas. For "fighting with another student," rural schools most commonly reported discipline by teacher, followed by discipline by principal and parent contact. In urban schools, discipline by principal was most common. For "using drugs other than alcohol or tobacco at school" in both areas, disciplinary procedures included parent contact and short-term suspension. A list of disciplinary problems and procedures was also reported for a second-time offense and third-time offense. For "fighting with another student," the most commonly reported procedure was discipline by principal and parent contact for a second-time offense, followed by short-term suspension and long-term suspension for a third-time offense. For "using drugs other than alcohol or tobacco at school," for a second time offense was long-term suspension and semester expulsion for a third-time offense. Principals were also asked in an open-response format, what program or procedures were currently in place to prevent violence in their school. Common responses were "conflict resolution training," followed by "the Boys Town social skills and behaviors model" and "a zero tolerance policy." CONCLUSIONS: School principals generally reported similar disciplinary actions for most disciplinary problems. For recurrent offenses and for serious problems, principals generally used tougher disciplinary procedures. Some of the discrepancies in disciplinary actions, however, suggest the need for consistent enforcement and communication of policies for both urban and rural schools.     

I'm Not Prejudiced,But…: A Deeper Look at Racial Attitudes*

Black and white people experience race relations from different social positions, thus, interpret questions about race relations differently. Structured interviews were conducted with a random sample of residents from Philadelphia and Meridian, Mississippi. These data indicate that there is a wider gap between these blacks and whites regarding their perceptions of discrimination than the gap documented by national surveys. There are interesting differences in the way the two groups interpret themselves as racist, or not, and how they remember their communities’interracial history. Content analysis is used to demonstrate differences in the understanding of race relations, segregation, and perceived danger from racially motivated violence.     

Characterizing urban butterfly populations: the case for purposive point-count surveys

Developing effective butterfly monitoring strategies is key to understanding how butterflies interact with urban environments, and, in turn, to developing local conservation practices. We investigated two urban habitat types (public gardens and restored/reconstructed prairies) and compared three survey methods (Pollard transects, purposive point counts, and random point counts) to determine which was most productive for detecting butterflies and assessing family diversity. We conducted 66 butterfly surveys by using each method (198 total) from May through September in 2015 and 2016 at six sites (three public gardens and three prairies) in Ames, Ankeny and Des Moines, Iowa. All survey methods were used on 11 sampling dates at each site. Overall, we observed 2,227 butterflies representing 38 species: 1,076 in public gardens and 1,151 in prairie areas. We used a smaller data set standardized for survey effort, including 1,361 of these sightings, to compare survey methods and habitat types. Although there were no significant differences in number of butterfly sightings between the two habitats, more sightings (798) were documented by using purposive point counts when compared to Pollard transects (297) or random point counts (266) (for both comparisons, p?<?0.0001). Occupancy modeling also indicated that purposive point counts were most effective in detecting certain species of butterflies, most notably those within the Pieridae (whites, sulphurs) and Papilionidae (swallowtails). We conclude that public gardens and restored/reconstructed prairies in urban settings can provide important butterfly habitat, and that purposive point-count surveys are most effective for detecting butterflies in these relatively small-scale landscape features.

     

A Study of the Properties of the Item Count Technique

The item count technique is an indirect questioning techniquethat is used to estimate the proportion of people who have engagedin stigmatizing behavior. This technique is expected to yielda more appropriate estimate than the ordinary direct questioningtechnique because it requests respondents to indicate, basedon a list of several items, simply the number of items thatare applicable to them, including the target key item. An experimentalweb survey was conducted in an attempt to compare the directquestioning technique and the item count technique. Comparedwith the direct questioning technique, the item count techniqueyielded higher estimates of the proportion of shoplifters bynearly 10 percentage points, whereas the difference betweenthe estimates using these two techniques was mostly insignificantwith respect to innocuous blood donation. The survey resultssuggest that in the item count technique respondents tend toreport fewer total behaviors compared to the direct questioncase. This tendency is more pronounced in the case of longeritem lists. Three domain estimators for the item count techniquewere compared, and the cross-based method appeared to be themost appropriate method. Large differences in domain estimatesfor shoplifting between the item count and direct questioningtechniques were found among female respondents, middle-agedrespondents, respondents living in urban areas, and highly-educatedrespondents.     

Women and pensions: a decade of progress?

This Issue Brief compares changes from 1989 to 1998 in pension participation, accumulation, and allocation for employed women, versus employed men, ages 18-62. In addition, it provides an estimate of the gender "pension gap" in defined contribution accumulations, contrasts this with the gender "earnings gap," and provides explanations for these differences. Between 1989 and 1998, the percentage of employed women with a pension or retirement plan at their current job increased from 43 percent to 45 percent, compared with a decline from 53 percent to 52 percent for employed men. For both women and men, the percentage with defined contribution retirement plans increased dramatically, while the percentage with defined benefit pension plans dropped sharply. Between 1989 and 1998, the ratio of women's to men's defined contribution plan accumulations increased from 40 percent to 44 percent, indicating a narrowing of the gender pension gap. However, the narrowing was concentrated among the cohort ages 45-53 in 1998. The gender pension gap increased for women in other age groups. Gender differences in defined contribution plan accumulations can be attributed to differences in earnings and job characteristics. Between 1989 and 1998, for workers with defined contribution plans, the ratio of women's to men's earnings remained unchanged at 57 percent. Employed women with defined contribution plans are more than twice as likely to earn less than $25,000 per year than employed men with defined contribution plans, but almost five times less likely to earn more than $100,000 per year. From 1989 to 1998, the percentage of employed men with defined contribution balances invested mostly in low-risk, low-return assets declined much more than the percentage of employed women who followed that investment strategy. Whereas the percentages of men and women with retirement plans invested mostly in bonds were nearly equal at 31 percent and 32 percent in 1989, respectively, by 1998, 20 percent of women (compared with 14 percent of men) had their retirement plans invested mostly in bonds. The trend toward defined contribution plans and riskier retirement portfolios has resulted in significant wealth accumulation over the decade. In real terms, both men and women have greater retirement plan wealth, but increases have been larger for men than for women. Since there is no evidence that plan provisions vary by gender, improvements in the gender pension gap will come only with changes in women's labor force experience and investment decision-making.     

Factors associated with the use of evidence-based practices to treat psychological trauma by psychotherapists with trauma treatment expertise

This paper investigates 10 socio-demographic and case characteristic variables as predictors of use of evidence-based practice and non-evidence-based practice in the treatment of psychological trauma. A national random sample of 2,400 trauma treatment specialists in all 50 states and the District of Columbia were sent surveys with a response rate of 29.6% (N = 711) usable surveys returned. Stepwise regressions conducted on evidence-based practice use indicated that special trauma training, older age, and higher percentage of PTSD on the case load were the only significant predictors of evidence-based practice use. Implications for trauma practices are indicated.     

Personal account retirement plans: an analysis of the survey of consumers finances

This report is based on data from the Federal Reserve Board's triennial Survey of Consumer Financies (SCF), which provides the most comprehensive data available on the wealth of American households. The most recent SCF data are for 1998, and this report tracks information from the 1992, 1995, and 1998 surveys. The percentage of families with a participant in a pension plan from a current job increased from 38.8 percent to 41.0 percent over this six-year period. If one focuses exclusively on those families with a worker and in which the head is under age 65, the percentage for 1998 increases to 56.8 percent. The previously documented trend toward defined contribution plans was confirmed and the significance of 401(k)-type plans for those families participating in a pension plan more than doubled, from 31.6 percent in 1992 to 64.3 percent in 1998. The percentage of family heads eligible to participate in a defined contribution plan that did so increased from 73.8 percent in 1995 to 77.3 percent in 1998. Of those families choosing not to participate in a defined contribution plan, 40.3 percent were already participating in a defined benefit plan. Overall, "personal account plans" represented nearly one-half (49.5 percent) of all the financial assets for those families with either a defined contribution plan account, IRA, or Keogh, in 1998. This was a significant increase from 43.6 percent in 1992. The average total account balance in personal account plans for families with a plan in 1998 was $78,417, an increase of 54 percent in real terms over the 1992 balance of $50,914 (expressed in 1998 dollars). For families participating in a defined contribution plan, IRA, or Keogh in 1998, 52 percent of the overall average was attributed to IRA/Keogh balances (43 percent from IRAs alone), and 44 percent of the average was from account balances in defined contribution plans with the current employer. There is a marked tendency for lower-income families to have larger percentages of their total personal account plan retirement portfolio in IRAs, although this trend appears to be fading with time. The impact of rollovers on the average total account balance for all individual account balances appears to be quite large: $152,451 for those with at least one rollover, versus $78,471 for all families participating in at least one personal account plan, regardless of whether they have had a rollover. The vast majority of the rollovers would appear to be going to IRAs, as opposed to a defined contribution plan with a new employer.