Using cognitive interviews to refine translated survey questions: an example from a cross-national crime survey

Lately, social researchers have become increasingly interested and involved in comparative and cross-national research. In part this reflects a growing realisation that key social, economic and political processes are not merely ‘domestic’ but are common across countries and cultures. Although numerous good quality data sets now exist (e.g. the European Social Survey or International Social Survey Programme), in many instances researchers need to design their own bespoke comparative surveys. Such surveys require careful design if they are to produce meaningful data. Herein we introduce, outline and report on one technique which may usefully be employed in order to design better survey items for use in cross-national surveys.     

Cross-national variation in political network size,distribution, and prediction

Network size has a fundamental influence on other network properties. As studies of social network size have accumulated beyond the U.S. and Western Europe, diversity in the networks examined and methods used to construct size estimates have hindered the ability to make direct cross-national comparisons. We employ data from a summary political discussion network size measure in 17 surveys conducted in 15 countries between 2012 and 2018. We offer improved cross-national data on political network size distributions and estimate aggregate and country-specific models to understand the factors, both social and political, that predict political network size within countries.     

Who Is a Part-Time Worker Around the World and Why Does It Matter? Examining the Quality of Employment Measures and Workers’ Perceived Job Quality

Theory and research do not fully account for the cross-national variation in part-time work definitions and measures, which may affect conclusions. Using the 2004 to 2009 International Social Survey Programme (ISSP) as a special case, this study analyzes the reliability of different part-time work measures for 47 countries, finding high levels of consistency between what resembles a country-specific measure and measures based on 30- and 35-hr thresholds. Bivariate analyses reveal that sex is consistently related to—and suggestive of the construct validity of—part-time measures, except in some non–Organisation for Economic Co-operation and Development (OECD) countries. Analyses of selected countries in 1 year indicate that the choice of part-time work measure matters for understanding workers’ perceptions of their earnings, advancement opportunities, and job security in a few countries. Our results yield good news for ISSP users but highlight the need for the more exhaustive conceptualization of part-time work we offer, research outside of the OECD, and care in interpreting data in some countries.     

The Role of Beliefs in Sexual Behavior of Adolescents: Development and Validation of an Adolescent Sexual Expectancies Scale

This article reports on the development and psychometric properties of the Adolescent Sexual Expectancies Scale (ASEXS). Data were obtained from three annual longitudinal surveys of youth aged 10–17 at the first administration (N=932 at Wave 3). Confirmatory factor analyses indicated that 4 correlated factors corresponding to Social Risk, Social Benefit, Health Risk, and Pleasure adequately represented the expectancy items. High alpha coefficients and stability across survey waves indicated good internal and test–retest reliability for these factors. Latent variable autoregressive analyses indicated that Social Risk expectancies predicted changes in sexual behavior over the course of the study. Multigroup analyses established that the measurement and predictive models were equivalent for males and females and across age groups. The ASEXS provides researchers with a useful measure for examining adolescents' expectancy beliefs about the consequences of sexual behavior.     

Literacy Gaps by Educational Attainment: A Cross-National Analysis

Existing cross-national research on educational attainment does not fully address whether the same level of educational attainment generates the same level of literacy skills in different countries. We analyze literacy skills data for young adults from 19 countries in the 1994-1998 International Adult Literacy Survey and find that in all countries, individuals with a higher level of educational attainment tend to have greater literacy skills. However, there is substantial variation across countries in the size of literacy gaps by levels of educational attainment. In particular, young adults in the United States show the largest literacy gaps. Using two-level hierarchical linear models, we find that cross-national differences in the literacy gap between more- and less-educated individuals are systematically linked to the degree of between-school inequality in school resources (instructional materials, class size, teachers' experience and certification).     

No two ageisms are the same: testing measurement invariance in ageism experience across Europe

The article focuses on the topic of measurement invariance (equivalence) testing in comparative research. Measurement invariance is the basic requirement for comparative data and should be addressed by researchers before any international comparison begins. In the article, multiple-group confirmatory factor analysis is first performed to describe three methods of measurement invariance testing (configural, metric, and scalar). Subsequently, this procedure is applied to the case of measuring experience with ageism against seniors in the quantitative European Social Survey, Round 4. Measurement invariance in the concept is tested across 29 states participating in the survey. Hypotheses about the sources of the lack of measurement invariance between some countries are then tested by comparing the parameters of the models identified for different groups of European states. Finally, evidence of international comparability of items measuring experience with ageism against seniors is summarized and implications for other areas of comparative social research are discussed.     

Comparison of indirect and direct approaches to measuring patient satisfaction

One indirect (PSQ) and two direct (CSQ-18B and ERS) measures of patient satisfaction were compared across a series of psychometric, acceptability, and concurrent validity criteria. None of the three measures was significantly related to health status or demographic characteristics. Each measure performed as expected from prior research on their psychometric properties. The scales contrasted, however, in regard to acceptability, with the PSQ being much less acceptable to patients than the other two. The PSQ also produced more missing data. The results on the PSQ reflected patient inconsistency on cross-check items and patients tended to be uncertain in their responses to many items. In multivariate analyses the three measures were used as predictors of patient-rated indices of global service satisfaction. The CSQ-18B and the ERS were significant predictors of patient-rated indices of global service satisfaction, whereas the PSQ was unrelated to these indices. It was concluded that the direct and indirect approaches measure different satisfaction domains. The PSQ likely assesses more generalized attitudes about health services while the CSQ-18B and the ERS efficiently reflect opinions about the specific setting in which they are administered.     

Measuring economic competence of youth with a short scale

We present a 12-item scale measuring the cognitive component of economic competence and document the psychometric properties of the scale. Using a data set with >12,000 secondary school students in Germany, the scale shows high discriminatory power and covers a wide range of ability levels. Analyses of ‘Differential Item Functioninǵ show no item bias across key demographic characteristics, and scores show meaningful associations with scores obtained from adjacent test instruments. Student-level correlates mirror estimates documented in earlier literature as well as results relying on a more extensive version of the scale with 31 items. The presented short scale enables researchers and practitioners to efficiently measure economic competence of youth.     

Services for the elderly in Europe: a cross-national survey

In 1989, a cross-national comparative survey on services for the elderly in Europe was initiated and funded by the Commission of the European Communities. It included the 12 countries of the current European Community (EC). Those services for the elderly studied were classified as temporary vs. permanent and conventional vs. innovative residential and community services. This article discusses developments within the EC relevant to public policymaking on aging and reports on the services for the elderly examined by the survey. It also delineates various new tasks for the EC to safeguard the appropriate use of services and thereby promote a dignified old age for all Europeans involved.     

Services for the Elderly in Europe:

In 1989, a cross-national comparative survey on services for the elderly in Europe was initiated and funded by the Commission of the European communities. It included the 12 countries of the current European Community (EC). Those services for the elderly studied were classified as temporary vs. permanent and conventional vs. innovative residential and community services. This article discusses developments within the EC relevant to public policymaking on aging and reports on the services for the elderly examined by the survey. It also delineates various new tasks for the EC to safeguard the appropriate use of services and thereby promote a dignified old age for all Europeans involved.     

National Culture on the Cross-National Variation of Homicide: An Empirical Application of the Inglehart–Welzel Cultural Map

Although there has been a growing literature on the effects of culture on the cross-national variation of homicide, this literature remains limited in the operationalization of national culture as well as in the modeling of the cultural effects. Adopting a multidimensional measure of national culture developed in the World Values Survey, this study examines the effects of various aspects of national culture, as well as their interaction, on the cross-national variation of homicide. The findings of this study provide evidence for the effect of national culture on homicide variation across countries while painting a more complex picture about the potential mechanisms of these effects.     

Does National Identification Always Lead to Chauvinism? A Cross-national Analysis of Contextual Explanations

This study examines the premise that national identification inevitably leads citizens to endorse chauvinism in the international arena. It argues that the relationship between national identification and chauvinism is affected by three contextual factors: globalization, inter-state conflict, and social division. A multilevel analysis of cross-national survey data from the International Social Survey Program National Identity II (2003) was employed in order to observe these three contextual effects, the sample consisting of 33 countries. The results demonstrate that closer relations exist between national identification and chauvinism in more globalized countries. The effects of inter-state conflict vary according to the indices used to measure conflict. Those countries that experienced a large number of deaths during the 2000s and mobilized resources and personnel evinced a higher level of chauvinism. This effect is not evidenced by other measures of conflict, however. Social division did not affect the level of chauvinism or its relation with national identification. These findings are used to critically evaluate the notion that citizens who identify with their nation tend to be nationally chauvinistic.     

Intergenerational Mobility in East Asian Countries: A Comparative Study of Japan, Korea and China

Abstract:  There have been fewer cross-national comparative works on social mobility in East Asian countries than in Western countries. The present study aims to explore the similarities and differences of intergenerational social mobility in three East Asian countries and to examine whether the Featherman–Jones–Hauser (FJH) hypothesis would fit well into the mobility tables for Japan, Korea, and China. Log-linear and log-multiplicative models are applied to the mobility tables of respondents' occupations according to the father's occupation in the three countries. The analysis of relative rates of mobility did not completely support the FJH hypothesis in a strict sense, even though it reasonably accounts for the total association of origin and destination in the three countries. My analysis demonstrates that a level of social fluidity has been higher in Korea and China than in Japan. Focusing on how relative mobility patterns differ between the three countries, intergenerational mobility in Korea is characterized by lower rates of class inheritance compared to Japan, while there seems to be a similarity between the patterns of social fluidity in Japan and Korea, even though they are to a certain extent deviant from Erikson and Goldthorpe's core model. In contrast to Korea, class inheritance rates in China are almost as large as in Japan. However, the mobility pattern between classes appears to differ substantially between the post-socialist and two capitalist countries. In particular, both downward and upward mobility between the white collar bloc and the unskilled manual position are more pervasive in China than in Japan and Korea. These results imply that sociologists interested in cross-national comparison of the social mobility between the East Asian countries should pay attention to both political institutions and the local labor market situatuion which can substantially affect social mobility.     

Sexual Self-Concept and Sexually Abused Males: Early Data on the Development of a Scale

ABSTRACT

This article describes the early development and psychometric piloting of the Sexual Self-Concept Scale for Sexually Abused Males, a one-dimensional measure assessing the gender-based self-perceptions of males with histories of childhood sexual abuse (CSA). A sample of 459 males with histories of CSA completed a demographic survey and responded to a pool of items using a Likert-type scale. Strong and promising evidence was rendered for face validity, content validity, convergent validity, criterion validity, discriminant validity, test-retest reliability, split-half reliability, internal consistency and item-total correlations. The scale may be used by helping professionals and clients to identify beliefs, emotions and behavior emanating from CSA with respect to gender-based sexual self-concept, design interventions to modify or ameliorate such effects, and to mark progress across phases of treatment.     

The development of a new measure for work-family research: Community resource fit

We describe the development and validation of a quantitative measure of community resource fit; i.e., satisfaction with the extent to which community resources meet the needs of working families of school-aged children. The measure has good psychometric properties, and preliminary results suggest that the measure warrants further study. The measure is composed of six moderately intercorrelated subscales assessing resource fit in the areas of work, public transportation, school, school transportation, after-school programs and after-school transportation resources. We found interesting patterns of results linking community resource fit, especially in the areas of work and school resource fit, to a variety of quality-of-life and well-being outcomes among employed parents of school-aged children. These outcomes include work-to-family and family-to-work conflict and enhancement, psychological distress, job–role quality, likelihood of losing or leaving one's job and likelihood of leaving one's line of work.     

Comparing ‘deep’ insider knowledge: developing analytical strategies for cross-national qualitative studies

ABSTRACT

One overarching question in scholarly methodological discussions on qualitative comparative approaches concerns how it is possible to compare and generalise deep insider knowledge across (nationally) specific contexts. The aim of this article is to propose a research strategy that both facilitates the comparison and theorisation of such knowledge across nations and limits the risks of reproducing naturalised national ‘truths’. The strategy is developed within a feminist, cross-national, qualitative comparative analysis of how European countries addressed military deaths in connection with the ISAF mission in Afghanistan. The article underlines the importance of collective analytical work and of strategies that allow continuous movement between insider and outsider positions throughout the research process. A number of analytical strategies are presented: collective project design, alternating between analytical closeness and distance, and de-familiarising writing practices.     

The Recalled Childhood Gender Questionnaire-Revised: a psychometric analysis in a sample of women with congenital adrenal hyperplasia

We administered the 18-item Recalled Childhood Gender Questionnaire-Revised (RCGQ-R), female version, to 147 adult women with congenital adrenal hyperplasia (CAH) representing three different degrees of prenatal androgenization due to 21-hydroxylase deficiency and to non-CAH controls. A principal components analysis generated three components accounting for 46%, 9%, and 6% of the variance, respectively. Corresponding unit-weighted scales (high scores = feminine) were labeled Gender Role (13 items; Cronbach alpha = .91), Physical Activity (3 items; alpha = .64), and Cross-Gender Desire (2 items; alpha = .47). Discriminant validity was demonstrated in terms of highly significant comparisons across the four groups. We conclude that the first 2 RCGQ-R scales show good psychometric qualities, but that the third scale needs to be further evaluated in a sample that includes women with gender identity disorder.     

The Causal Effects of the Number of Children on Female Employment - Do European Institutional and Gender Conditions Matter?

This paper contributes to the discussion on the effects of the number of children on female employment in Europe. Most previous research has either (1) compared these effects across countries, assuming an exogeneity of family size; or (2) used methods that dealt with endogeneity of family size, but that focused on single countries. We combine these two approaches by taking a cross-country comparative perspective and applying quasi-experimental methods. We use instrumental variable models, with multiple births as instruments, and the harmonized data from the European Survey on Income and Living Conditions (EU-SILC). We examine the cross-country variation in the effects of family size on maternal employment across groups of European countries with different welfare state regimes. This step gives us an opportunity to investigate whether the revealed cross-country differences in the magnitude of the effect of the family size on maternal employment can be attributed to the diversity of European institutional arrangements, as well as the cultural and the structural conditions for combining work and family duties.     

The Secretary as Supporter,Team Worker and Independent: A Case for Societal Comparison

Recent developments in cross-national comparisons of women's work have led to some new and challenging findings. This article aims to contribute to this debate by presenting one study which examined secretarial work in three European countries. The methodological issues connected with cross-national research on gender are addressed, and it is suggested that the ‘societal’ approach offers the most potential for cross-national gender research. Findings are discussed in the areas of demographics, education and training, job content and mobility, and interlinkages between national institutions and the secretarial occupation are explored. Differences between the national samples highlight the importance of carrying out cross-national research in this area.     

Which probes are most useful when undertaking cognitive interviews?

This paper reports the use of verbal probes in cognitive interviews (CIs) undertaken to test the usefulness, validity and reliability of survey questions. Through examining the use of probes by three interviewers undertaking interviews as part the piloting of a cross-national crime survey, we examine which of the various types of probes used in CIs produce the most useful information. Other influences on interview quality are examined, including differences between interviewers and respondents themselves. The analyses rely on multi-level modelling and suggest that anticipated, emergent and conditional probes provide the most useful data. Furthermore, age, gender and educational levels appear to have no bearing on the quality of the data generated.