Meta-analysis: a case study

This article raises some questions about the usefulness of meta-analysis as a means of reviewing quantitative research in the social sciences. When a meta-analytic model for SAT coaching is used to predict results from future studies, the amount of prediction error is quite large. Interpretations of meta-analytic regressions and quantifications of program and study characteristics are shown to be equivocal. The match between the assumptions of the meta-analytic model and the data from SAT coaching studies is not good, making statistical inferences problematic. Researcher subjectivity is no less problematic in the context of a meta-analysis than in a narrative review.     

Disordered Gambling among College Students: A Meta-Analytic Synthesis

The purpose of this study was to use a meta-analytic procedure to synthesize the rates of disordered gambling for college students that have been reported in the research literature. In order to identify all possible studies that met stringent inclusion criteria, Medline, PsychINFO, and SocioIndex databases were searched with the terms “gambling,” and “college student”. This process resulted in 15 studies concerning gambling among college students that were published through July 2005. To synthesize the 15 studies, a random effects model for meta-analysis was applied. The estimated proportion of disordered gamblers among college students was 7.89%. This estimate is noteworthy because it is higher than that reported for adolescents, college students or adults in a previous study using meta-analytic procedures with studies conducted prior to 1997.     

How family-friendly work environments affect work/family conflict: A meta-analytic examination

We report a meta-analytic examination of the value of five facets of family-friendly work environments (FFWEs) in reducing work/family conflict (WFC). Cumulation of 38 studies (total N = 13,605) suggests that facets of FFWE may provide less assistance to workers in managing WFC than one may hope, as none explained more than seven percent of the variance in WFC. A family-friendly work culture seems most influential in reducing WFC. Importantly, spousal support and FFWEs explain different portions of variance in WFC, suggesting that FFWEs are uniquely valuable to workers in achieving work/ family balance. Implications for research and practice are discussed. When authors reported separate correlations for different subgroups (e.g., males and females), samples, administrations (as in a longitudinal study), or measures of the same construct, those correlations were examined separately. The studies included in this meta-analysis are listed in the references prefixed with an asterisk.     

A characterization of income distributions in terms of generalized Gini coefficients

Most commonly used parametric models for the size distribution of incomes possess only a few finite moments, and hence cannot be characterized by the sequence of their moments. However, all income distributions with a finite mean can be characterized by the sequence of first moments of the order statistics. This is an attractive feature since the generalized Gini coefficients of Kakwani (1980), Donaldson and Weymark (1980, 1983) and Yitzhaki (1983) are simple functions of expectations of sample minima. We present results which streamline these characterizations motivated by Aaberge (2000). Received: 8 March 2001/Accepted: 31 July 2001     

Risk and protective factors, longitudinal research, and bullying prevention

This chapter presents the results from two systematic/meta-analytic reviews of longitudinal studies on the association of school bullying (perpetration and victimization) with adverse health and criminal outcomes later in life. Significant associations between the two predictors and the outcomes are found even after controlling for other major childhood risk factors that are measured before school bullying. The results indicate that effective antibullying programs should be encouraged. They could be viewed as a form of early crime prevention as well as an early form of public health promotion. The findings from a systematic/meta-analytic review on the effectiveness of antibullying programs are also presented. Overall, school-based antibullying programs are effective, leading to an average decrease in bullying of 20 to 23 percent and in victimization of 17 to 20 percent. The chapter emphasizes the lack of prospective longitudinal research in the area of school bullying, which does not allow examination of whether any given factor (individual, family,. or social) is a correlate, a predictor, or a possible cause for bullying. This has important implications for future antibullying initiatives, as well as implications for the refinement of theories of school bullying. It is necessary to extend the framework of the traditional risk-focused approach by incorporating the notion of resiliency and investigating possible protective factors against school bullying and its negative consequences.     

Wilderness adventure therapy effects on the mental health of youth participants

Adventure therapy offers a prevention, early intervention, and treatment modality for people with behavioural, psychological, and psychosocial issues. It can appeal to youth-at-risk who are often less responsive to traditional psychotherapeutic interventions. This study evaluated Wilderness Adventure Therapy (WAT) outcomes based on participants’ pre-program, post-program, and follow-up responses to self-report questionnaires. The sample consisted of 36 adolescent out-patients with mixed mental health issues who completed a 10-week, manualised WAT intervention. The overall short-term standardised mean effect size was small, positive, and statistically significant (0.26), with moderate, statistically significant improvements in psychological resilience and social self-esteem. Total short-term effects were within age-based adventure therapy meta-analytic benchmark 90% confidence intervals, except for the change in suicidality which was lower than the comparable benchmark. The short-term changes were retained at the three-month follow-up, except for family functioning (significant reduction) and suicidality (significant improvement). For participants in clinical ranges pre-program, there was a large, statistically significant reduction in depressive symptomology, and large to very large, statistically significant improvements in behavioural and emotional functioning. These changes were retained at the three-month follow-up. These findings indicate that WAT is as effective as traditional psychotherapy techniques for clinically symptomatic people. Future research utilising a comparison or wait-list control group, multiple sources of data, and a larger sample, could help to qualify and extend these findings.     

DOES SEX STILL MATTER? NEW EVIDENCE FROM THE 1980s

Recent analyses of gender based salary differentials among University faculty have focused on colleges and universities which are quite different from those used in earlier (pre-1975) studies. To facilitate comparisons with previous studies we examine gender earnings differentials during the 1983–84 academic year at Kent State University, a school similar to those studied in the pre-1975 period. Our results suggest that after controlling for differentials in productivity-related characteristics and market influences, there is little evidence of gender based salary discrimination at Kent State during 1983–84.     

A Comparison of Children Living in Single-Mother and Single-Father Families

Research comparing children living in single-mother and single-father families has become important due to the increase in the number of parents contesting custody in divorce cases and as the number of single custodial fathers increases. The present study was designed to investigate a number of characteristics relating to children living in single-father families (SFFs) and in single-mother families (SMFs). Previous research has suggested that in the case of separation and/or divorce the mother is the more competent parent to raise the children. However, recent studies have provided some support for the idea that single fathers can be effective single parents. The subject sample included 42 single divorced custodial parents (21 single mothers and 21 single fathers) and their 62 (6- to 16-year-old) children. The measures employed were The Self-Perception Profile for Children (SPPC; Harter, 1985) and The Child Behaviour Checklist (CBCL; Achenbach & Edelbrock, 1983). One-way MANCOVA and ANCOVA procedures were performed and it was found that the overall scores of children from single-father families (SFFs) did not differ significantly from children in single-mother families (SMFs) on the SPPC and the CBCL. The implications of these findings are discussed and suggestions for future research are provided.     

Increasing the degrees of freedom in existing group randomized trials: the df* approach

This study describes a method for incorporating external estimates of intraclass correlation to improve the precision for the analysis of an existing group-randomized trial. The authors use a random-effects meta-analytic approach to pool the information across studies, which takes into account any interstudy heterogeneity that may exist. This approach can be used in several different situations to estimate the degrees of freedom available for an adjusted test of the intervention effect in a study where the challenges of group-randomized trials were not fully considered when the study was planned. The authors discuss the limitations of this approach and the circumstances in which it is likely to be helpful.     

Critical Thinking in Social Work Education: A Research Synthesis

In a meta-analytic review of critical thinking in social work education, findings revealed variability in research designs, methods, and subsequent findings. The 10 studies reviewed assessed different components of critical thinking and highlighted different potential moderator variables. Although there are significant limitations to all the studies analyzed in this review, findings suggest some practical significance and point to the need for future research to include more rigorous experimental research designs that include control groups and a comprehensive analysis of potential moderator variables that may include socioeconomic status, ethnicity, academic rank, and teaching methodology.     

Testing the immigrant assimilation hypothesis with longitudinal data

This paper is concerned with the comparison of alternative empirical techniques (cross section, synthetic cohort and panel data) for testing the immigrant assimilation hypothesis (IAH). The IAH specifies that immigrants acquire destination relevant human capital, but at a decreasing rate, with duration in the destination. Hence, ceteris paribus, earnings would be expected to increase, at a decreasing rate, with duration. The true assimilation effects may be obscured, however, in analyses of synthetic cohort or panel (longitudinal) data if there are period effects. That is, if the effect on earnings of duration in the destination varies over time. The empirical analysis uses a matched sample of adult male immigrants from the 1983 and 1995 Censuses of Israel. The matched data show that selective exit from the labor force (due to death, absence from the labor force, and inability to match, but not the remigration of immigrants) is associated with lower earnings in 1983. This biases upward the earnings assimilation estimated by the synthetic cohort method. The earnings data are also consistent with the hypothesis that the mass immigration to Israel from 1989 to 1995 raised the return (price) to Israeli-specific human capital among long-duration immigrants sufficiently to more than offset the greater increase in units of destination human capital acquired by more recent immigrants. As a result, long-duration immigrants experienced a steeper increase in earnings from 1983 to 1995. This annulled the true assimilation effects from 1983 to 1995. Longitudinal tests of IAH which assume that the returns to destination-specific skills remained constant (i.e., that there are no period effects) are biased against supporting the IAH if these returns increased.     

Online programs as tools to improve parenting: A meta-analytic review

Background

A number of parenting programs, aimed at improving parenting competencies, have recently been adapted or designed with the use of online technologies. Although web-based services have been claimed to hold promise for parent support, a meta-analytic review of online parenting interventions is lacking.

Method

A systematic review was undertaken of studies (n = 19), published between 2000 and 2010, that describe parenting programs of which the primary components were delivered online. Seven programs were adaptations of traditional, mostly evidence-based, parenting interventions, using the unique opportunities of internet technology. Twelve studies (with in total 54 outcomes, N_tot parents = 1615 and N_tot children = 740) were included in a meta-analysis.

Results

The meta-analysis showed a statistically significant medium effect across parents outcomes (ES = 0.67; se = 0.25) and child outcomes (ES = 0.42; se = 0.15).

Conclusions

The results of this review show that web-based parenting programs with new technologies offer opportunities for sharing social support, consulting professionals and training parental competencies. The meta-analytic results show that guided and self-guided online interventions can make a significant positive contribution for parents and children. The relation with other meta-analyses in the domains of parent education and web-based interventions is discussed.     

Technology, skills, and health care labor markets

Conclusion I examine the role of technological change in health care labor markets. One of the biggest issues in the U.S. economy over the recent past has been rapidly rising health care costs. Conventional wisdom holds the main factor driving these costs increases has been technological change. These changes in technology have lead to a direct transformation in the delivery of health care and have also lead to indirect transformations through the reshaping of the private and public insurance industry. My findings are consistent with the idea that technological change has resulted in an increase in demand for higher skilled workers in the industry, while being labor-saving among low-skill workers. The earnings of RNs and health therapists rose rapidly over the 1983 to 1993 period, declined between 1993 and 1996, but then began to rise again after 1996. It is generally believed that managed care has had a one-time cost reducing effect in the industry by eliminating some of the inefficiencies associated with fee-for-service health insurance (Newhouse, 1992). The results here suggest a similar finding in the labor market. I appreciate helpful comments from Laurence Baker, James Bennett, Ann Frost, Barry Hirsch, Joanne Spetz, and Daphne Taras.     

Work–Family Conflict and Couple Relationship Quality: A Meta-analytic Study

This study examined the association between work–family conflict and couple relationship quality. We conducted a meta-analytic review of 49 samples from 33 papers published between 1986 and 2014. The results indicated that there was a significant negative relationship between work–family conflict and couple relationship quality (r = ?.19, k = 49). Several moderators were included in this analysis: gender, region, parental status, dual-earner status, and the measures used for work–family conflict and marital quality variables. The strength of the relationship varied based on the region of the sample—samples from Europe and Asia had a significantly weaker relationship between work–family conflict and relationship quality than those from North America. In addition, the relationship was significantly weaker in samples of dual-earner couples and when non-standardized scales were used. Implications of the results and directions for future research are suggested.     

KNOWLEDGE OF NONVERBAL CUES,GENDER, AND NONVERBAL DECODING ACCURACY

The Test of Nonverbal Cue Knowledge (TONCK), a paper and pencil test measuring explicit knowledge of nonverbal cue meanings and uses, was developed and found to be reliable and to have predictive ability. In four studies, participants were given the TONCK and one or more tests of accuracy in decoding nonverbal cues the adult faces and voices tests of the Diagnostic Analysis of Nonverbal Accuracy 2 (DANVA) and the video and audio tests of the Profile of Nonverbal Sensitivity (PONS), as well as a test of general cognitive ability (the Wonderlic Personnel Test). Results are reported for the four studies individually and also in a meta-analytic summary. Females scored higher than males on the TONCK and on the PONS, and the TONCK predicted accuracy on the PONS and DANVA. Knowledge of nonverbal cues did not account for the gender difference in decoding ability on the PONS. Cognitive ability was related to the TONCK but did not compromise relations with other variables. The future utility of the TONCK, content specificity of tests, as well as the automaticity of judging cues are discussed.The authors thank Crystal Mileti, Jenna Lavery, Erica Dempsey, and Jared Majerle for their assistance in conducting this research.     

Filipino workers in the Middle East: social profile and policy implications

The characteristics of international labor migrants from the Philippines are examined. "Based on an analysis of a sample of 495 Middle East workers and 1983 worker placements, this paper presents a profile of Filipino workers in the Middle East with reference to such variables as occupation, gender, age and work experience, civil status, education, Philippine residence, household size and number of dependents." (summary in FRE, SPA)     

Effects of Genital Response Measurement Device and Stimulus Characteristics on Sexual Concordance in Women

Investigations of sexual concordance suggest that, on average, women exhibit a low, positive correlation between their subjective sexual arousal and genital response. However, this relationship appears to be stronger, on average, when genital response is measured via the external tissues of the vulva than within the vagina walls. Given the methodological variations, such as stimulus content and duration, in studies of vulvar and vaginal concordance, direct comparisons between individual studies are limited. In the current study, sexual concordance was examined using concurrent measures of vulvar blood flow (using laser Doppler imaging) and vaginal vasocongestion (using vaginal photoplethysmography) to investigate potential differences in vulvar and vaginal concordance. Twenty-five women viewed two erotic films that differed in sexual activity content (foreplay and intercourse) while their subjective sexual arousal and genital response was measured. In support of previous meta-analytic findings, results suggest that subjective sexual arousal is more strongly correlated with vulvar blood flow than with vaginal vasocongestion. Likewise, perceived genital response and actual vulvar blood flow are more strongly related than are vaginal vasocongestion and self-reported genital sensations. Practical implications for the study of women’s sexual concordance as well as broader implications for the understanding of female sexuality are considered.     

The added impact of parenting education in early childhood education programs: A meta-analysis

Many early childhood education (ECE) programs seek to enhance parents' capacities to support their children's development. Using a meta-analytic database of 46 studies of ECE programs that served children age three to five-years-old, we examine the benefits to children's cognitive and pre-academic skills of adding parenting education to ECE programs for children and consider the differential impacts of: 1) parenting education programs of any type; 2) parenting education programs that provided parents with modeling of or opportunities to practice stimulating behaviors and 3) parenting education programs that were delivered through intensive home visiting. The results of the study call into question some general longstanding assertions regarding the benefits of including parenting education in early childhood programs. We find no differences in program impacts between ECE programs that did and did not provide some form of parenting education. We find some suggestive evidence that among ECE programs that provided parenting education, those that provided parents with opportunities to practice parenting skills were associated with greater short-term impacts on children's pre-academic skills. Among ECE programs that provided parenting education, those that did so through one or more home visits a month yielded effect sizes for cognitive outcomes that were significantly larger than programs that provided lower dosages of home visits.     

Indiscriminate data aggregations in meta-analysis. A cause for concern among policy makers and social scientists

Whether you are a policy maker of social scientist, you are slowly being drowned in a sea of often inconsistent research data. Proponents of meta-analysis claim that such data can be objectively and usefully summarized for you. The author notes how the assumptions of the meta-analytic model preclude the synthesis of experimental data (which has a clear cause-and-effect logic) with quasi-experimental and/or nonexperimental data (both of which lack such clarity). Yet in the author's review of 64 recent meta-analytic articles, 11 were found to improperly make such aggregations. Why? The author shows how the guidance provided by the leading proponents of meta-analysis either blurs the distinction or is misleading.