Collaborative evaluation of a high school prevention curriculum: How methods of collaborative evaluation enhanced a randomized control trial to inform program improvement

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome variables that can be used for program improvement. Throughout this project there were many examples of client learning from evaluator and evaluator learning from client. This article presents convincing evidence that collaborative evaluation can improve the design, implementation, and findings of the randomized control trial. Throughout this paper, we discuss many examples of good science, good evaluation, and other practical benefits of practicing collaborative evaluation. Ultimately, the authors created the term pre-formative evaluation to describe the period prior to data collection and before program implementation, when collaborative evaluation can inform program improvement.     

An approach to design long-term monitoring and evaluation frameworks in multi-actor systems--a case in water management

Learning-by-doing and adaptive management require careful monitoring and evaluation of the outcomes of environmental policies and programs under implementation. Selecting relevant indicators is difficult, especially when monitoring over a longer period of time. Further challenges arise when policies are developed as a collaborative effort among multiple actors. This paper discusses an approach to design frameworks for long-term monitoring and evaluation in multi-actor systems. It uses Dynamic Actor Network Analysis (DANA) as an actor-sensitive method to reconstruct program theories. This is combined with elements of assumption-based planning to identify critical assumptions and associated indicators to incorporate the dynamic aspects related to long-term monitoring. An application of this approach is described for a case of water management in the Netherlands. Here, mapping multiple perspectives and identifying critical assumptions helped to broaden the scope of monitoring in important ways. Identifying associated indicators and expectations on their development in response to policy implementation proved more difficult. From this case, it can be concluded that the approach is feasible, useful, but also demanding. However, with continuing trends of networked governance and adaptive management, additional efforts to reflect these trends in monitoring and evaluation, through this and similar approaches, are needed.     

Facilitating Mental Health Service Use for Caregivers: Referral Strategies among Child Welfare Caseworkers

Unmet needs for mental health care are common among caregivers involved in the child welfare system. Although child welfare caseworkers are well positioned to identify service needs and refer caregivers to treatment, little is known about the types of referral strategies used in practice, or their effectiveness for promoting mental health service use. The current study examined child welfare caseworkers' use of different referral strategies and the extent to which these strategies are associated with caregivers' receipt of mental health services within a national sample of child welfare cases. Analyses of the second cohort of families from the National Survey of Child and Adolescent Well-Being suggest that child welfare workers more often use informational strategies for referring caregivers, including suggesting treatment or providing information about treatment options. However, social referral strategies such as providing caregivers with direct assistance in completing applications and making and attending appointments were associated with a greater likelihood of caregivers receiving mental health services. Findings support evidence from other service contexts that service use is facilitated by caseworkers' direct support for arranging services. Implications for research and for child welfare managers and administrators are discussed.     

Academic Performance of Subsequent Schools and Impacts of Early Interventions: Evidence from a Randomized Controlled Trial in Head Start Settings

The role of subsequent school contexts in the long-term effects of early childhood interventions has received increasing attention, but has been understudied in the literature. Using data from the Chicago School Readiness Project (CSRP), a cluster-randomized controlled trial conducted in Head Start programs, we investigate whether the intervention had differential effects on academic and behavioral outcomes in kindergarten if children attended high- or low-performing schools subsequent to the preschool intervention year. To address the issue of selection bias, we adopt an innovative method, principal score matching, and control for a set of child, mother, and classroom covariates. We find that exposure to the CSRP intervention in the Head Start year had significant effects on academic and behavioral outcomes in kindergarten for children who subsequently attended high-performing schools, but no significant effects on children attending low-performing schools. Policy implications of the findings are discussed.     

Family therapy for drug abuse: review and updates 2003-2010

Just 15 years ago, Liddle and Dakof (Journal of Marital and Family Therapy, 1995; 21, 511) concluded, based on the available evidence, that family therapy represented a "promising, but not definitive" approach for the treatment of drug problems among adolescents and adults. Seven years later, Rowe and Liddle (2003) review described considerable progress in this specialty with encouraging findings on adolescent-focused models based on rigorous methodology, as well as advances with adult-focused family-based treatments. The current review brings the field up to date with highlights from research conducted in the intervening 7 years, cross-cutting issues, recommendations for new research, and practice implications of these findings. Adolescent-focused family-based models that attend to the ecology of the teen and family show the most consistent and strongest findings in recent studies. Adult-focused models based on behavioral and systems theories of change also show strong effects with drug abusers and their families. The overarching conclusion is that family-based models are not only a viable treatment alternative for the treatment of drug abuse, but are now consistently recognized among the most effective approaches for treating both adults and adolescents with drug problems.     

Couple and family interventions in health problems

Intervention research for couples and families managing chronic health problems is in an early developmental stage. We reviewed randomized clinical trials of family interventions for common neurological diseases, cardiovascular diseases, cancer, and diabetes, which is similar to the content of previous reviews discussed later. One overriding theme of these studies is that patients with chronic illnesses and their families face a variety of challenges to which researchers have responded with an array of treatment modalities. Very few of the interventions reviewed, with the exception of treatment for adolescents with diabetes, tested family psychotherapy models. Most interventions were time-limited therapeutic interventions that trained families to improve their communication and problem-solving skills, individual and family coping skills, and medical management. Researchers more clearly described mechanisms of change in intervention studies with cancer and diabetes than with other diseases, and not surprisingly, they found greater empirical support for their interventions. Family interventions show promise to help patients and family members manage chronic illnesses. To develop an empirical base for family approaches to managing chronic illnesses, interventions must be based on theories that delineate mechanisms of change in family processes and skills in medical management necessary to maintain patients' and family members' health.     

The relationship of intimate partner aggression to head injury, executive functioning, and intelligence

Measures of head injury, executive functioning, and intelligence were given to a community sample composed of 102 male perpetrators of intimate partner aggression (IPA) and 62 nonaggressive men. A history of head injury and lower mean score on a measure of verbal intelligence were associated with the frequency of male-perpetrated physical IPA as reported by male perpetrators and their female partners. Lower mean scores on a measure of verbal intelligence also predicted frequency of psychological IPA perpetration. Using the perpetrator subtypes outlined by Holtzworth-Munroe et al. (2000), analyses revealed that compared with other groups, the most severely aggressive subtypes (i.e., borderline-dysphoric and generally violent-antisocial) were the most likely to report a history of head injury and to have significantly lower mean scores on a neuropsychological test of verbal intelligence. The possible role of neuropsychological factors in IPA perpetration and implications for prevention and intervention programs are discussed.     

The Efron dice voting system

About 50?years ago, Efron noted some counterintuitive properties of the long-term behavior of contests involving dice. For instance, consider the 6-sided dice whose sides are labeled (4,4,4,4,0,0), (3,3,3,3,3,3), (6,6,2,2,2,2), and (5,5,5,1,1,1). Each die has a 2/3 probability of rolling a higher number than the next one in the list and the last has the same 2/3 probability of rolling a higher number than the first. The non-transitivity of games involving non-identical dice was popularized by Gardner (Sci Am, 223:110–114, 1970). Although Gardner and other authors have observed that non-transitive dice serve to illustrate the complexities of the theory of voting, it does not seem that much attention has been paid to the corresponding voting system. Our purpose in this article is to present this voting system and compare its properties with those of other voting systems. One of the most interesting properties is the fact that cancellation with respect to the Efron dice voting system can replace cancellation with respect to pairwise preferences in Young’s characterization of the social choice function associated with the Borda Count.