Evaluation research,policy, and politics

This article discusses the role of politics in the Juvenile Justice Educational Enhancement Program's effort to use evaluation research data to inform Florida's juvenile justice education policies and practices. Through consideration of the Juvenile Justice Education Enhancement Program's experiences with privatization and the tough love and economy of scale rationales for larger and more custodial juvenile institutions, the variable role of politics is examined. Although the two examples are different, the discussion demonstrates that by maintaining an overriding commitment to its evaluation research purpose, the Juvenile Justice Educational Enhancement Program has been able to continue its data-driven policy efforts despite operating in a politically charged environment.     

Data integration in the evaluation of juvenile justice education

This article discusses the Juvenile Justice Educational Enhancement Program's evaluation of juvenile justice education through the integration of multiple data sources. Validation of best practices and the corresponding effect of these practices on community reintegration require the development of a comprehensive database of program-level and individual-level indicators from several sources. These sources include the Florida Department of Education, the Florida Department of Juvenile Justice, the Florida Department of Law Enforcement, the Florida Department of Corrections, and the Juvenile Justice Educational Enhancement Program's own educational quality assurance and pre-/posttest data. This article examines the methodological, political, and bureaucratic obstacles encountered in this effort. Evaluation research analyses are also explored with an emphasis on the meaning of outcome measures relative to causal issues of time order, the modeling of events, and analytic strategy. The article provides a framework for future juvenile justice educational evaluation and a critical examination of problems encountered in integrating data sources for the purpose of outcome evaluation.     

Pre-, post-, and longitudinal evaluation of juvenile justice education

This article describes two stages of the Juvenile Justice Educational Enhancement Program's pre-, post-, and longitudinal evaluation research. Pilot studies were used to explore how to design statewide research of pre- and postassessment scores and community reintegration outcomes. Preliminary findings suggest that higher performing educational programs produce greater educational gains as measured by academic achievement tests, credits earned, and pupil progression rates. The findings also indicate that these programs have more students returning to school and lower recidivism rates. Building on the pilot studies, refinements were made to the research designs to enable more comprehensive statewide evaluation. Current research includes collection of pre- and postassessment scores from official sources on approximately 16,000 juvenile justice youths. In addition, a research design has been developed to examine program effectiveness by measuring community reintegration variables. Multiple data sources, including official and self-reported data on family, school, employment, and subsequent crime involvement, will be used in the longitudinal study.     

Before you open the doors: ten lessons from Florida's Faith and Community-Based Delinquency Treatment Initiative

In 2003, the Office of Juvenile Justice and Delinquency Prevention (OJJDP) provided Florida's Department of Juvenile Justice (DJJ) with a $3.5 million grant to design and implement a faith- and community-based program for incarcerated juveniles. Florida's experience developing a completely new program has provided valuable lessons regarding important issues to consider and address before clients are referred to the program. This article enumerates 10 key lessons from Florida's experience that can provide important guidance to other sites across the country hoping to develop faith-related services for youths in institutions.     

Exploring the implementation of justice-based alcohol and drug intervention strategies with juvenile offenders: Reclaiming Futures,enhanced adolescent substance abuse treatment,and juvenile drug courts

There is a need for practice models to match young people in the juvenile justice system with timely and effective alcohol and drug intervention services. This paper reports on a new demonstration effort in which three practice approaches currently operating in the United States are being combined to maximize the potential of each. This public–private partnership combines juvenile drug courts (funded by the Office of Juvenile Justice and Delinquency Prevention), adolescent treatment improvement efforts (funded by the Center for Substance Abuse Treatment) and the Reclaiming Futures initiative (funded by the Robert Wood Johnson Foundation). Benefits of combining the three strategies are discussed, emergent implementation challenges and lessons are presented, and suggestions for further evolution are provided.     

Peer contagion among youth in Medicaid-funded residential mental health treatment programs

This paper examined juvenile justice encounters among youth with mental health needs who received care in residential treatment settings. The focus is on peer effects, and whether delinquent peers negatively affected youth. Matched data from Florida Medicaid and Juvenile Justice were used to examine the association between peer delinquency in a residential treatment setting and juvenile justice encounters during and after the treatment episode. Fifteen percent of youth had justice encounters during residential out-of-home treatment, while the rate was lower in the six months after (23.0%) treatment than the six months before (33%). Support for peer contagion was found as youth treated in Medicaid-funded residential mental health treatment programs were more likely to have JJ encounters when the other youth in the facility had histories of delinquency.     

The production of criminological experiments

This article examines the production of crime and justice field experiments during the 1990s. Data were collected on the characteristics of criminological experiments funded by the National Institute of Justice (NIJ), the principal research agency of the U.S. Department of Justice, during the 10-year period from 1991 through 2000. The analyses find that, whereas the funds available for research and evaluation at the NIJ increased during this period, the number of projects and the amount of funds awarded supporting field experiments declined. The article describes the characteristics of the experiments funded and assesses the extent to which the reduced support can be attributed to the characteristics of NIJ research funding, research topics, researchers, or criminal justice operational agencies.     

Profiles of Systems Involvement in a Sample of High-Risk Urban Adolescents with Unmet Treatment Needs

This study examined profiles of involvement in four systems (education, child welfare, legal, and treatment) in a sample of 253 high-risk urban adolescents with unmet behavioral health needs. Self-report data were collected on multiple dimensions of involvement within each system, demographics, and DSM-IV diagnoses. Latent class analysis revealed four profiles: Education System: Academic and Disciplinary, Education System: Academic Only, Legal/Juvenile Justice Involved, and Multiple Systems/Child Welfare. Profiles differed based on gender and psychiatric diagnoses. Boys were overrepresented in Education System: Academic and Disciplinary and Legal/Juvenile Justice Involved, and girls were overrepresented in Multiple Systems/Child Welfare. The two education system focused classes were characterized by depressive disorders and ADHD. Youth in Legal/Juvenile Justice Involved and Multiple Systems/Child Welfare were characterized by conduct disorder and substance abuse. Implications for assessment and treatment planning for high-risk youth and for the organization of community-based behavioral health services are discussed.     

Examining the link between child maltreatment and delinquency for youth with emotional and behavioral disorders

This study examined service delivery and risk factors for 93 youth with emotional and behavioral disorders who were served by one jurisdiction's child welfare, juvenile justice, and special education agencies. The researchers collected data through an archival review of agency records. The article discusses findings as they relate to the link between maltreatment and delinquency for youth with emotional and behavioral disorders and includes recommendations for integrating early intervention efforts.     

The school-police partnership. Identifying at-risk youth through a truant recovery program

A substantial amount of research has established that truancy is a consistent at-risk indicator of future criminality. This article studies the experiences of 178 juveniles targeted by the Truant Recovery Program, a collaborative and nonpunitive school-law enforcement effort, and considers questions regarding its impact through examination of juvenile justice and school information in the years before and after the truancy sweep. In particular, the article suggests that intensive cooperative efforts between school and police may be effective in identifying troubled youth and raises questions about appropriate school and justice system responses for children who demonstrate at-risk behavior.     

Integrating child welfare, juvenile justice, and other agencies in a continuum of services

This article presents a comprehensive strategy framework for integrating mental health, child welfare, education, substance abuse, and juvenile justice system services. It proposes an infrastructure of information exchange, cross-agency client referrals, a networking protocol, interagency councils, and service integration models. This infrastructure facilitates integrated service delivery.     

Evaluation of a complex, multisite, multilevel grants initiative

The Safe Schools/Healthy Students (SS/HS) national evaluation seeks to assess both the implementation process and the results of the SS/HS initiative, exploring factors that have contributed to or detracted from grantee success. Each site is required to forge partnerships with representatives from education, mental health, juvenile justice, and law enforcement, coordinating and integrating their efforts and working together to contribute to comparable outcomes (e.g., reduced violence and alcohol and drug use, improved mental health services). The evaluation uses multiple data collection techniques (archival data, surveys, site visits, interviews, and focus groups) from a variety of sources (project directors, community partners, schools, and students) over several years. Certain characteristics of the SS/HS initiative represent unique challenges for the evaluation, including the absence of common metrics for baseline, outcome data, and lack of comparison group. A unifying program theory was required to address these challenges and synthesize the large amounts of qualitative and quantitative information collected. This article stresses the role of program theory in guiding the evaluation.     

Recommendations for the elder abuse, health, and justice fields about medical forensic issues related to elder abuse and neglect

Increased interest in pursuing criminal and civil actions against alleged perpetrators of elder abuse and neglect has revealed that efforts to assist victims and hold perpetrators accountable are often hampered by the lack of knowledge about the detection and diagnosis of elder abuse and neglect and by the lack of resources for development of research and assistance in these matters. This report sets forth recommendations for the elder abuse, health, and justice fields about medical forensic issues related to elder abuse and neglect. The recommendations were developed by a "working group" of experts representing an array of disciplines that address the problem of elder abuse and neglect pursuant to a grant project funded by the National Institute of Justice (NIJ), the research, development, and evaluation agency of the U.S. Department of Justice (DOJ). The project was conducted by the American Bar Association Commission on Law and Aging, with the assistance of the University of California, Irvine School of Medicine. This was the final project report submitted to NIJ; it has been updated to include information about the availability of reports and products from other grant projects that are discussed in the report.     

Recommendations for the elder abuse, health, and justice fields about medical forensic issues related to elder abuse and neglect

Increased interest in pursuing criminal and civil actions against alleged perpetrators of elder abuse and neglect has revealed that efforts to assist victims and hold perpetrators accountable are often hampered by the lack of knowledge about the detection and diagnosis of elder abuse and neglect and by the lack of resources for development of research and assistance in these matters. This report sets forth recommendations for the elder abuse, health, and justice fields about medical forensic issues related to elder abuse and neglect. The recommendations were developed by a "working group" of experts representing an array of disciplines that address the problem of elder abuse and neglect pursuant to a grant project funded by the National Institute of Justice (NIJ), the research, development, and evaluation agency of the U.S. Department of Justice (DOJ). The project was conducted by the American Bar Association Commission on Law and Aging, with the assistance of the University of California, Irvine School of Medicine. This was the final project report submitted to NIJ; it has been updated to include information about the availability of reports and products from other grant projects that are discussed in the report.     

Delinquency Interrupted: Research into and Inside of Juvenile Correctional Facilities

Incarcerated juveniles are a largely invisible population in the United States. Research into training schools and other juvenile correctional facilities has uncovered a sordid history of good intentions and bad practices. Juvenile correctional facilities are generally considered a less harmful environment for serious delinquents than adult prisons but a much more severe sanction than alternative community programs or treatment would be. Studies have shown that juveniles in confinement face similar deprivation and many of the same pains of imprisonment ( Sykes 1958 ) as adult prisoners, but the more pressing threat to adolescent inmates may be to their psychological selves rather than their physical being. New research on adolescent brain development is garnering attention and sparking questions about the culpability and punishment of juvenile offenders, even as public opinion research suggests that there is fairly widespread support for rehabilitation and a social‐welfare oriented juvenile justice system. By listening and giving voice to incarcerated youth and their “keepers,” investigating the conditions of confinement, and continuing to evaluate programs and outcomes, researchers are positioned to play a key role in the future of juvenile justice and juvenile correctional facilities.     

The whole elephant: Defining evaluation

Definitions help us understand the characteristics of an object or phenomenon and are a necessary precursor to understanding what a good version of it looks like. Evaluation as a field has resisted a common definition (Crane, 1988; Morell & Flaherty, 1978; M. F. Smith, 1999), which has implications for marketing, training, practice, and quality assurance. In this position paper, I describe the benefits and challenges of not having a clear, agreed-upon definition, then propose and explore the implications of two definitions for the evaluation profession based on values and valuation as the core of evaluation practice. The purpose is to describe a possible way forward through definition that would increase our professional profile, power, and contribution to social justice. The paper concludes with implications for evaluator competencies and evaluation education and questions for further research.     

Framework for culturally competent decisionmaking in child welfare

This article provides a framework to understand the cultural, social, political, and economic factors that affect decisionmaking when working with ethnically and racially diverse families in the child welfare system. The article describes external factors affecting the decisionmaking process, including community environment, agency structure, and family characteristics. It then reviews the core stages of the casework process, describing key decisions during intake, assessment, service planning, implementation, evaluation, and closure. Although the framework is based on casework process in the child welfare system, it can be adapted to other child-serving systems, including education, mental health, and juvenile justice.