Piloting different approaches to personalised offender management in the English criminal justice system

Various approaches to personalisation are well-established in the UK social care sector and are now starting to ‘travel’ to other sectors. In this paper we report findings from an evaluation of a pilot to test elements of personalisation in the management of offenders in probation services within the English criminal justice system. Following a review of evidence from social care, three different approaches to personalised-practice were developed and tested on a small-scale in three separate sites. The evaluation finds that all three approaches were implemented reasonably successfully, but challenges were identified including that personalised approaches are more time-consuming, that staff need support to exercise professional discretion and that balancing greater choice with managing criminogenic risk requires new ways of conceptualising the relationship between case manager and service user. Overall, ‘deeper’ approaches to personalisation, such as co-production, will take time to emerge. This paper makes two important contributions to the debate on personalisation in public services. First, it addresses the question of how transferable the concept of personalisation is from the social care sector to other sectors in the UK, in this case the criminal justice system. Secondly, it outlines a methodology for developing and evaluating personalisation pilots, prior to a wider roll-out.     

“They give you back that dignity”: understanding the intangible resources that make a transitional house a home for homeless families

Approximately 33% of the homeless population in 2017 were families with children. Researchers have argued for interventions and services to employ an ecological perspective to mitigate some of the unintentional harm inherent in existing programs designed to serve homeless families. This paper utilized a case study of a transitional housing program to (1) determine how well the program met recommendations informed by the ecological perspective and (2) explore underlying themes across recommendations. Past and current residents of a transitional housing program participated in semi-structured interviews as part of a program evaluation. Interviews were deductively and inductively coded to meet aims 1 and 2, respectively. Residents identified several aspects of the program that met the ecological perspective recommendations including culturally-responsive services, trauma-based services, and social and community connections. Major underlying themes spanning across recommendations included interpersonal resources (caring, sense of belonging), empowerment resources (advocacy, family preservation), impact on family (family life, positive youth development), and communication. This case study, through the analysis of residents’ perspectives, highlights the important role of transitional housing programs in providing a supportive, empathetic, empowering, and home-like environment that gives residents “back that dignity” so they are emotionally and physically prepared to successfully transition to permanent housing.     

Transitional Victimisation: Collaborators’ Offspring as Children at Risk

Using transition theory from political science, this paper analyses how the lives of children of Norwegians punished for collaboration with the German occupant were adversely influenced by transitional justice after the return to democracy in 1945. The paper highlights how the complexity and hectic character of such regime changes are associated with a high risk for unintended social outcomes, a risk that has received little attention in the study of transition processes. Findings indicate that in order to avoid harmful consequences for children, careful and long‐term thinking is essential when new democracies select a transitional justice strategy.     

The significance of definitions of success in program evaluation

In program evaluation, determining whether a program is reaching its goals is key to evaluating program success. Having an understanding of the stated goals of each organization within a system rather than simply knowing the formal system goals can aid in program development and inform policy making. Evaluability assessments were used initially as a means of learning how Philadelphia delinquency prevention programs define success. Subsequent surveys of programs were conducted to ascertain whether any changes in definitions of success had occurred. Examination of programs' definitions of success over time has facilitated program development and policy making.     

Dissemination and implementation science in program evaluation: A telemental health clinical consultation case example

Increased attention has been placed on evaluating the extent to which clinical programs that support the behavioral health needs of youth have effective processes and result in improved patient outcomes. Several theoretical frameworks from dissemination and implementation (D&I) science have been put forth to guide the evaluation of behavioral health program implemented in the context of real-world settings. Although a strong rationale for the integration of D&I science in program evaluation exists, few examples exist available to guide the evaluator in integrating D&I science in the planning and execution of evaluation activities.This paper seeks to inform program evaluation efforts by outlining two D&I frameworks and describing their integration in program evaluation design. Specifically, this paper seeks to support evaluation efforts by illustrating the use of these frameworks via a case example of a telemental health consultation program in pediatric primary care designed to improve access to behavioral health care for children and adolescents in rural settings. Lessons learned from this effort, as well as recommendations regarding the future evaluation of programs using D&I science to support behavioral health care in community-based settings are discussed.     

Evaluating the integration of strategic priorities within a complex research-for-development funding program

This paper examines the application of Complexity Theory constructs to a research-for-development program evaluation and presents an overview of the implications and promising approaches for evaluating complex programs. We discuss lessons learned from an evaluation completed for the International Development Research Centre’s Food, Environment and Health (FEH) program, which investigated the integration and outcomes of five strategic program priorities: partnerships, southern leadership, gender and equity, scale, and environmental sustainability. We present interpretations from a secondary, thematic content analysis that categorized evaluation findings across four complexity constructs: emergence, unpredictability, contradiction and self-organization. Viewing the evaluation through these constructs surfaced some important features of the FEH program to date, specifically its evolving approach, adaptiveness to emergent issues, non-linear outcomes, and self-organizing agents, which had several implications for the evaluative process. We conclude that the most appropriate evaluation designs for complex funding programs are participatory (to explore all stakeholders' influence), adaptive (to capture the unexpected) and assess external contexts. The application of complexity constructs may be useful for evaluators to gain a deeper understanding of how program contexts change in the face of complexity and why some evaluation methods work more effectively than others.     

Mixed-methods library evaluation integrating the patron,library, and external perspectives: The case of Namibia regional libraries

The article presents a mixed-methods evaluation of regional libraries in Namibia, which incorporates three perspectives: the patron perspective (library users), the library perspective (library staff, management, and related officials), and the external perspective (including evaluators and monitoring data). Seven data collection methods were used: patron surveys, patron panel studies, focus group discussions, key informant and staff interviews, secondary data analysis, media analysis, and observations. The goal of the evaluation was to assess library performance for both formative and summative purposes by addressing evaluation questions on areas such as library services, use, and operations. Building upon the literature review of how mixed-methods approaches can contribute to library evaluation, the aim of this article is to show how a mixed-methods evaluation can be designed to examine multi-faceted library performance and to illustrate how the evaluation design allows information complementarity and can be utilized to present diverse viewpoints of the above three perspectives. The evaluation design, analysis process, and lessons learned from this study may be useful to evaluators engaged in evaluation of public services or programs (including public libraries) that examine multiple aspects of service performance and involve a variety of stakeholders.     

Treatment of older adolescents in transitional programs: rapprochement crisis revisited

This paper draws largely upon the developmental theory of Margaret Mahler. It depicts a seemingly regressive revisitation of disturbed older male adolescents in transitional residential programs to unresolved elements of the original rapprochement crisis in the separation-individuation process. It describes how cycles of intense shadowing and darting behavior reemerge as the possibility of authentic independent functioning beckons. There is presentation of the way in which this revisitation, including splitting phenomena, is manifested in the residential treatment environment. Therapeutic implications are focused upon. A practic vignette illuminates the metapsychological thrust of the paper.     

Evaluating development programs: building joint activity

There has been an increasing plea for a participatory approach to development programs. It is argued that for programs to be sustainable, they need to be conceptualized, negotiated and run by all stakeholders jointly. Concurrent with this trend, there has been an increasing demand for program evaluation strategies, which encompass an evaluation of the nature and level of participation of the different stakeholders in the program. This paper proposes a theoretical framework for such evaluation and subsequently provides a practical illustration. Engeström's notions of ‘activity system’ and ‘learning by expanding’, are proposed as a framework for the evaluation of participation and an action evaluation methodology is suggested as the practical tool within such framework. The paper presents a case study of a conservation education program in Africa, which operationalizes the proposed theoretical framework.     

Post‐Authoritarian State Formation in Argentina: Transitional Justice as the Accumulation of Symbolic Power

Looking at the transitions to democracy in Latin America during the late 20^th century, a number of scholars observed that human rights and transitional justice had become the central legitimizing axis of the new, post‐authoritarian order. But the question of how human rights and transitional justice measures became such powerful sources of legitimacy in the first place was left unexplored. In this article I use Bourdieu's concept of symbolic capital along with Mara Loveman's explanation of the accumulation of this capital to explain how transitional justice came to function as a form of post‐authoritarian state formation in Argentina.     

PARTICIPATORY EVALUATION: NAVIGATING THE EMOTIONS OF PARTNERSHIPS

This paper describes the processes involved in evaluating a Sure Start programme in one inner city area. The evaluation was set up in the spirit of participatory action research in which the researchers aimed to work in partnership with key stakeholders to both enable and sustain supportive evaluative processes. The evaluation supported the aspirations espoused by the national Sure Start agenda to improve the lives of children under four, their parents and communities through an expressed commitment to partnership working. The paper draws on ethnographic reflections to describe and analyse the processes involved in setting up the evaluation over two years. Issues of trust, ambiguity and conflict associated with partnership working are explored. In particular the emotional components of relationships required to work in partnership are described as a means of managing ambiguity and conflict and promoting trust. Emotional labour is taken as a conceptual starting point to analyse the relationships and the organisational conditions required to sustain partnerships. Further psychoanalytic and sociological studies are drawn upon to aid the analysis and in particular to understand the emotional components of partnerships in the relatively uncharted waters of inner city regeneration work.     

Fostering CHANGE: Lessons from implementing a multi-country,multi-sector nutrition-sensitive agriculture project

Using agriculture to improve nutrition is an approach growing in popularity, with programs becoming increasingly complex and multisectoral. While there is an active line of research assessing the impacts of such programs, little has been written about the process of successfully implementing them. As such, this paper uses a multisectoral nutrition-sensitive agriculture program implemented in four African countries as a case study to address key challenges in and lessons learned from implementation. We highlight the overall flexibility of nutrition-sensitive agriculture but also the need to adapt certain aspects to the particular context, as well as the opportunities for cross-context learning (and the limits to this). Integrating rigorous evaluation into such complex programs and forging diverse cross-sectoral partnerships offer both rewards and challenges, upon which we reflect. Main lessons learned from the program include the importance of carefully sequencing interventions, retaining flexibility in implementation, allowing for considerable time for cross-sector integration and coordination, and considering community impacts when designing research.     

Security Sector Reform and Development Assistance: Explaining the Diffusion of Policy Priorities among Donor Agencies

This paper employs a world society theoretical framework to examine the recent trend among foreign aid donors to focus on security sector reform as an aid priority. Through a comparative qualitative case study based on interview data collected from aid officials and development workers in Canada, Sweden, and the United States (n = 41) in 2006–2007, this paper finds that the extent to which the security sector reform agenda is integrated into donor policy and programs is mediated by catalytic policy processes linked to intergovernmental organizations and the degree of donor agency autonomy from the rest of government. These findings are used to illustrate how common processes of globalization in world society shape similar approaches to foreign aid among donor agencies despite disparate domestic contexts. These processes lead to convergence of donor policy around security issues and at the same time can account for decoupling of practice from world society policy models.     

A novel approach for evaluating programs designed to serve military veterans: Using an adapted common components analysis

Evaluators are challenged to keep pace with the vast array of Veteran support programs operating in the United States, resulting in a situation in which many programs lack any evidence of impact. Due to this lack of evidence, there is no efficient way to suggest which programs are most effective in helping Veterans in need of support. One potential solution to this dilemma is to reconceptualize program evaluation, by moving away from evaluating programs individually to evaluating what is common across programs. The Common Components Analysis (CCA) is one such technique that aggregates findings from programs that have undergone rigorous evaluation at the level of program components (e.g., content, process, barrier reduction). Given that many Veteran programs lack outcome evidence from rigorous studies, an adaptation to CCA is needed. This report examines cross-sectional data from a pilot study using an adapted CCA across five domains of well-being (i.e., employment, education, legal/financial/housing, mental/physical health, and social/personal relationships). The purpose of this preliminary study is to determine the feasibility of eliciting program nominations and program components from Veterans via an online survey. When coupled with a longitudinal research design, this adaptation to CCA will allow for stronger causal claims about the expected impact of different program components within and across a variety of domains.     

Mediators and moderators in the evaluation of programs for children. Current practice and agenda for improvement

The author examines the role of mediators and moderators in the evaluation of programs for children. The terms are defined and examples of each are presented. Using bibliometric analysis, the author examines how evaluators use mediators and moderators in treatment studies in education, juvenile justice, health care, child protection, and mental health. The use of mediators and moderators is sporadic and vague at best. An agenda for improvement is outlined that includes greater use of program theory and intensive case studies to find out why researchers in prevention and health promotion incorporate mediators and moderators more effectively in their evaluations.     

Experiments in evaluation capacity building: Enhancing brain disorders research impact in Ontario

This paper is the introductory paper on a forum on evaluation capacity building for enhancing impacts of research on brain disorders. It describes challenges and opportunities of building evaluation capacity among community-based organizations in Ontario involved in enhancing brain health and supporting people living with a brain disorder. Using an example of a capacity building program called the “Evaluation Support Program”, which is run by the Ontario Brain Institute, this forum discusses multiple themes including evaluation capacity building, evaluation culture and evaluation methodologies appropriate for evaluating complex community interventions. The goal of the Evaluation Support Program is to help community-based organizations build the capacity to demonstrate the value that they offer in order to improve, sustain, and spread their programs and activities. One of the features of this forum is that perspectives on the Evaluation Support Program are provided by multiple stakeholders, including the community-based organizations, evaluation team members involved in capacity building, thought leaders in the fields of evaluation capacity building and evaluation culture, and the funders.     

Valuing and embracing complexity: How an understanding of complex interventions needs to shape our evaluation capacities building initiatives

This paper describes some of the main challenges of evaluating complex interventions, as well as the implications of such challenges for evaluation capacity building. It discusses lessons learned from a case study of an evaluation of Dancing with Parkinson’s, an organization that provides dance classes to people with Parkinson’s disease in Toronto, Canada. These implications are developed from a realist evaluation lens. Key lessons include the need to develop skills to understand program mechanisms and contexts, recognize multiple models of causality, apply mixed method designs, and ensure the successful scaling up and spread of an intervention.     

Non-formal educator use of evaluation results

Increasing demands for accountability in educational programming have resulted in increasing calls for program evaluation in educational organizations. Many organizations include conducting program evaluations as part of the job responsibilities of program staff. Cooperative Extension is a complex organization offering non-formal educational programs through land grant universities. Many Extension services require non-formal educational program evaluations be conducted by field-based Extension educators. Evaluation research has focused primarily on the efforts of professional, external evaluators. The work of program staff with many responsibilities including program evaluation has received little attention. This study examined how field based Extension educators (i.e. program staff) in four Extension services use the results of evaluations of programs that they have conducted themselves. Four types of evaluation use are measured and explored; instrumental use, conceptual use, persuasive use and process use.Results indicate that there are few programmatic changes as a result of evaluation findings among the non-formal educators surveyed in this study. Extension educators tend to use evaluation results to persuade others about the value of their programs and learn from the evaluation process. Evaluation use is driven by accountability measures with very little program improvement use as measured in this study.Practical implications include delineating accountability and program improvement tasks within complex organizations in order to align evaluation efforts and to improve the results of both. There is some evidence that evaluation capacity building efforts may be increasing instrumental use by educators evaluating their own programs.