Using a community-created multisite evaluation to promote evaluation use across a sector

Multisite evaluation has been cited as a strategy for building knowledge about a program's success and enhancing evaluation capacity and evaluation use. Even so, the extent to which data from these efforts are made available to individual project participants varies. This article describes a community-created multisite evaluation that was designed to overcome this limitation. Based on the negotiated centralized evaluation model and implemented within the context of a community of practice, the EvalFest community-created multisite evaluation has used shared measures to collect data from 30,000 participants from partner sites across the community over three years. This article describes the design characteristics of the community-created multisite approach, and presents results related to partners’ evaluation use that demonstrate the early success of this method. We conclude by considering how this strategy might be applied to other sectors.     

Examining the Underlying Values of Food Security Programming: Implications for the Social Work Profession

Incongruities between social work’s professional values and dominant public and private responses to food insecurity are discussed. Empowerment theory is positioned as an evaluative lens to examine food assistance. Summaries of dominant forms of food assistance are provided and scrutinized using basic tenets of empowerment theory. Contrasting examples of innovative programs are also discussed in terms of their potential for empowering outcomes. Highlighted programs, including community gardens, community kitchens, and the food rescue and redistribution program D.C. Central Kitchen, demonstrate the evaluative potential of empowerment theory. Implications for social work research and practice are explored.     

Intrahousehold empowerment gaps in agriculture and children's well‐being in Bangladesh

This article examines the relationship between empowerment gaps between spouses and children's nutritional status and education using nationally representative data from the 2012 Bangladesh Integrated Household Survey. We measure relative empowerment of spouses using the recently developed Women's Empowerment in Agriculture Index. We find that while gender gaps in empowerment are only linked weakly to children's nutritional status, fathers’ and mothers’ empowerment have different effects on investment in children. Fathers’ empowerment is positively associated with younger children's nutrition and schooling, while mothers’ empowerment is more important for girls’ education in general and in keeping older boys and girls in school.     

The use of ‘empowerment’ among organisations supporting victims of domestic violence in Sweden

The aim of this paper is to investigate the prevalence and character of ‘empowerment’ as an approach used by women's shelters, crime victim support groups and municipal crisis centres that provide support for battered women in Sweden. The study was based on a mail survey distributed among representatives of local crime victim support groups and women's shelters run by non-governmental organisations and the public sector (N = 207). The survey showed that empowerment was perceived as similar to the already established ‘help to self-help’ approach. Empowerment was described as a tool for individual change rather than collective action, although links to power-sensitivity and social change did appear in some answers. Contrary to expectations, the use of empowerment was not accompanied by comments on problematic power relations and difficulties in merging a non-directive approach with professional responsibilities. Using neo-institutional theory, results were interpreted in relation to established work-patterns and problems that empowerment was perceived to solve.     

The Power of Peer Support: The Development of the Empowerment Center

ABSTRACT

This article discusses the events and influences that led to the author’s contribution to the development of a mental health consumer-run organization, The Empowerment Center in Mount Vernon, NY. The author traces some events that influenced him personally and the evolution of the Empowerment Center from an organization that provided advocacy services (to those with mental health problems who were hospitalized) and community-based support services to people in mental health recovery, to an organization focusing on economic empowerment and economic development for the community of those in recovery. The transition of the Empowerment Center into an organization focused on economic empowerment underscores the importance of economic independence for people in recovery and the consumer-run organizations that support them. The Empowerment Center endeavors to demonstrate that mental health consumer-survivors can take control of their economic health as well as their physical and mental health.     

Empowerment as a Framework for Indigenous Workforce Development and Organisational Change

Abstract

Relative powerlessness resulting from colonial dispossession and associated passive welfare policies has long been recognised as a critical factor influencing the health and wellbeing of Indigenous Australians, yet it is hard to find well-evaluated health and social interventions that take an explicit empowerment approach. This paper presents the findings of a Family Wellbeing Empowerment programme pilot delivered to Cairns Region Department of Families Indigenous youth workers and family and community workers in 2003/2004. The aim of the pilot was to build the capacity of these workers to address personal and professional issues as a basis for providing better support for their clients. The pilot demonstrated the effectiveness of the programme as a tool for worker empowerment and, to a lesser degree, organisational change.     

Advances in collaborative evaluation

Collaborative evaluation is an approach that offers, among others, many advantages in terms of access to information, quality of information gathered, opportunities for creative problem-solving, and receptivity to findings. In the last decade, collaborative evaluation has grown in popularity along with similar participatory, empowerment, and utilization-focused evaluation approaches. Increasingly, it has brought together evaluators and stakeholders from different sectors, disciplines, and cultures to exchange knowledge on how collaboration can be used as a strategic tool for fostering and strengthening evaluation practice. O'Sullivan (2004) and Rodríguez-Campos (2005, 2008) brought forward collaborative evaluation conceptual frameworks that distinguished it from other approaches. This article updates those conceptual frameworks and discusses key collaborative evaluation advances in order to further clarify and facilitate engagement in sound practice.     

From plan to action: Adapting evaluation to serve the developmental needs of a newly-funded multidisciplinary research center

The Southeastern Coastal Center for Agricultural Health and Safety (SCCAHS) is one of many newly-funded federal research centers, housing five multidisciplinary research projects and seven pilot projects, and serving a multi-state region. In the early stages of such a complex project, with multiple teams separated by geography and disciplines, the evaluation program has been integral in connecting internal and external stakeholders at the center and project levels. We used a developmental evaluation (DE) framework to respond to the complex political environment surrounding agricultural health and safety in the southeast; to engage external stakeholders in guiding the center’s research and outreach trajectories; to support center research teams in a co-creation process to develop logic models and tailored indicators; and to provide timely and feedback within the center to address communications gaps identified by the evaluation program. By using DE principles to shape monitoring and evaluation approaches, our evaluation program has adapted to the dynamic circumstances presented as our center’s progress has been translated from a plan in a grant proposal to implementation.     

Advances in collaborative evaluation

Collaborative evaluation is an approach that offers, among others, many advantages in terms of access to information, quality of information gathered, opportunities for creative problem-solving, and receptivity to findings. In the last decade, collaborative evaluation has grown in popularity along with similar participatory, empowerment, and utilization-focused evaluation approaches. Increasingly, it has brought together evaluators and stakeholders from different sectors, disciplines, and cultures to exchange knowledge on how collaboration can be used as a strategic tool for fostering and strengthening evaluation practice. O'Sullivan (2004) and Rodríguez-Campos, 2005, Rodríguez-Campos, 2008 brought forward collaborative evaluation conceptual frameworks that distinguished it from other approaches. This article updates those conceptual frameworks and discusses key collaborative evaluation advances in order to further clarify and facilitate engagement in sound practice.     

Rethinking Empowerment

Empowerment is one of the most frequently invoked concepts in social work theory and practice of the 1980s. In the United States, social workers representing diverse practice methods and working with highly varied populations articulate a commitment to empowerment, one that they have inherited from both the political movements of the 1960s and the much older mutual aid tradition. Though the concept of empowerment succeeds in constituting a mission for social work congruent with the historic and contemporary values of the profession, re-examination and clarification of this important notion is necessary to prevent conceptual confusion and an unintended paternalism from intruding on the relationship of social workers and their clients.     

Mediating role of citizen empowerment in the relationship between participatory monitoring and evaluation and social sustainability

The paper explores the mediating influence of citizen empowerment on the relationship between participatory monitoring and evaluation (PM&E) and social sustainability as a contribution to the ongoing debate on sustainability. The empirical investigation took the form of mixed-methods approach with the objectives to determine the extent to which PM&E influences citizen empowerment; establish the extent to which citizen empowerment influences social sustainability; and establish the joint influence of PM&E and citizen empowerment on social sustainability. The study employed a concurrent parallel design, in which samples for quantitative and qualitative components were different but drawn from the same population, and data collected within the same timeframe. Quantitative data were analyzed through bivariate and multiple regression analyses. Conversely, the qualitative component utilized interpretive technique and recursive abstraction to organize, summarize and give impression on the causal mechanisms at play in the quantitative data. Evidence from the study revealed that citizen empowerment mediates the relationship between PM&E and social sustainability. The study also found that when PM&E and citizen empowerment are considered together, citizen empowerment seems to have a dominant influence on social sustainability than PM&E itself. Thus, the government and development practitioners should prioritize citizen empowerment as a precursor to the realization of social sustainability outcomes.     

Empowerment revisited—social work,resistance and agency in black communities

The struggles of black communities in Britain are well established in modes of empowerment that shape and direct their social welfare activities. This legacy of empowerment continues to be expressed in social action orientated strategies found in many black community based groups and organisations. The introduction of anti-racist and multicultural approaches in the empowerment process has been useful in highlighting the ways in which black communities have been disempowered through social work policies and practice. However, the experiences, values and interpretations instituted by black communities themselves are often marginalised in the public realm. These considerations draw attention to the ways in which empowerment is largely defined by professionals and policy makers who have considerable influence in shaping and directing the life chances of black families and communities. These insights provide important ways to address marginalisation through publicly affirming and recognising the specific experiences, cultures, social realities and contributions of black com­munities. This paper examines concepts of empowerment in social work and maintains that the profession has ignored black communities’ interpretations, cultural antecedents, experiences and values as a source of new perspectives and ideas for practice. These new approaches can assist social workers and policymakers in responding to the changing needs of black communities.     

Learning by doing: an experiential approach to program evaluation

Despite the increased importance of producing social workers who are prepared to evaluate practice, there is a paucity of literature addressing the pedagogical challenges of program evaluation courses. The present study evaluates the structure and pedagogical approach of a newly implemented program evaluation course. The course took a team-based, experiential learning approach to designing a program evaluation plan for students’ field placement agencies. Five student-led focus groups (N = 44) were conducted at the conclusion of the course to address two aims: to assess student perceptions of the content, structure, and delivery of the course; and to assess student attitudes toward applied research. Data were collected between November and December 2014 in a public university’s graduate social work program in the northeastern United States. The steps of grounded theory data analysis were implemented to elicit major themes from student feedback. Results suggest that perceptions of applicability play a central role in shaping student attitudes toward program evaluation. Experiential and team-based pedagogical approaches appear to increase accessibility of course content for students. Suggestions for enhancing student learning in program evaluation courses are discussed.     

Policy advocacy and leadership training for formerly incarcerated women: An empowerment evaluation of ReConnect,a program of the Women in Prison Project,Correctional Association of New York

There is limited knowledge on re-entry initiatives for formerly incarcerated women specifically on building women's advocacy and leadership skills. Our research highlights an empowerment evaluation on ReConnect, a 12-session; innovative advocacy and leadership development program rooted in an integrated framework of empowerment, and transformational leadership theories. Using thematic analysis, we coded three focus groups with 24 graduates, for themes that matched our framework's concepts. ReConnect graduates reported being empowered by the information they received on parental rights, housing, and employment. Participants agreed that ReConnect improved their communication skills, preparing them to advocate for themselves and community members.     

Celebrating the 21st anniversary of empowerment evaluation with our critical friends

This special topic edition of E&PP presents the insights of luminaries in the field who have helped shape empowerment evaluation with their critiques, concerns, and congratulations. We celebrate their contributions to empowerment evaluation. This special topic edition of E&PP presents their comments about an evaluation approach that, according to president Stewart Donaldson, has “gone viral” across the globe (Donaldson, 2015).To set the stage for these critical friends’ comments, additional context for their discussion is provided. In addition, this special topic edition concludes with a brief comment on their thoughts.     

Taking the measure: a participatory approach to measuring and monitoring psychological empowerment in social work practices

For years, the professional field has been searching for methods that could be used to chart the effectiveness of social services. Social workers report that their current systems for registration and measurement do not provide an accurate reflection of their practice. The measurement of client empowerment might offer an alternative method that could improve the evaluation of social services. This method links the need for measuring instruments to the paradigm of empowerment, which has shaped contemporary social work. A scale for measuring psychological empowerment was developed in consultation with social workers and people in poverty, based on validated and reliable measurement scales. It was subsequently tested in two Public Centres for Social Welfare in Flanders (Belgium). Results of confirmatory factor analysis indicate that the measurement model provides a good fit for the data and that the various latent factors are reliable and valid. We conclude that the factors emerging from the factor analysis reflect the three dimensions that form the concept of psychological empowerment: the intrapsychic, interpsychic and behavioural components. Results from the post-test survey indicate that this measurement instrument is promising, and that it could offer a response to current needs, although additional efforts are needed in order to improve the instrument’s handling.     

Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned

Organizational evaluation capacity (EC) has received significant attention in the evaluation research literature in the past decade. Much of the focus has been on defining organizational evaluation capacity, which can be thought of as the competencies and structures required to conduct high-quality evaluation studies (capacity to do), as well as the organization's ability to integrate evaluation findings into its decision-making processes (capacity to use). This paper seeks to contribute to this growing body of knowledge through a multiple case study of EC across three different organizations (e.g., non-profit, provincial government and federal government, herein named sectors); the novelty of this particular study is that each case study is based on the use of a common measurement tool developed by Bourgeois, Toews, Whynot and Lamarche (2013). The cross-case analysis presented in the paper reveals that evaluation capacity tends to be higher, both in terms of capacity to do and capacity to use, in organizations that have developed systematic mechanisms to institute an evaluation culture within their walls. Interestingly, however, we also found that capacity to use does not first require capacity to do, as evidenced in the non-profit organization under study.     

Building relationships and facilitating immigrant community integration: An evaluation of a Cultural Navigator Program

Despite the long history of immigration in the United States, communities around the country struggle to integrate newcomers into the economic, cultural, and political spheres of society. Utilizing results from the program evaluation of one public library's Cultural Navigator Program, the authors illustrate how communities and public institutions can promote integration and relationship-building between newly arrived immigrants and long-time residents. Existing social networks within receiving communities, conceptualized in this article as social capital, were leveraged to build capacity among newly arrived immigrants and foster inclusivity and integration at the community level. As a place of intervention, public libraries are suggested as a safe and shared space where community integration can be fostered. Insights derived from the evaluation inform a discussion on engaging approaches to immigrant integration. Lessons learned and recommendations for program evaluators and administrators are provided.     

Using developmental evaluation as a system of organizational learning: An example from San Francisco

In the last 20 years, developmental evaluation has emerged as a promising approach to support organizational learning in emergent social programs. Through a continuous system of inquiry, reflection, and application of knowledge, developmental evaluation serves as a system of tools, methods, and guiding principles intended to support constructive organizational learning. However, missing from the developmental evaluation literature is a nuanced framework to guide evaluators in how to elevate the organizational practices and concepts most relevant for emergent programs. In this article, we describe and reflect on work we did to develop, pilot, and refine an integrated pilot framework. Drawing on established developmental evaluation inquiry frameworks and incorporating lessons learned from applying the pilot framework, we put forward the Evaluation-led Learning framework to help fill that gap and encourage others to implement and refine it. We posit that without explicitly incorporating the assessments at the foundation of the Evaluation-led Learning framework, developmental evaluation’s ability to affect organizational learning in productive ways will likely be haphazard and limited.     

The manifestations of politics in evaluation: An exploratory study across the evaluation process

Evaluation has been described as a political act. Programs and policies are generated from a political process, and the decision to evaluate and how to use the evaluation are manifestations of the political dynamic. This exploratory study was conducted with practicing evaluators to understand what they view as political situations in the evaluation process and how they responded to these situations. Findings suggest that, in relation to the potential evaluation phases in which each respondent has been involved, evaluations are susceptible to politics when initially attempting to identify stakeholders and when it’s time to report the evaluation findings. Evaluators have also developed multiple strategies for dealing with these situations, including finding allies for the evaluation and working to explain the evaluation process and its implications. We hope that this study will help to inform novice and expert evaluators about the various political situations they may encounter in their practice.