Communication of results and utilization in participatory program evaluation

This paper explores the linkages between utilization and stakeholder participation in the evaluation process, specifically in the communication of results phase of evaluation. It is argued that key elements of a participatory evaluation process can be linked to meaningful and multiple forms of results utilization through cognitive and political uses of the evaluation process. Data from two small-scale participatory evaluations are used to support this argument.     

Evaluating development programs: building joint activity

There has been an increasing plea for a participatory approach to development programs. It is argued that for programs to be sustainable, they need to be conceptualized, negotiated and run by all stakeholders jointly. Concurrent with this trend, there has been an increasing demand for program evaluation strategies, which encompass an evaluation of the nature and level of participation of the different stakeholders in the program. This paper proposes a theoretical framework for such evaluation and subsequently provides a practical illustration. Engeström's notions of ‘activity system’ and ‘learning by expanding’, are proposed as a framework for the evaluation of participation and an action evaluation methodology is suggested as the practical tool within such framework. The paper presents a case study of a conservation education program in Africa, which operationalizes the proposed theoretical framework.     

Stakeholder participation in evaluation design: Is it worth the effort?

Naturalistic approaches to program evaluation generate multiple interpretations and possible conclusions, but lack inherent guidance on how to choose among or reconcile them. A practical frame of reference for addressing this challenge emphasizes criteria of utility and credibility, as well as an evaluation process that is participatory and openly value-pluralistic. This paper discusses the logic of justification underlying such a practical perspective.     

Justifying conclusions in naturalistic evaluations: A practical perspective

Naturalistic approaches to program evaluation generate multiple interpretations and possible conclusions, but lack inherent guidance on how to choose among or reconcile them. A practical frame of reference for addressing this challenge emphasizes criteria of utility and credibility, as well as an evaluation process that is participatory and openly value-pluralistic. This paper discusses the logic of justification underlying such a practical perspective.     

Towards an assessment for organizational participatory research health partnerships: A systematic mixed studies review with framework synthesis

Within the health sciences, organizational participatory research (OPR) is defined as a blend of research and action, in which academic researchers partner with health organization members. OPR is based on a sound partnership between all stakeholders to improve organizational practices. However, little research on the evaluation of OPR health partnership exists. This systematic mixed studies review sought to produce a new theoretical model that structures the evaluation of the OPR processes and related outcomes of OPR health partnerships. Six bibliographic databases were searched together with grey literature sources for OPR health partnership evaluation questionnaires. Six questionnaires were included, from which a pool of 95 OPR health partnership evaluation items were derived. The included questionnaires were appraised for the quality of their origin, development and measurement properties. A framework synthesis was performed using an existing OPR framework by organizing questionnaire items in a matrix using a hybrid thematic analysis. This led to our proposed Organizational Participatory Research Evaluation Model (OPREM) that includes three axes, Trust, Collective Learning and Sustainability (with specific dimensions) and 95 items. This model provides information to help stakeholders comprehensively structure the evaluation of their partnerships and subsequent improvement; thus, potentially helping to improve health organization practices.     

The Semarang Environmental Agenda: a stimulus to targeted capacity building among the stakeholders

Recently, a sustainable development action plan was prepared for the city of Semarang, Indonesia. An essential component of this action plan was the building up of an effective partnership among the diverse stakeholders to ensure their commitment. Such an approach in which the city's stakeholders are actively involved in decision making was new for Semarang. This paper describes the roles and attitudes of the different stakeholders in the development of the action plan, and the complex and difficult process to forge partnerships between them. The paper explores the constraints faced during this participatory process, and suggests that there is a need to build the capacity of all stakeholders involved — local government, civil society and the private sector — so that they will be able to play their envisaged respective roles better.     

Stakeholder participation in evaluation: How important is diversity?

Participatory evaluation seeks in part to promote exchanges of perspective and experience among diverse program stakeholders. Diversity in such exchanges is viewed as integral to the substantive and political strands of the participatory process. Yet, using the grounded experience of two participatory evaluation case studies, we offer some challenges to this ideal of diversity and suggest that when action is the desired outcome, somewhat less rather than more diversity of stakeholder participation is actually desired.     

Lost in translation? The dilemma of alignment within participatory technology developments

As an instrument for participatory technology development, Scenario-Based Design offers significant potential for an early inclusion of future users. Over the course of a 3-year research project, this method was examined as a procedure for participatory technology development. Methods and instruments aimed at achieving a potential user??s participation, and the resulting cooperation of heterogeneous social groups can be seen as translation tools. Their purpose is to act as translators between different social fields and the specific knowledge associated with them. These translation capabilities and participatory methods should result in the best possible convergence of different orientations and purposes. In this paper, attempting to achieve the best possible convergence is described as a dilemma of alignment. Several approaches will be used to describe the dynamic of the alignment dilemma within the above-mentioned project. The reconstruction follows one question that is proposed as a heuristic pattern to meet the requirements of an accurate analysis of holistic participatory methods: Who or what has to adjust to whom or what, why, when, and in which way? The main conclusions include the finding that the alignment dilemma is not equally balanced, that the agency of epistemic objects within the process has to be captured, and that it is easy for translation??provided primarily by core instruments of the participatory method used??to begin to overwrite the needs and purposes of one social group with the interests and orientations of another.     

Expanding the Horizons of Forgiveness Therapy: A Cross-Cultural Application with a Bedouin-Arab Woman

This paper highlights forgiveness therapy, a therapeutic approach that has received limited attention in the social work literature. This study represents a beginning exploration of its cross-cultural application. Theoretical and empirical perspectives on forgiveness and forgiveness promoting psychotherapy are provided, and Enright’s pioneering process model is delineated. A case study of the utilization of forgiveness therapy with an Israeli Bedouin-Arab woman who experienced a traumatic interpersonal injury within the polygamous family is presented. LaRoche and Maxie’s guidelines for conducting cross-cultural psychotherapy are utilized to examine important dynamics that emerge in the case study. These authors’ recommendations, which are consistent with a process-oriented conceptualization of culture, as well as intersectionality, appear potentially useful for informing the transcultural application of forgiveness therapy. Future research questions and directions related to the utilization of forgiveness promoting psychotherapy with diverse populations are considered.     

Evaluation of a cross-sector community initiative partnership: Delivering a local sport program

Corporate community initiatives (CCI) are often established via cross-sector partnerships with nonprofit agencies to address critical social problems. While there is a growing body of literature exploring the effectiveness and social impact of these partnerships, there is a limited evaluative research on the implementation and execution processes of CCIs. In this paper, we examined the implementation and operational processes in the delivery of a professional sport organization's CCI initiative using program theory evaluation. The findings showed discrepancies between the associate organization and the implementers regarding understanding and fulfilling responsibilities with performing certain aspects (maintaining accurate records and program marketing) of the service delivery protocol. Despite program stakeholders being satisfied overall with the program delivery, contradictions between program stakeholders’ satisfaction in the quality of program delivery was found in critical components (marketing and communications) of the service delivery. We conclude that ongoing evaluations are necessary to pinpoint the catalyst of the discrepancies along with all partners valuing process evaluation in addition to outcome evaluation.     

“Compassion,pleasantry, and hope”: A process evaluation of a volunteer-based nonprofit

As funders continue to emphasize the importance of documented results, nonprofit organizations must work to complete program evaluations that are both valuable and feasible. The purpose of this paper is to document a practical process evaluation of a southeastern nonprofit, a local Meals on Wheels. Using a mixed methods approach, we sought to answer four evaluation questions: (1) What are the essential program components, as identified by key stakeholders; (2) To what extent are volunteers implementing the identified essential components as intended; (3) What is the level of volunteer satisfaction with the program; and (4) What suggestions do stakeholders have for improving the program? Our findings indicate that most aspects of the program were implemented as intended, but inconsistencies occurred when volunteers were unsure of their assigned duties. In addition, volunteers had high levels of satisfaction and specific suggestions for improvement. From these results, we developed a conceptual model of factors contributing to quality of implementation and volunteer satisfaction that may be generalizable to other volunteer-based nonprofits. Specifically, we identified three factors that helped to facilitate satisfaction and performance: leadership, social contact, and fulfillment. Finally, this process evaluation demonstrates the feasibility of developing and implementing evaluation tools in similar organizations.     

Participatory research: Three models and an analysis

This article examines three models of participatory research: what we call the parallel process model, the mutual engagement model, and the University of Central America (UCA) model. These models represent successively greater degrees of academic engagement with outside communities—from complementary, though not necessarily uncommitted, engagement by academic(s); to compromised full engagement between the academic(s) and the community; to institutional engagement between the entire university and the community. Our analysis outlines the tensions that may arise within participatory research between service and scholarship. We conclude that for participatory research to capture the attention and involvement of the broader discipline, it must provide a spectrum of theory, methods, and substance that sociologists find of importance independent of the participatory way in which such contributions are generated. This article is based on a paper presented at the 87th Annual Meeting of the American Sociological Association, D.L. Lawrence Convention Center, Pittsburgh, Pennsylvania, August 20–24, 1992.     

The role of empathy in crisis communication: Providing a deeper understanding of how organizational crises and crisis communication affect reputation

This study advances our theoretical knowledge of how organizational crises and crisis communication affect reputation. Prior research solely emphasizes the importance of organizational crisis responsibility in this process. Three experiments show that stakeholders’ empathy toward the organization provides a second explanation. The first two experiments demonstrate that victim crises not only inflict less reputational damage than preventable crises because stakeholders consider the organization less responsible for the events, but also because they are more likely to empathize with the company. The third study shows that empathy can also explain the outcomes of crisis communication. An apology arouses empathy among stakeholders and subsequently increases reputation repair, unlike denial. The role of empathy in the crisis communication process has implications for both theory and practice.     

The unique character of involvement in multi-site evaluation settings

As the number of large federal programs increases, so, too, does the need for a more complete understanding of how to conduct evaluations of such complex programs. The research literature has documented the benefits of stakeholder participation in smaller-scale program evaluations. However, given the scope and diversity of projects in multi-site program evaluations, traditional notions of participatory evaluation do not apply. The purpose of this research is to determine the ways in which stakeholders are involved in large-scale, multi-site STEM evaluations. This article describes the findings from a survey of 313 program leaders and evaluators and from follow-up interviews with 12 of these individuals.Findings from this study indicate that attendance at meetings and conferences, planning discussions within the project related to use of the program evaluation, and participation in data collection should be added to the list of activities that foster feelings of evaluation involvement among stakeholders. In addition, perceptions of involvement may vary according to breadth or depth of evaluation activities, but not always both. Overall, this study suggests that despite the contextual challenges of large, multi-site evaluations, it is feasible to build feelings of involvement among stakeholders.     

Building communities: The postmodern CSR practitioner as a dialectical activist-agent in India

This study examined the conceptualization of the postmodern public relations practitioner as an organizational activist who contributes to democratic processes in the context of community relations and corporate social responsibility (CSR) in India. Elite, in-depth conversations with 19 senior executives at 16 companies in India well known for their CSR initiatives revealed that practitioners appeared to be both organizational activists and agents. As organizational activists, they acknowledged the existence of diverse voices in local communities, interjected these voices into management discourse, situated decision-making in local contexts, identified tensors in the relationship between the corporation and its publics, and negotiated new meanings through dissensus. Paradoxically, as organizational agents, they used these participatory, open processes of dialogic communication to shape public opinion in favour of the organization, feeding modern organizations’ proclivity for consensus. While the findings of the study support the postmodern conceptualization of the public relations practitioner as an organizational activist in the context of CSR and community relations, the agent aspect of the activist-agent dialectic problematizes and complicates this conceptualization thus enhancing understandings of the postmodern practitioner who enacts CSR within a modernist framework and navigates the intricacies of the activist-agent dialectic in their daily performance of building community relations and enabling democratic processes.     

Planning and implementation of a participatory evaluation strategy: A viable approach in the evaluation of community-based participatory programs addressing cancer disparities

Community-based participatory research (CBPR) has been posited as a promising methodology to address health concerns at the community level, including cancer disparities. However, the major criticism to this approach is the lack of scientific grounded evaluation methods to assess development and implementation of this type of research. This paper describes the process of development and implementation of a participatory evaluation framework within a CBPR program to reduce breast, cervical, and colorectal cancer disparities between African Americans and whites in Alabama and Mississippi as well as lessons learned. The participatory process involved community partners and academicians in a fluid process to identify common ground activities and outcomes. The logic model, a lay friendly approach, was used as the template and clearly outlined the steps to be taken in the evaluation process without sacrificing the rigorousness of the evaluation process. We have learned three major lessons in this process: (1) the importance of constant and open dialogue among partners; (2) flexibility to make changes in the evaluation plan and implementation; and (3) importance of evaluators playing the role of facilitators between the community and academicians. Despite the challenges, we offer a viable approach to evaluation of CBPR programs focusing on cancer disparities.     

Seeking consensus: Defining foundational concepts for a graduate level introductory program evaluation course

Existing scholarship indicates a wide range of variance and inconsistency in how evaluation is taught in introductory-level graduate courses between and across instructors and universities and within programs. The purpose of this study was to empirically explore faculty, student, and employers’ perceptions of what should be included in a graduate level introductory course on program evaluation. The researchers used a mixed methods approach (survey and focus group) to build consensus on and articulate key elements of an introductory evaluation course, as well as the approximate amount of time that stakeholders feel should be invested in each topic area. The results suggested consistency among stakeholders, suggesting the topics covered could form the basis for an introductory course that is consistent across instructors, programs, and institutions.     

Evaluation of competence-based teaching in higher education: From theory to practice

Competence-based teaching in higher education institutions and its evaluation have become a prevalent topic especially in the European Union. However, evaluation instruments are often limited, for example to single student competencies or specific elements of the teaching process. The present paper provides a more comprehensive evaluation concept that contributes to sustainable improvement of competence-based teaching in higher education institutions. The evaluation concept considers competence research developments as well as the participatory evaluation approach. The evaluation concept consists of three stages. The first stage evaluates whether the competencies students are supposed to acquire within the curriculum (ideal situation) are well defined. The second stage evaluates the teaching process and the competencies students have actually acquired (real situation). The third stage evaluates concrete aspects of the teaching process. Additionally, an implementation strategy is introduced to support the transfer from the theoretical evaluation concept to practice. The evaluation concept and its implementation strategy are designed for internal evaluations in higher education and primarily address higher education institutions that have already developed and conducted a competence-based curriculum.     

Empathy as a main ingredient of impactful crisis communication: The perspectives of crisis communication practitioners

Sixteen in-depth interviews with Belgian crisis communication practitioners were conducted to get their perspectives on the role of empathy in crisis communication. While demonstrating empathy toward crisis-affected stakeholders is considered an essential best practice, the role of empathy in responding to crises has only received limited attention in research. The aims of this study were to examine how practitioners interpret the concept of empathy in crisis communication; to what extent they value the expression of empathy in crisis responses and for what reasons; and what exactly constitutes an empathetic crisis response according to them. In addition, instances in which stakeholders may in turn experience empathy for organizations in crisis were examined. As such, the aspiration behind this study was to provide insights that can initiate further theorization and research on the role of empathy in the context of organizational crisis communication.