Zimbabwe's AIDS action programme for schools

AIDS education started very early in Zimbabwe and is now compulsory in all schools. This evaluation documents the development of the primary and secondary school AIDS Action Programme from 1991 to 1998. Six programme aspects were evaluated: programme start-up, planning and management, development of syllabi and materials, teacher training, research, monitoring and evaluation, and co-ordination. The Programme drew on resources from within the existing educational system. Due to broad-based consultation and participation, it was supported by Government and partners. Flexible management ensured implementation of mid-course corrections. It was concluded that school AIDS programmes should stress participatory teaching and learning methods and life-skills training. Curriculum writers and teachers needed training and supervision in participatory techniques. Cluster workshops between district and school levels were needed to strengthen the cascade model of teacher training that had been adopted. Systematic research, monitoring and evaluation were essential, including follow-ups to baseline surveys. The Zimbabwean programme was well-designed, relatively inexpensive and replicable.     

Income generation and Facts for Life communication in Vietnam

In1997, UNICEF Vietnam conducted a participatory evaluation, Pathways Forward, to assess the impact of the Income Generation and Facts for Life (FFL) Communication Project, which communicates basic health messages to women who are also participating in a revolving loan scheme. Twenty-five people from UNICEF and the Vietnam Womens Union participated in the month-long evaluation. They used a combination of case studies, interviews, surveys, and social indicators to determine the projects impact on women finishing their final loan cycle in two project communes. A second group studied the projects microcredit component in order to recommend a financial strategy for the future.Evaluators found that while the project had successfully met many of its objectives, the degree of its success varied dramatically between the two communes studied. Further, the evaluation found the projects operational capacity needed to be strengthened so as to support existing microcredit activities.Based on these findings, the Pathways evaluation recommended that UNICEF postpone any future loan disbursements and focus instead on building the projects capacity through increased staffing levels and additional training. It also recommended a number of measures be taken to strengthen FFL communication activities, including training for group leaders and simpler, locally-produced FFL materials.     

Participatory analytic hierarchy process for resource allocation in agricultural development projects

In the framework of 2030 Agenda sustainable agriculture plays a central role. In the field of international aid the participatory approaches to assessment, research, management and budgeting have been widely studied and applied in the last decades, mostly because international aid initiatives, in all their relevant phases from planning through implementation, monitoring and evaluation, are faced with the problem of identifying initiatives that could be successfully and sustainably implemented. The present paper investigates the use of a modified form of analytic hierarchy process, namely the participatory analytic hierarchy process, as a tool for choice criteria elicitation and resource allocation in the framework of an operational planning for agricultural development projects in the Dioceses of Goma, Nord Kivu, Democratic Republic of Congo. During our research, we were able to make use of most frequent inconsistencies in pairwise comparison matrixes in order to stimulate the debate, to adjust local preferences and to build consensus across the group. From an operational point of view, the PAHP methodology was also suited for training the project team and for identifying a shared resource allocation pattern, which matches the existing international guidelines for agricultural development in the region.     

Qualitative evaluation of HIV/AIDS IEC activities in Madagascar

UNICEF commissioned a qualitative evaluation of a program in which two partners conducted prevention activities concerning HIV/AIDS and STIs amongst youth in Madagascar. The evaluation assessed project activities and conducted a practical exercise in focus group methodology, following a workshop on evaluation techniques. Information was collected from a wide range of people, including 441 participants in 46 focus groups in nine sites. Six sites were areas of project activities and three sites were targeted for future activities. Clubs that integrated HIV/AIDS and STI information into their regular activities were most successful in their information, education and communication activities. A more participatory approach is recommended, to achieve a better match between target groups and messages delivered, and to foster trust and support. Another challenge remaining is to address fears and stigma through advocacy for the care and support of affected people.     

Cross Cultural Programming for AIDS Orphans: What We are Learning from Other Contexts

The problem of AIDS orphans in Zambia has reached alarming proportions because of the extent of poverty and poor social and economic policies. Worldwide, 15 million children have been orphaned due to AIDS, with 11.6 million orphans in sub-Saharan Africa alone (UNICEF, The State of the World’s Children 2009: maternal and newborn health, UNICEF, New York, 2008), and 670,000 children under the age of 17 in Zambia (UNAIDS, UNAIDS report on the global AIDS epidemic, UNAIDS, Geneva, 2010). Resulting from this situation are child headed households (CHHs) who face hunger, poor health, sanitation and water problems. Despite the challenges, there seems to be an absence of political will to support CHHs. To assess if Africans living in Richmond Virginia in the United States might offer something programmatically useful for CHHs in Zambia, important elements in the Richmond Independent Living services model were identified. Specific elements were screened using Africans as a cultural screen. Important implications for programming and practice suggest that feasibility, content and quality are key areas for appropriate CHHs programming. In this article, the terms orphans, vulnerable children, and young people are used interchangeably.     

Making Connections: Community Organizing,Empowerment Planning,and Participatory Research in Participatory Evaluation

This article shows the intersections of participatory research, popular education, empowerment planning, and community organizing with participatory evaluation. It argues that a truly successful participatory evaluation involves participants in guiding and even conducting the research, doing a process of self- and program study, creating plans for change, and organizing themselves for implementing these plans. Next, the chapter shows how these elements played out in a participatory evaluation of a community organizing training and technical assistance project in Toledo, Ohio. The first year of the project was facilitated by participatory evaluation that helped identify early successes and problems so participants could make programmatic changes early in the process. The telling of the story also develops practices of participatory evaluation, including planning the evaluation, doing the research and adapting it to changing conditions, uncovering creative tensions, participatory validity checking, and linking the process to planning and action. The chapter concludes with some lessons for participatory evaluation practice.     

Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches

Collaborative Evaluation systematically invites and engages stakeholders in program evaluation planning and implementation. Unlike "distanced" evaluation approaches, which reject stakeholder participation as evaluation team members, Collaborative Evaluation assumes that active, on-going engagement between evaluators and program staff, result in stronger evaluation designs, enhanced data collection and analysis, and results that stakeholder understand and use. Among similar "participant-oriented" evaluation approaches (Fitzpatrick, Sanders, & Worthen, 2011), Collaborative Evaluation distinguishes itself in that it uses a sliding scale for levels of collaboration. This means that different program evaluations will experience different levels of collaborative activity. The sliding scale is applied as the evaluator considers each program's evaluation needs, readiness, and resources. While Collaborative Evaluation is a term widely used in evaluation, its meaning varies considerably. Often used interchangeably with participatory and/or empowerment evaluation, the terms can be used to mean different things, which can be confusing. The articles use a comparative Collaborative Evaluation Framework to highlight how from a theoretical perspective, Collaborative Evaluation distinguishes itself from the other participatory evaluation approaches.     

Constructing collaborative processes through experiential learning: Participatory planning in Kaymaklı, Turkey

Prevalent discussions in urban planning focus on collaboration. However, they do not explain the underlying processes that allow collaborative work. This paper argues that experiential learning processes can help construct and maintain collaborative platforms in urban planning. To show this, the paper uses the participatory planning of Kaymaklı, a small agricultural and touristic town in the internationally well-known cultural landscape, the Cappadoccia region. This entails the second stage of a two-year long Participatory Action Research project initiated by a local civil association in collaboration with the Municipality of Kaymaklı. The paper presents how the local capacity for collaboration is built through the development of local knowledge in a practice-based training programme. The attention is particularly directed to the pedagogic design of the training programme, the shared knowledge that came out of it, the affects of the programme on the collaborative capacity of participants, and their future collaborative action.     

Participatory research and the voice‐centred relational method of data analysis: is it worth it?

Much has been written about participatory approaches in planning and implementing research. The process and practice of participatory data interpretation and analysis is, however, less discussed. This article explores the value of the Voice‐Centred Relational (VCR) method of data analysis and interpretation in the context of participatory research with socially excluded teenagers. Focussing on participatory data analysis, the implementation of the method with teenagers in a West of Ireland town who have left school early is described. Written from the perspectives of academic researchers, members of a research team that included teenagers and a Community Arts artist, this article introduces the VCR method and describes the participatory processes, practices and pitfalls. The challenges of doing participatory interpretation with VCR method are considered.     

Evaluation of HIV/AIDS prevention through peer education, counselling, health care, training and urban refuges in Ghana

Since 1996, Ireland Aid has supported UNICEF Ghana in the implementation of five activities that promote behaviour change to limit the spread of HIV/AIDS and STIs. The interventions are run by different local organisations, and have provided over 75,000 in and out-of-school youth and commercial sex workers with preventive education. People retained the information given, are knowledgeable about how HIV is transmitted, and can name key preventive methods. They pass on the information to their friends. Commercial sex workers were empowered to support one another in negotiating for safer sex. The programme was supported by community members, leading to demand for condoms and for education on HIV/AIDS/STIs. One project gave support to pregnant girls, including health care and vocational training. A drawback in the programme has been the occasional delays in the supply of the educational materials to the peer educators.     

Effects of a participatory ergonomics intervention computer workshop for university students: a pilot intervention to prevent disability in tomorrow's workers

A participatory approach was used to create a computer ergonomics workshop for college students, incorporating an instructional systems design process and adult learning inquiry perspectives. The primary objective of this participatory ergonomic pilot intervention was to involve students throughout the training design process in solving computer workstation ergonomic problems and adopting healthy computing behaviors. Students' level of participation included becoming part of the training design team, a co-facilitator, or a student trainee. A second objective was to examine the translation of an industrial office ergonomics training program into a college computer ergonomics training program. The long term goal was to reduce upper extremity symptoms and disability. The program was piloted at one private university. The three student trainees significantly increased their knowledge of computer ergonomics from 69% of test items answered correctly pre-training to 82% post-training. Trainees were also successful in conducting computer ergonomic evaluations of students' computing work areas. They achieved 100% accuracy in identifying ergonomic problems and proposing solutions in five ergonomic workstation assessments in the field. This approach was successful in creating a sense of ownership among the student developers and facilitators as reflected in their self-reports during a post-intervention debriefing. The results of this pilot study justify formal controlled trials of this intervention in university students, who will become tomorrow's workers.     

Exploring the Potential of Shadow Controls in the Evaluation of Children’s Services

This article describes and presents a framework for an under‐used evaluation technique in the context of an evaluation of a programme for disaffected young people. Shadow controls—the use of expert judgement to estimate the success of a programme—are often dismissed in research design as an unreliable form of comparison, but can be useful in situations where there is limited scope for a control group or to enhance the causal inference attributable to non‐experimental evaluations. The exercise described uses a practice tool as a structure for making predictions about the situations of the young people on the programme, assuming they do not receive an intervention. These predictions (shadow controls) are then compared to outcome data for the young people at the end of the programme. The results of the exercise provide some important messages about the programme’s effectiveness and the potential for strengthening non‐experimental evaluation methods. The article also discusses how the method can usefully inform evaluations of social programmes and encourage agency and user collaboration.     

New initiatives in training

Training Update will bring you news from trainers and ideas about training. In each edition we plan to provide a round-up of currently available training materials around a particular theme, drawn from our evaluation programme.     

Evaluation plan of the 6for6 research skills program for rural and remote physicians

Overwhelming issues and barriers often prevent rural and remote physicians (RRPs) from pursuing the many socially accountable research questions they encounter on a daily basis. Although research training programs can empower RRPs to rise to these challenges, there is a lack of evidence on how they should be developed and refined. At Memorial University, a faculty development program (FDP) called 6for6 has been helping RRPs surmount their research quagmires and engage in scholarship since 2014. After an initial three-year (2014-17) pilot, we prepared a detailed plan to evaluate the 6for6 research FDP for RRPs and inform future years of delivery. Using a modified Delphi method and participatory action model a group of program team members, stakeholders and evaluation experts developed an evaluation plan including a logic model and an evaluation matrix addressing five key themes. To our knowledge, this is the first evaluation plan for a research-focused FDP targeting RRPs. While this plan was developed specifically for the 6for6 FDP, our approach to its development may be useful to any institution interested in evaluating an FDP with limited resources.     

The participatory research approach in non‐Western countries: practical experiences from Central Asia and Zambia

This paper focuses on the application of the participatory research approach in non‐Western contexts. The aim is to provide critical insights into the participatory research discourse through an examination of its theory and practice based on our own experiences of using this approach in our doctoral research in five Central Asian countries and Zambia. Firstly, we summarise the published literature on the approach in both disability and development studies which are our academic disciplines. Secondly, we critically analyse some of the challenges we came across in applying participatory research in our doctoral studies in practice. By this we wish to contribute to making this approach more viable and to increase research participation by disabled persons in non‐Western contexts. Finally, based on our own experiences we give some suggestions for the use of the participatory research approach in non‐Western contexts.     

A meta-evaluation, or quality assessment, of the evaluations in this issue, based on the African Evaluation Guidelines: 2002

Fourteen evaluations of UNICEF and UNAIDS HIV/AIDS programmes underwent a quality control process called ‘meta-evaluation’, which is an evaluation of the quality of the evaluation. This process of evaluating an evaluation is considered professional good practice by African Evaluation Associations and Networks, as well as the American, Canadian, German, Korean, Swiss and other national professional associations of evaluators. The quality control criteria used were the African Evaluation Guidelines—a list of 30 quality-enhancing standards that can be used as a checklist by evaluators to ensure that evaluation studies are performed well. These quality control criteria can also be applied retrospectively as a quality control method, to assess completed evaluation studies. The results of this quality control review process, for the 14 papers in this issue of Evaluation and Program Planning, are presented and discussed. An indication of a possible area of weakness in HIV/AIDS programme evaluation emerges from this analysis.     

Active participation in preventionist professional practices: a specific ergonomics training course

Currently, the different forms of corporate management methods (quality circles, lean manufacturing, etc.) are usually based on employee mobilisation. Very often, the goal of this type of approach is at best, to ensure that employees embrace corporate projects, or otherwise to impose changes on them without taking into account the real work and difficulties that they face daily. However, do these employee solicitation methods converge with participatory approaches as envisaged by ergonomists and more generally, preventionists? Based on the observation that the activity of institutional preventionists evolves with regulatory constraints and work related to the monitoring of indicators, the implementation and steering of the participatory approach within companies may be a major lever for prevention. After describing the foundations of a participatory approach, this paper will present a training experiment aimed at implementing and promoting employee participation in prevention efforts. The content of this training is then analysed with regard to the balance between methodology and the use of trainees' narratives. The results of the training session are presented from the point of view of the development prospects for pedagogical tools and the organisation of the training.     

Time out for team practice: structured participation as a method of training for team maintenance and development

The social Work service in most Social Services Departments is organised in distinct geographically based units. Indeed, such units have become synonymous with social work teams. These “teams” rely heavily for their success, as responsive and resourceful units, on their ability to work together. The authors believe there are advantages in a collective approach to a team's decision making and service delivery. They describe a method for developing teamwork, centred around a study day, organised on the basis of structured participation and planned in a way which addresses a range of team concerns.

The programme used on the study day stresses the importance of clear objectives, agreed democratically, and the active involvement of all team members. Although this training method developed gradually and pragmatically it is based on a model of team development which identifies a progression through stages of team functioning. The model of learning emphasises change through participation in a climate of openness and trust.

Examples of a programme used by the authors, together with a report prepared as feedback to a team, are included as appendices.     

Using process evaluation to determine effectiveness of participatory ergonomics training interventions in construction

The construction industry continues to experience high rates of musculoskeletal injuries despite the widespread promotion of ergonomic solutions. Participatory ergonomics (PE) has been suggested as one approach to engage workers and employers for reducing physical exposures from work tasks but a systematic review of participatory ergonomics programs showed inconclusive results.. A process evaluation is used to monitor and document the implementation of a program and can aid in understanding the relationship between the program elements and the program outcomes. The purpose of this project is to describe a proposed process evaluation for use in a participatory ergonomic training program in construction workers and to evaluate its utility in a demonstration project among floor layers.