Bilateral Donors' Efforts to Assess and Manage UN Agencies' Effectiveness: Seeking a Balance between Paris Principles and Domestic Accountability

Since the turn of the century, results orientation and performance measurement have become buzzwords in development aid. Bilateral donors are increasingly concerned about the effectiveness of core‐funded UN specialised agencies, funds and programmes, through which they channel a substantial part of their funds. While the logic for increased monitoring is clear, it is less evident how to assess this information for purposes of accountability, performance‐based funding and governance. This article describes the general evolution in this area and discusses two major initiatives to harmonise and align bilateral assessments, using case‐study material from three bilateral donors (Belgium, the Netherlands and the UK) with clearly diverging performance‐management cultures, and exploring the extent to which they rely upon joint and aligned initiatives.     

Evaluation of a multi-site program designed to strengthen relational bonds for siblings separated by foster care

Sibling relationships in foster care settings have received increased attention in recent years. Despite growing evidence regarding the protective potential of sibling relationships for youth in care, some sibling groups continue to experience foster care related separation, and few programs exist to address the needs of these youth.This study describes and evaluates Camp To Belong, a multi-site program designed to provide short-term reunification to separated sibling groups through a week-long summer camp experience. Using a pre-test post-test survey design, this paper examines changes in youth ratings of sibling conflict and sibling support across camps located in six geographically distinct regions of the United States. The effects of youth age, number of prior camp exposures, and camp location were tested using multilevel modeling procedures.Findings suggest that participation in Camp To Belong may reduce sibling conflict, and improvements in sibling support are noted for youth who have had prior exposure to the camp’s programming. Camp-level variance in the sibling support outcome highlight the complex nature of relationships for siblings separated by foster care, and suggest the need for additional research. Lessons learned from this multi-site evaluation and future directions are discussed.     

学分绩点制在武汉七所部属高校的实践与探索

学分绩点制是一种能客观衡量学生学习质量的教学管理制度。学分绩点制在武汉地区七校联合办学实践中的应用,为联合办学的可持续发展提供了有益的探索。     

C330. The variability of evaluations of speculative ideas

This article reports results of an extensive simulation study which investigated the performances of some commonly used methods of estimating error rates in discriminant analysis. Earlier research papers limited their comparisons of these methods to independent training data. This study allows for a simple auto-regressive dependence among the training data. The results suggest that the estimation methods based on the normal distribution perform adequately well under conditions of negative or mild positive correlation in the data, and small dimensions (p) of the observation vectors. For large p or strong positive correlation structures the conclusion is that one of the better non-parametric methods should be used. Special circumstances and conditions which notably affect the relative performances of the methods are identified.     

The role of titles in enhancing data visualization

Many in the data visualization and evaluation communities recommend conveying the message or takeaway of the visualization in the visualization's title. This study tested that recommendation by examining how informative or generic titles impact a visualization’s visual efficiency, aesthetics, credibility, and the perceived effectiveness of the hypothetical program examined. Furthermore, this study tested how simple or complex graphs, and positive, negative, or mixed results (i.e., valence of the results) affected outcomes. Participants were randomly assigned to one of 12 conditions, representing a 2 (graph: simple or complex) x 2 (title: generic or informative) x 3 (valence: positive, negative, mixed) between-subjects study. The results indicated that informative titles required less mental effort and were viewed as more aesthetically pleasing, but otherwise did not lead to greater accuracy, credibility, or perceived effectiveness. Furthermore, titles did not interact with graph type or the valence of the findings. While the results suggest it is worthwhile to consider adding an informative title to data visualizations as they can reduce mental effort for the viewer, the intended goal of the visualization should be taken into consideration. Considering the goal of the visualization can be a deciding factor of the type of graph and title that will best serve its intended purposes. Overall, this suggests that data visualization recommendations that impact evaluation reporting practices should be scrutinized more closely through research.     

Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria

Logic models (LMs) have been used in programme evaluation for over four decades. Current debate questions the ability of logic modelling techniques to incorporate contextual factors into logic models. We share experience of developing a logic model within an ongoing realist evaluation which assesses the extent to which, and under what circumstances a community health workers (CHW) programme promotes access to maternity services in Nigeria. The article contributes to logic modelling debate by: i) reflecting on how other scholars captured context during LM development in theory-driven evaluations; and ii) explaining how we explored context during logic model development for realist evaluation of the CHW programme in Nigeria. Data collection methods that informed our logic model development included documents review, email discussions and teleconferences with programme stakeholders and a technical workshop with researchers to clarify programme goals and untangle relationships among programme elements. One of the most important findings is that, rather than being an end in itself, logic model development is an essential step for identifying initial hypotheses for tentative relevant contexts, mechanisms and outcomes (CMOs) and CMO configurations of how programmes produce change. The logic model also informed development of a methodology handbook that is guiding verification and consolidation of underlying programme theories.     

Toward a model of school inspections in a polycentric system

Many education systems are developing towards more lateral structures where schools collaborate in networks to improve and provide (inclusive) education. These structures call for bottom-up models of network evaluation and accountability instead of the current hierarchical arrangements where single schools are evaluated by a central agency. This paper builds on available research about network effectiveness to present evolving models of network evaluation. Network effectiveness can be defined as the achievement of positive network level outcomes that cannot be attained by individual organizational participants acting alone. Models of network evaluation need to take into account the relations between network members, the structure of the network, its processes and its internal mechanism to enforce norms in order to understand the achievement and outcomes of the network and how these may evolve over time. A range of suitable evaluation models are presented in this paper, as well as a tentative school inspection framework which is inspired by these models. The final section will present examples from Inspectorates of Education in Northern Ireland and Scotland who have developed newer inspection models to evaluate the effectiveness of a range of different networks.     

Eliciting children’s recall regarding home life and relationships

Abstract

Child interviews form an important component of custody evaluations. Yet, research on children’s responses to questions about home life and relationships is lacking. In the present study, children (N?=?47) aged 6 to 10?years were interviewed about their daily routines and family relationships. Responses to four categories of questions were compared: open and specific questions about routines, and negative and positive aspects of family relationships. Responses were coded for amount of information, informativeness, topic pertinence, and refusals to answer. Results suggested that questions about everyday routines and relationships elicit relevant and informative responses from children. It is suggested that interviewers begin with open-ended questions regarding daily routines to structure family law interviews with children.     

Coastal Adaptation Planning in Fairbourne,Wales: lessons for Climate Change Adaptation

ABSTRACT

Adaptation to climate change is emerging as a central objective and policy frame for coastal communities. This paper examines recent climate change adaptation efforts in the UK, centring on the case of Fairbourne, Wales. The village is facing the long-term prospect of flooding and inundation due to the impacts of sea level rise. The recent Shoreline Management Plan for the area has recommended realignment of the coast and eventual decommissioning of Fairbourne. The paper draws on a qualitative research methodology of interviews, policy review and observation to narrate the case and provide key insights and lessons for planners working in environmentally vulnerable coastal settings and managing climate adaptation processes.     

Experiences with systematic triangulation at the Global Environment Facility

Systematic triangulation may address common challenges in evaluation, such as the scarcity or unreliability of data, or the complexities of comparing and cross-checking evidence from diverse disciplines. Used to identify key evaluation findings, its application has proven to be effective in addressing the limitations encountered in country-level evaluation analysis conducted by the Independent Evaluation Office of the Global Environment Facility (GEF). These include the scarcity or unreliability of national statistics on environmental indicators and data series, especially in Least Developed Countries; challenges in evaluating the impacts of GEF projects; and inherent difficulties in defining the GEF portfolio of projects prior to the undertaking of the evaluation. In addition to responding to the need for further developing triangulation protocols, procedures and/or methodologies advocated by some authors, the approach offers a contribution to evaluation practice. This applies particularly to those evaluation units tasked with country-level evaluations in international organizations, facing similar constraints.