Building national M&E systems in the context of changing aid modalities: The underexplored potential of National Evaluation Societies

Changes in the aid architecture have provided a renewed impetus for monitoring and evaluation (M&E), while simultaneously imposing a major reform agenda on the key players involved. More specifically, since 1999, aid-dependent countries have been facing pressure to strengthen their national M&E systems, while donors have been asked to refrain from using their own parallel systems and to rely instead on country systems. Surprisingly, attempts to strengthen national M&E frameworks have thus far largely overlooked the potential of national evaluation societies (NES). Similarly, NES have also remained off the academic radar. Our study aims to fill this gap by mapping key features of NES, as well as their perceived contributions to country-led M&E. In this effort, we rely upon evidence from our survey of 23 NES in Sub-Saharan African countries with Poverty Reduction Strategy Papers (PSRP). Our findings show that there is quite some diversity among NES. Overall, NES are active organisations, whose unique membership features a wide variety of national M&E stakeholders who potentially play key roles in country-led and localised M&E development. Major obstacles faced by NES include the lack of financial resources, donor support and political influence. Survey findings also demonstrate that the increasing interest of donors and governments in NES has yet to materialise into strategic support.     

A critical friend: Monitoring and evaluation systems,development cooperation and local government. The case of Tuscany

The role of monitoring and evaluation (M&E) systems in the field of development cooperation has globally increased in last decades. International and regional organizations, as well as states, local governments and NGOs have largely adopted the tools provided by M&E in order to enhance transparency, effectiveness and efficiency. The paper aims at verifying how and to what extent the implementation of M&E systems has affected the overall quality of international cooperation projects financed by a local government. After a literature review on M&E in development cooperation, the research analyzes the wide range of activities (evaluation ex-ante, mid-term, final, monitoring, consultancy) carried out by the Evaluation Team of the XY in the last eight years in behalf of an Italian local government: the Region of Tuscany. The paper reveals the strategic significance of adopting M&E systems in the medium-long term.     

Advances in monitoring and evaluation in low- and middle-income countries

Data to inform and improve health care systems in low- and middle-income countries (LMICs) has been facilitated by the development of monitoring and evaluation (M&E) systems. The drivers of change in M&E systems over the last 50 years have included a series of health concerns that have animated global donors (e.g., family planning, vaccination campaigns, and HIV/AIDS); the data requirements of donors; improved national economies enabling LMICs to invest more in M&E systems; and rapid advances in digital technologies. Progress has included the training and expansion of an M&E workforce, the creation of systems for data collection and use, and processes for assessing and ensuring data quality. Controversies have included the development of disease-specific systems that do not coordinate with each other, and a growing burden on health care deliverers to collect data for a proliferating number of health and process indicators. Digital technologies offer the promise of real time data and quick adaptation but also raise ethical and privacy concerns. The desire for speed can cast large-scale evaluations, considered by some to be the gold standard, in an unfavorable light as slow and expensive. Accordingly, there is a growing demand for speedy evaluations that rely on routine health information systems and privately collected “big data” from electronic health records and social media.     

A call for reconfiguring evaluation models,pedagogy, and practice: Beyond reporting media-centric outputs and fake impact scores

Evaluation is an essential part of strategic communication practices including public relations and corporate, government, and organizational communication. Evaluation can potentially demonstrate effectiveness and value, facilitate learning to improve practice, and provide insights to inform future strategy. In the age of digitalization of media and information, there are more readily available data to enable evaluation than ever before. However, despite half a century of theorizing and discussing measurement and evaluation (M&E), literature reports “stasis” and a “deadlock” well into the 21st century In some cases, invalid practices such as advertising value equivalents (AVEs) continue to be used. Furthermore, when M&E collectively is part of communication practice, evidence shows that reports continue to focus on outputs and algorithm-generated fake impact scores, rather than genuine outcomes and impact. Despite the body of literature on M&E, it is clear that more needs to be said – and done. This discussion calls for reconfiguring common evaluation models to overcome false logics and for a reconfiguring and broadening of pedagogy and practice to break the deadlock and realize the benefits of evaluation.     

Assessment of the monitoring and evaluation system of a population-based program: Theory-driven evaluation approach

The purpose of this study was to assess the performance of a monitoring and evaluation system (M&E) for a population-based program and to identify factors influencing its performance. The state health department provided a grant, guide, and training for health districts to implement the program and report quarterly and annually to the M&E. Some health districts did not meet this requirement. We also found that the M&E did not adequately capture the actual work of the districts in communities as related to the work plan. We identified several factors that negatively affected the performances of the M&E. For example, the state and health districts did not have a shared vision on how to develop and implement the program, and what to report to the M&E. We proposed strategies that could improve the timeliness and content of reports to the M&E, as well as its utilization in the future.     

Monitoring and evaluation design of Malawi’s Right Foods at the Right Time nutrition program

Child stunting is a public health problem in Malawi. In 2014, the Government of Malawi launched the Right Foods at the Right Time (RFRT) program in Ntchisi district delivering nutrition social and behavior change communication, a small-quantity lipid-based nutrient supplement to children 6–23 months, and nutrition sensitive activities. Monitoring and evaluation (M&E) systems are key aspects of successful program implementation. We describe these and the methodology for an impact evaluation that was conducted for this program. Two monitoring systems using traditional and electronic platforms were established to register and track program delivery and processes including number of eligible beneficiaries, worker performance, program participation, and to monitor input, output, and outcome indicators. The impact evaluation used comparative cross-sectional and longitudinal designs to assess impact on anthropometric and infant and young child feeding outcomes. Three cross-sectional surveys (base-, mid-, and end-line) and two longitudinal cohorts of children followed in 6-month intervals from 6 to 24 months of age, were conducted in sampled households in the program and a neighboring comparison district. Additional M&E included qualitative studies, a process evaluation, and a cost-effectiveness study. The current paper describes lessons from this program’s M&E, and demonstrates how multiple implementation research activities can inform course-correction and program scale-up.     

Strategically creative: A case of the library planning process

Abstract

Best planning day ever, green, fun, play, flexible … are terms not usually associated with strategic planning. At the University of Technology, Sydney (UTS), plans to open a new library building in 2016 have ignited discussions about the library of the future, its meaning and role in a digital world, and the implications of further rapid changes. The Library's strategic planning process in 2010 was an opportunity to take the collective energy further by involving library staff in the strategic thinking and planning. The process aimed to provide conditions for open conversation through Gadamerian play, and creative modelling based on ideas of revealing and imagineering. The planning started with a playful engagement to generate ideas and promote divergent thinking, and moved through several stages of increasingly convergent thinking to arrive at strategic actions, which were created and supported by all participants. The formal goals of strategic planning were maintained throughout the process and resulted in a client-focused annual plan. This article considers issues of organizational creativity and strategic planning, and relates them to experiences with the strategic planning at the UTS Library. It argues for a systematic approach to fostering creativity and innovation in libraries.     

Reviewing the ‘march to standards’ in public relations: a comparative analysis of four seminal measurement and evaluation initiatives

To many, development and adoption of professional standards for measurement and evaluation (M&E) is one of the most promising approaches for advancing public relations practice. In recent years, there has been a surge in efforts to develop standards for M&E in different parts of the world. Prominent examples of this include standard terminologies, metrics, principles for best practice in the field, and evaluation frameworks. Regardless of their alleged importance, however, the acceptance and application of such M&E standards in the practice varies significantly. To better understand the process by which standards in this field are developed and adopted, this article draws on recent concepts from organization studies (cf. Slager, Gond, & Moon, 2012) to analyze the trajectories of four seminal standards attempts: The Barcelona Principles and the AMEC Integrated Evaluation Framework on an international level, the DPRG/ICV Framework used in German-speaking countries, and the GCS Framework in the United Kingdom. The article reveals, by way of an interpretive qualitative approach, the various strategies undertaken to a) develop common sets of terms and rules, b) engage relevant actors in the design, promotion, and implementation of proposed standards, and c) to reinforcing standards symbolically.     

SenseMaker® as a monitoring and evaluation tool to provide new insights on gender-based violence programs and services in Lebanon

Monitoring and evaluation (M&E) of gender-based violence (GBV) programs is challenging in humanitarian settings. To address these challenges, we used SenseMaker® as a mixed methods M&E tool for GBV services and programs in Lebanon. Over a three-month period in 2018, a total of 198 self-interpreted stories were collected from women and girls accessing GBV programs from six service providers across five locations. The resultant mixed-methods analysis provided holistic and nuanced insights on how perceived benefits differed by type of GBV program, how motivations for accessing programs differed by location, and how feelings while accessing programs differed by participant nationality. SenseMaker reinforced the intersectionality between events leading up to the accessed services, the experiences of accessing the services, and subsequent outcomes as a result of having accessed the services, helping to contextualize the findings within the broader experiences of participating women and girls. Limited literacy and technology skills among participants proved to be a challenge and future work should investigate how technology might facilitate use of the tool among participants with lower literacy and technology skills in addition to exploring the feasibility and added value of SenseMaker as an M&E tool in acute humanitarian settings.     

A Strategic Planning Process for a Small Nonprofit Organization: A Hospice Example

Strategic planning is an essential part of management. However, planning processes can consume great amounts of time and resources that small, nonprofit organizations may lack. Moreover, the process that is used can be tedious and may result in plans that are discarded before or during their implementation. In this article, a strategic planning process is presented that incorporates a Policy Delphi group technique and Situation Structuring, a computer program that assists participants in structuring or defining the problems to be addressed in the plan. The organization to which the process is applied is a small, nonprofit hospice. Both the planning process and an evaluation of the implementation of the resultant strategic plan are examined.     

Video–based observation in impact evaluation

This paper reviews the literature on the use of video–based observation (VBO) with particular attention to monitoring and evaluation (M&E) on development projects. While the use of video both as a research tool and as a strategy for supporting professional development is well–documented across several disciplines, the extent to which VBO has been utilized in M&E contexts is less clearly defined. In order to provide theoretically–grounded recommendations for the development and implementation of one organisation’s innovative VBO impact evaluation scheme, this review examines recent evidence of VBO in M&E contexts and draws on VBO literature more generally to identify its advantages and challenges together with advice for enhancing its effectiveness. Based on this analysis, the paper highlights a number of practical issues that should be considered when VBO is being developed for M&E in development contexts, particularly where videos are being made by participants themselves. The value of VBO in responding to COVID–19 and reducing carbon emissions is also noted.     

Asian urban sustainability in the era of globalization

The process of achieving urban sustainable development is uncharted. We only know that plans should address the economic, environmental and social health of the city and this task can only be accomplished by approaching each of these issues at different scales. For rapidly developing world cities, “sustainability” is becoming an increasingly elusive objective, in part, because of impacts by forces beyond their borders. Using the Asia-Pacific region as a case study, a framework relates regional transnational flows to the state of the urban environment and the social conditions of linked rapidly developing cities. The “functional city system” within the Asia-Pacific increasingly is both the engine of urban growth and the force behind differentiating urban environmental and social issues. At the same time, while globalization forces have been particularly strong within cities in the Asia-Pacific, local factors also play a crucial role in urban development. Globalization driven growth has not translated into a single path of development, rather localities have demonstrated contextually specific paths.     

Taking Stock of Monitoring and Evaluation Arrangements in the Context of Poverty Reduction Strategy Papers: Evidence from 20 Aid‐Dependent Countries in Sub‐Saharan Africa

Shifts in thinking and practice concerning aid have posed serious challenges in monitoring and evaluation (M&E) for both donors and recipients, but progress in reform remains slow. An important first step in any initiative involving M&E capacity development is the diagnosis of the systems' current status. This article presents a diagnostic checklist that captures issues of M&E policy: indicators, data collection and methodology; organisation; capacity‐building; participation of nongovernmental actors; and use. It applies it to a review of the PRSP M&E arrangements of 20 aid‐dependent countries in sub‐Saharan Africa to demonstrate comparative strengths and weaknesses.     

The Manageable Approach to College Health Services Planning Part I

Abstract

College health services must function effectively within two dynamic and complex environments-higher education and health. College health services are also increasingly expected to respond to emerging forces within the fields of adolescent medicine and college health itself and to seize opportunities to improve the health status and lifestyles of college-age adults. Thus, it is essential that college health services be well planned to meet student needs more completely and to ensure a more appropriate role for college health in both the health and higher education fields. This article depicts the strategic and operational planning that is necessary and delineates how such planning can be accomplished within 10 weeks through the application of the “Manageable Approach.” Basic concepts of the Manageable Approach are presented, its six major stages are described, and detailed guides to the success of each stage are included. These guides focus on the purposes and results of each stage, the group maintenance activities necessary, the specific steps to be followed, the special techniques to be employed, and the planning aids to be used in the discussion of substantive issues. In this first part of a two-part series, strategic and operational health planning are described, the Manageable Approach is introduced, and the first two stages—organization and education, and mission and program goals—are delineated.     

Planning from below: using feminist participatory methods to increase women's participation in urban planning

Women have been often invisible in urban planning. Col·lectiu Punt 6, a feminist organisation of women architects and urban planners based in Barcelona, Spain, has developed planning techniques and tools to include women both as objects and subjects of urban planning, and as experts of everyday life. The article focuses on the participatory tools that Col·lectiu Punt 6 has developed that can be used in the different stages of planning, from assessing to evaluating planning policies and interventions, as well as tools of women's empowerment with the goal of promoting bottom-up models of planning. Evidence is shown of how, using these tools, planning becomes more inclusive not only of women in their diversity, but also of children, youth, the elderly, as well as people of different race, ethnicity, income, sexual and gender identities, and capacities. Examples from different contexts are provided, from Spain and Latin America.     

Design of a collaborative monitoring and evaluation system for a community-based nutrition project in rural Bangladesh

Poor diet and undernutrition are common among children living in Bangladesh. To promote appropriate complementary feeding of young children, an economic development (ED) program involving income-generating asset transfer was implemented alongside a social and behavior change (SBC) program. This paper introduces a collaborative monitoring and evaluation (M&E) system in which diverse collaborators (“research group”, “implementation team”, and “coordinators”) facilitate M&E data acquisition by leveraging their comparative advantages. The implementation team built a monitoring system to track the ED (n = 2960) and SBC participants (n=∼10,000) over 12 months. Based on the baseline design and the monitoring records, the collaborators planned an impact evaluation introducing a quasi-experimental design using two cross-sectional surveys and a prospective cohort survey of child feeding and nutritional status. Using various data sources generated from the M&E system, the collaborators will also reveal the program impact pathway through which each intervention component is delivered, received, and utilized alongside the context-specific facilitators and barriers, including the programs’ uptake. The collaborative M&E system enables the sharing of program goals, strengthens collaborators' commitment to the program, and extends the understanding of the program's progress and evaluation activities.     

Institutional constraints on housing development: The urban areas of Nigeria: The land-use decree and the building plan approval process

This paper argues that the land-use decree and the building approval process have become additional institutional bottlenecks which prospective urban housing developers must contend with. Contrary to these experiences, these two para-legal institutions were made to ease in part, the difficulties associated with housing development in the urban areas in Nigeria. The paper then briefly analyses how these institutions operate that made them cogs in the wheels of housing development. Some remedial measures suggested to obliterate the difficulties associated with these institutional constraints include: the possession, acquisition and servicing of urban land for sale to the general public; the need for a complete land information system through the use of cadastral survey; public enlightenment with regard to the requirements of the planning authorities on local building plans; and the provision of official vehicles or car loans to site inspectors.     

Mediating role of citizen empowerment in the relationship between participatory monitoring and evaluation and social sustainability

The paper explores the mediating influence of citizen empowerment on the relationship between participatory monitoring and evaluation (PM&E) and social sustainability as a contribution to the ongoing debate on sustainability. The empirical investigation took the form of mixed-methods approach with the objectives to determine the extent to which PM&E influences citizen empowerment; establish the extent to which citizen empowerment influences social sustainability; and establish the joint influence of PM&E and citizen empowerment on social sustainability. The study employed a concurrent parallel design, in which samples for quantitative and qualitative components were different but drawn from the same population, and data collected within the same timeframe. Quantitative data were analyzed through bivariate and multiple regression analyses. Conversely, the qualitative component utilized interpretive technique and recursive abstraction to organize, summarize and give impression on the causal mechanisms at play in the quantitative data. Evidence from the study revealed that citizen empowerment mediates the relationship between PM&E and social sustainability. The study also found that when PM&E and citizen empowerment are considered together, citizen empowerment seems to have a dominant influence on social sustainability than PM&E itself. Thus, the government and development practitioners should prioritize citizen empowerment as a precursor to the realization of social sustainability outcomes.     

Physical planning within the framework of National Development Planning in Nigeria

Nigeria has had experience with 5 Development Plans. In each of these successive national and state development plans, mention is made of physical planning problems — especially the urbanisation problem in Nigeria — a problem compounded by the ever-increasing rate of rural to urban migration. In this paper, an attempt will be made to review the priority given to physical planning in each of the successive National Development Plans, to examine the constraints on effective physical planning in Nigeria, and to propose strategies that will ensure proper integration of physical planning with national development planning.     

From Iraq to Katrina and Back: Bureaucratic Planning as Strategic Failure,Fiction, and Fantasy

Bureaucratic planning is a complex sociocultural process rife with vested interests, contradictory aims, and internal conflicts. In the wake of Hurricane Katrina and the US occupation of Iraq, contemporary media discourses on planning have downplayed this complexity and diverted attention away from a more nuanced understanding of planning as a sociological process. In this article, I show how policy and press debates regarding bureaucratic planning in Iraq and New Orleans are rooted in technocratic and rationalist perspectives, constituting a kind of modernist faith. These contemporary discourses are often one‐dimensional, failing to consider the conflicts, contradictions, and counter‐intuitive outcomes that are inherent in planning in complex situations, including urban reconstruction, military occupation, and disaster response. I show how flawed approaches in the present are deeply linked to the past, connecting Iraq and Katrina to a detailed ethnographic case study on failed urban planning in colonial Zanzibar. Even though the three cases of Iraq, Katrina, and Zanzibar differ across time, context, and conditions, they reveal that bureaucratic planning is best understood as a multidimensional strategic practice that is intrinsically shaped by sociocultural forms. Plans may ostensibly fail in one sense and yet still produce quite enduring effects. They may remain ‘fictions’ at one level but still wield real impact. And even if they are informed by fantasy, bearing little utility from a functional perspective, plans still serve as performative and persuasive acts, informed by political ends.