Urban community as resource: Evaluation of the mentors in Toledo schools program

We report findings from an evaluation of the Mentors in Toledo Schools program (MITS), which pairs adult community volunteers with elementary school students in need of reading support. Begun in 2012, MITS now operates in five elementary schools in Toledo Public Schools. A quasi-experimental study of 379 students (n = 128 mentored, n = 251 comparison) tested the effectiveness of MITS, using changes (fall to spring) in students’ scores on STAR Reading. Multilevel models demonstrate students who participated in MITS made statistically significant, and educationally meaningful, reading gains compared to students who did not participate. Further, an investigation of the importance of dosage revealed that number of mentoring sessions is related to students’ outcomes; students who participated in reading mentoring at least once per week showed the most educationally meaningful improvements. We also report qualitative feedback from adult program participants, including teachers, mentors, and site coordinators, to help contextualize the STAR Reading results. These findings speak to the opportunity for community volunteers to act as low-cost and effective supports for urban elementary school students who are at-risk for reading failure.     

Towards an assessment for organizational participatory research health partnerships: A systematic mixed studies review with framework synthesis

Within the health sciences, organizational participatory research (OPR) is defined as a blend of research and action, in which academic researchers partner with health organization members. OPR is based on a sound partnership between all stakeholders to improve organizational practices. However, little research on the evaluation of OPR health partnership exists. This systematic mixed studies review sought to produce a new theoretical model that structures the evaluation of the OPR processes and related outcomes of OPR health partnerships. Six bibliographic databases were searched together with grey literature sources for OPR health partnership evaluation questionnaires. Six questionnaires were included, from which a pool of 95 OPR health partnership evaluation items were derived. The included questionnaires were appraised for the quality of their origin, development and measurement properties. A framework synthesis was performed using an existing OPR framework by organizing questionnaire items in a matrix using a hybrid thematic analysis. This led to our proposed Organizational Participatory Research Evaluation Model (OPREM) that includes three axes, Trust, Collective Learning and Sustainability (with specific dimensions) and 95 items. This model provides information to help stakeholders comprehensively structure the evaluation of their partnerships and subsequent improvement; thus, potentially helping to improve health organization practices.     

Using logic model and visualization to conduct portfolio evaluation

Portfolio evaluation is the evaluation of multiple projects with a common purpose. While logic models have been used in many ways to support evaluation, and data visualization has been used widely to present and communicate evaluation findings, adopting logic models for portfolio evaluation and using data visualization to share findings simultaneously is surprisingly limited in the literature. With the data from a sample portfolio of 209 projects which aims to improve the system of early care and education (ECE), this study illustrated how to use logic model and data visualization techniques to conduct a portfolio evaluation by answering two evaluation questions: “To what extent are the elements of a logic model (strategies, sub-strategies, activities, outcomes, and impacts) reflected in the sample portfolio?” and “Which dominant paths through the logic model were illuminated by the data visualization technique?” For the first question, the visualization technique illuminated several dominant strategies, sub-strategies, activities, and outcomes. For the second question, our visualization techniques made it convenient to identify critical paths through the logic model. Implications for both program evaluation and program planning were discussed.     

Using theory of change as a framework for examining community context and philanthropic impact

As part of the process of examining their theory of change (TOC) and reflecting on grant making activity, one Midwestern foundation employed a multipronged strategy to assess 209 community-based mental health grants across seven years of funding. This article details the evaluation approach, which comprised these areas of the TOC: grantees’ use of evidence-based interventions, cultural competency, quality improvement, community collaboration, and use of integrated care. Inductive analyses identified grantees’ use of innovative practices, trauma-informed care, and use of validated instruments. In an iterative exercise spanning multiple years, the foundation and researchers found ways to gather information about community capacity and impact while simultaneously minimizing grantee data collection/reporting burden. Findings revealed that community agencies were improving in their uptake of evidence-based interventions and use of validated instruments. Community grantee use of trauma informed care also improved over the study period. Grantee desire to collaborate and coordinate services within the community was also strong. The research also revealed that in multiple domains (such as technology and non-profit operating costs) grantees needed support in building agency capacity. These findings shed light on which aspects of the foundation’s TOC were being addressed through funding, and which areas of the community needed additional support.     

Using a community-created multisite evaluation to promote evaluation use across a sector

Multisite evaluation has been cited as a strategy for building knowledge about a program's success and enhancing evaluation capacity and evaluation use. Even so, the extent to which data from these efforts are made available to individual project participants varies. This article describes a community-created multisite evaluation that was designed to overcome this limitation. Based on the negotiated centralized evaluation model and implemented within the context of a community of practice, the EvalFest community-created multisite evaluation has used shared measures to collect data from 30,000 participants from partner sites across the community over three years. This article describes the design characteristics of the community-created multisite approach, and presents results related to partners’ evaluation use that demonstrate the early success of this method. We conclude by considering how this strategy might be applied to other sectors.     

Fostering CHANGE: Lessons from implementing a multi-country,multi-sector nutrition-sensitive agriculture project

Using agriculture to improve nutrition is an approach growing in popularity, with programs becoming increasingly complex and multisectoral. While there is an active line of research assessing the impacts of such programs, little has been written about the process of successfully implementing them. As such, this paper uses a multisectoral nutrition-sensitive agriculture program implemented in four African countries as a case study to address key challenges in and lessons learned from implementation. We highlight the overall flexibility of nutrition-sensitive agriculture but also the need to adapt certain aspects to the particular context, as well as the opportunities for cross-context learning (and the limits to this). Integrating rigorous evaluation into such complex programs and forging diverse cross-sectoral partnerships offer both rewards and challenges, upon which we reflect. Main lessons learned from the program include the importance of carefully sequencing interventions, retaining flexibility in implementation, allowing for considerable time for cross-sector integration and coordination, and considering community impacts when designing research.     

基于权变理论的高校教师评价的探讨

研究与分析了我国高校现行的奖惩性评价和发展性评价两种教师评价模式,指出两种模式的优点及弊端,并结合现行的高校发展状况,探讨在新的形势下的基于权变理论指导下的高校教师评价模式,旨在将个人发展与组织发展相结合,合理利用奖惩性和发展性评价,促进教师的专业发展和学校的发展。     

青藏高原特色农业国际竞争力评价指标体系的构建

基于农业国际竞争力评价指标体系相关文献回顾的基础上,从国内比较优势指标、显示性指标、解释性指标和实体性指标四个方面提出了青藏高原特色农业国际竞争力的评价标准,建立了包括4个一级指标、12个二级指标和24个三级指标的特色农业国际竞争力的评价指标体系,以期为进一步研究青藏高原特色农业发展提供参考.