C330. The variability of evaluations of speculative ideas

This article reports results of an extensive simulation study which investigated the performances of some commonly used methods of estimating error rates in discriminant analysis. Earlier research papers limited their comparisons of these methods to independent training data. This study allows for a simple auto-regressive dependence among the training data. The results suggest that the estimation methods based on the normal distribution perform adequately well under conditions of negative or mild positive correlation in the data, and small dimensions (p) of the observation vectors. For large p or strong positive correlation structures the conclusion is that one of the better non-parametric methods should be used. Special circumstances and conditions which notably affect the relative performances of the methods are identified.     

The role of titles in enhancing data visualization

Many in the data visualization and evaluation communities recommend conveying the message or takeaway of the visualization in the visualization's title. This study tested that recommendation by examining how informative or generic titles impact a visualization’s visual efficiency, aesthetics, credibility, and the perceived effectiveness of the hypothetical program examined. Furthermore, this study tested how simple or complex graphs, and positive, negative, or mixed results (i.e., valence of the results) affected outcomes. Participants were randomly assigned to one of 12 conditions, representing a 2 (graph: simple or complex) x 2 (title: generic or informative) x 3 (valence: positive, negative, mixed) between-subjects study. The results indicated that informative titles required less mental effort and were viewed as more aesthetically pleasing, but otherwise did not lead to greater accuracy, credibility, or perceived effectiveness. Furthermore, titles did not interact with graph type or the valence of the findings. While the results suggest it is worthwhile to consider adding an informative title to data visualizations as they can reduce mental effort for the viewer, the intended goal of the visualization should be taken into consideration. Considering the goal of the visualization can be a deciding factor of the type of graph and title that will best serve its intended purposes. Overall, this suggests that data visualization recommendations that impact evaluation reporting practices should be scrutinized more closely through research.     

考虑储存成本的集装箱公铁联运路径优化

联运企业的效益受运输路径和运输方式的影响。如何降低联运成本、提高联运企业的效益,已成为学术研究的重中之重。针对集装箱公铁联运路径优化问题,以运输成本、换装成本、储存成本、时间惩罚成本构成的运输总成本为目标函数,建立集装箱公铁联运路径优化模型,考虑运输时间限制、班期限制、节点储存成本,得出使运输总成本最低的运输路径及运输方式,并利用算例验证模型的有效性。     

Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria

Logic models (LMs) have been used in programme evaluation for over four decades. Current debate questions the ability of logic modelling techniques to incorporate contextual factors into logic models. We share experience of developing a logic model within an ongoing realist evaluation which assesses the extent to which, and under what circumstances a community health workers (CHW) programme promotes access to maternity services in Nigeria. The article contributes to logic modelling debate by: i) reflecting on how other scholars captured context during LM development in theory-driven evaluations; and ii) explaining how we explored context during logic model development for realist evaluation of the CHW programme in Nigeria. Data collection methods that informed our logic model development included documents review, email discussions and teleconferences with programme stakeholders and a technical workshop with researchers to clarify programme goals and untangle relationships among programme elements. One of the most important findings is that, rather than being an end in itself, logic model development is an essential step for identifying initial hypotheses for tentative relevant contexts, mechanisms and outcomes (CMOs) and CMO configurations of how programmes produce change. The logic model also informed development of a methodology handbook that is guiding verification and consolidation of underlying programme theories.     

Toward a model of school inspections in a polycentric system

Many education systems are developing towards more lateral structures where schools collaborate in networks to improve and provide (inclusive) education. These structures call for bottom-up models of network evaluation and accountability instead of the current hierarchical arrangements where single schools are evaluated by a central agency. This paper builds on available research about network effectiveness to present evolving models of network evaluation. Network effectiveness can be defined as the achievement of positive network level outcomes that cannot be attained by individual organizational participants acting alone. Models of network evaluation need to take into account the relations between network members, the structure of the network, its processes and its internal mechanism to enforce norms in order to understand the achievement and outcomes of the network and how these may evolve over time. A range of suitable evaluation models are presented in this paper, as well as a tentative school inspection framework which is inspired by these models. The final section will present examples from Inspectorates of Education in Northern Ireland and Scotland who have developed newer inspection models to evaluate the effectiveness of a range of different networks.     

1981—2014年中国资本服务流量测算——基于截断正态退役分布和几何役龄效率剖面

通过梳理资本核算理论框架,结合中国实际情况测量了1981-2014年资本服务流量。资本分类、组合役龄效率-退役剖面计算、资本回报率确定和初始单位效率生产性资本存量估计等方面与既有研究有所不同。测量结果表明:按照1990年不变价格,各类资本服务流量呈增加趋势,其中无形资产服务流量前期较为波动;设备类服务流量增速最高;总资本服务增长率前期呈V型后期呈倒U型,与GDP增速变动不一致,整体来看前者高于后者;比较发现测量结果高于其它研究,差距先收敛后扩大。     

Eliciting children’s recall regarding home life and relationships

Abstract

Child interviews form an important component of custody evaluations. Yet, research on children’s responses to questions about home life and relationships is lacking. In the present study, children (N?=?47) aged 6 to 10?years were interviewed about their daily routines and family relationships. Responses to four categories of questions were compared: open and specific questions about routines, and negative and positive aspects of family relationships. Responses were coded for amount of information, informativeness, topic pertinence, and refusals to answer. Results suggested that questions about everyday routines and relationships elicit relevant and informative responses from children. It is suggested that interviewers begin with open-ended questions regarding daily routines to structure family law interviews with children.     

Experiences with systematic triangulation at the Global Environment Facility

Systematic triangulation may address common challenges in evaluation, such as the scarcity or unreliability of data, or the complexities of comparing and cross-checking evidence from diverse disciplines. Used to identify key evaluation findings, its application has proven to be effective in addressing the limitations encountered in country-level evaluation analysis conducted by the Independent Evaluation Office of the Global Environment Facility (GEF). These include the scarcity or unreliability of national statistics on environmental indicators and data series, especially in Least Developed Countries; challenges in evaluating the impacts of GEF projects; and inherent difficulties in defining the GEF portfolio of projects prior to the undertaking of the evaluation. In addition to responding to the need for further developing triangulation protocols, procedures and/or methodologies advocated by some authors, the approach offers a contribution to evaluation practice. This applies particularly to those evaluation units tasked with country-level evaluations in international organizations, facing similar constraints.     

An evaluation capacity building toolkit for principal investigators of undergraduate research experiences: A demonstration of transforming theory into practice

This paper describes the approach and process undertaken to develop evaluation capacity among the leaders of a federally funded undergraduate research program. An evaluation toolkit was developed for Computer and Information Sciences and Engineering¹ Research Experiences for Undergraduates² (CISE REU) programs to address the ongoing need for evaluation capacity among principal investigators who manage program evaluation. The toolkit was the result of collaboration within the CISE REU community with the purpose being to provide targeted instructional resources and tools for quality program evaluation. Challenges were to balance the desire for standardized assessment with the responsibility to account for individual program contexts. Toolkit contents included instructional materials about evaluation practice, a standardized applicant management tool, and a modulated outcomes measure. Resulting benefits from toolkit deployment were having cost effective, sustainable evaluation tools, a community evaluation forum, and aggregate measurement of key program outcomes for the national program. Lessons learned included the imperative of understanding the evaluation context, engaging stakeholders, and building stakeholder trust. Results from project measures are presented along with a discussion of guidelines for facilitating evaluation capacity building that will serve a variety of contexts.     

Impact of a Homonegativity Awareness Workshop on Attitudes Toward Homosexuality

The current study evaluated the effects of a homonegativity awareness workshop on attitudes toward homosexuality and examined individual difference variables associated with attitude change. Participants included 71 female and 43 male students (approximate mean age?=?23). Compared to a control group of introductory psychology students, participants were less homonegative and erotophobic (F(1, 359)?=?62.47 and 15.92, ps?<?.001) after the workshop. Implications for contact with gay and lesbian persons through structured intervention programs on attitudes toward homosexuality are discussed in terms of both the practice and theory of reducing prejudice and discrimination.