An assessment of a municipal physical activity programme for seniors based on the EFQM model: Integrating the views of internal and external stakeholders

The growing proportion of senior citizens and the importance of encouraging active well-ageing behaviours explain the increasing efforts carried out by governments to develop exercise programmes particularly targeted to this group. The complexity associated with the running of these programmes and the resources committed make their quality assessment essential. The aim of the current paper is to validate and refine the use of the EFQM model in this context. A programme of a medium-sized municipality of the Centre region of Portugal was used as a case study to evaluate the adequacy and relevance of the EFQM model and to identify the strengths and weaknesses of the current programme based on the data collected from service users, instructors, municipal leaders and programme partners. High levels of customer satisfaction and loyalty, together with the clarity of programme mission and strategy emerged as main areas of strength. On the other hand, insufficient attention to the measurement of people satisfaction, as well as the leaders’ failure of to show appreciation for the efforts developed by the programme instructors are key areas for improvement.     

Evaluation of competence-based teaching in higher education: From theory to practice

Competence-based teaching in higher education institutions and its evaluation have become a prevalent topic especially in the European Union. However, evaluation instruments are often limited, for example to single student competencies or specific elements of the teaching process. The present paper provides a more comprehensive evaluation concept that contributes to sustainable improvement of competence-based teaching in higher education institutions. The evaluation concept considers competence research developments as well as the participatory evaluation approach. The evaluation concept consists of three stages. The first stage evaluates whether the competencies students are supposed to acquire within the curriculum (ideal situation) are well defined. The second stage evaluates the teaching process and the competencies students have actually acquired (real situation). The third stage evaluates concrete aspects of the teaching process. Additionally, an implementation strategy is introduced to support the transfer from the theoretical evaluation concept to practice. The evaluation concept and its implementation strategy are designed for internal evaluations in higher education and primarily address higher education institutions that have already developed and conducted a competence-based curriculum.     

The African Evaluation Guidelines: 2002. A checklist to assist in planning evaluations, negotiating clear contracts, reviewing progress and ensuring adequate completion of an evaluation

A review of the US ‘program evaluation standards’ (PES), undertaken in a series of workshops and meetings of networks of evaluators in Africa, resulted in modifications to those standards. The result was presented to a plenary session of the Inaugural Conference of the African Evaluation Association in September 1999, attended by over 300 evaluators from 35 countries. The AfrEA Conference decided that a systematic effort should be made to produce a list of African evaluation guidelines, similar to the PES, and that this checklist should be reviewed by national evaluation associations and networks in Africa and field tested in several countries. Ten national and regional networks and associations suggested modifications to the text and endorsed the final version of the guidelines.     

Developing a methodology to assess the impact of research grant funding: A mixed methods approach

This paper discusses the development of a mixed methods approach to analyse research funding. Research policy has taken on an increasingly prominent role in the broader political scene, where research is seen as a critical factor in maintaining and improving growth, welfare and international competitiveness. This has motivated growing emphasis on the impacts of science funding, and how funding can best be designed to promote socio-economic progress. Meeting these demands for impact assessment involves a number of complex issues that are difficult to fully address in a single study or in the design of a single methodology. However, they point to some general principles that can be explored in methodological design. We draw on a recent evaluation of the impacts of research grant funding, discussing both key issues in developing a methodology for the analysis and subsequent results. The case of research grant funding, involving a complex mix of direct and intermediate effects that contribute to the overall impact of funding on research performance, illustrates the value of a mixed methods approach to provide a more robust and complete analysis of policy impacts. Reflections on the strengths and weaknesses of the methodology are used to examine refinements for future work.     

高职院校专业教学质量评估体系构建研究

在职业教育下,建立专业教学质量评估体系是专业建设中较高层次的内容。文章对专业教学质量评估体系建设提出一些看法,并围绕日常教学运行管理中的环节、学生技能评估指标体系设计、职业资格证书的要求及就业质量评价指标分析等方面进行论述,为保障专业人才培养质量提供具体可操作的指导建议。     

Quality and rigor of the concept mapping methodology: a pooled study analysis

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.     

Evaluation of a cross-cultural training program for Pakistani educators: Lessons learned and implications for program planning

In this paper, we share the results of a summative evaluation of PEILI, a US-based adult professional development/training program for secondary school Pakistani teachers. The evaluation was guided by the theories of cultural competence (American Psychological Association, 2003, Bamberger, 1999, Wadsworth, 2001) and established frameworks for the evaluation of professional development/training and instructional design (Bennett, 1975, Guskey, 2002, King, 2014, Kirkpatrick, 1967). The explicit and implicit stakeholder assumptions about the connections between program resources, activities, outputs, and outcomes are described. Participant knowledge and skills were measured via scores on a pre/posttest of professional knowledge, and a standards-based performance assessment rubric. In addition to measuring short-term program outcomes, we also sought to incorporate theory-driven thinking into the evaluation design. Hence, we examined participant self-efficacy and access to social capital, two evidenced-based determinants or “levers” that theoretically explain the transformative space between an intervention and its outcomes (Chen, 2012). Data about program determinants were collected and analyzed through a pre/posttest of self-efficacy and social network analysis. Key evaluation findings include participant acquisition of new instructional skills, increased self-efficacy, and the formation of a nascent professional support network. Lessons learned and implications for the design and evaluation of cross-cultural teacher professional development programs are discussed.     

Utilizing the RE-AIM framework to determine effectiveness of a preschool intervention program on social-emotional outcomes

Social-emotional issues in preschoolers continue to be an area of concern across the nation. Models to determine effective implementation practices are needed. The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework is one model that can be used to evaluate program implementation in preschool settings. The Jump Start program provided short-term intervention to children in low-income preschools from multiethnic backgrounds. The goal was to promote prosocial behaviors while minimizing problem behaviors. Various evidenced-based practices (i.e., I Can Problem Solve, play therapy, Pyramid Model) were utilized. 305 children from 73 childcare centers participated in the program. The majority of participants were from ethnic minority backgrounds and resided in high-poverty areas of the county. The RE-AIM framework was utilized to determine program outcomes. Results showed successes on each level of RE-AIM with an at-risk population. The Jump Start program significantly increased positive behaviors and decreased challenging behaviors in preschool-aged children. Results indicated medium to large effect sizes. One year following program participation, the majority of children who participated in the program were at decreased risk of special education services and expulsion.     

Contribution analysis to analyze the effects of the health impact assessment at the local level: A case of urban revitalization

The health impact assessment (HIA) is a tool used to estimate the potential impact on health of non-health-related proposals prior to implementation. While it is increasingly used in Quebec, Canada, studies have not analyzed its medium-term impacts and potential long-term impacts. We conducted a contribution analysis using in-depth interviews with key stakeholders, as well as documents, observation and images related to HIA in order to analyze its impacts on the revitalization of road infrastructure, parks and green spaces, and residential housing. Our analysis not only reflects on the decision-making process through the adoption and implementation of HIA recommendations, but also on the link between actions implemented in the field and health outcomes.     

The impact of the crisis on homelessness in Spain: social intervention in a context of greater need and fewer resources

The purpose of this article is to deepen the understanding of the impact of the social and economic crisis on homeless people in Spain. To this end we present the results of research carried out prior to the crisis and later updated in the midst of the economic recession, underscoring the main results obtained in both periods and tracing their evolution. The number of homeless people in Spain has risen as a result of the crisis. Moreover, while some characteristics of this group remain the same, others have changed. Hence, difficulties in access to the labour market and cuts in certain economic benefits are resulting in an increase in the duration of social inclusion processes. This results in a worsening of the personal situation of the homeless, especially in relation to mental illness and substance abuse. The immigrant population is the group most affected by these impacts. The situation clearly demonstrates the need to reflect on how to deal with the problem of homelessness in Spain, and to rethink the way professionals intervene, especially during periods of economic recession.     

Challenges related to the analytical process in realist evaluation and latest developments on the use of NVivo from a realist perspective

ABSTRACT

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some documented challenges and solutions when analyzing RE data, including the development of Context-Mechanism-Outcome configurations. An analytic method using NVivo features is also presented. This method makes it possible to respond to certain analytic difficulties associated with RE by facilitating the identification of patterns and ensuring transparency in the analytical process.     

Child welfare workers go to court: The impact of race, gender, and education on the comfort with legal issues

This study examines perceptions of 425 public and private agency child welfare workers from one state in terms of their level of comfort with the court work components of their jobs, at baseline, which is after their initial training but before beginning their child welfare work, and again after six months on the job. Bivariate and multivariate analyses were undertaken. Bivariate analyses reveal that being older and male were associated with higher court comfort levels at baseline and at six months. Being white was associated with higher comfort levels at six months. In addition, private agency workers were more comfortable with court at baseline and six months. Workers with a social work education had higher court comfort levels at baseline and six months, when compared to those with criminal justice or other education. In the multivariate analysis, there were no significant differences in level of comfort with court work based upon worker characteristics at baseline. However, at six months on the job, predictors of comfort with child welfare work were being white and being male; having a social work education approached statistical significance. When change in comfort level was examined, the only significant positive predictor of court comfort was having a social work education.     

Micro-simulation of households: a new tool to assess the impact on society of food safety policies

Public concerns about the risk from food hazards have spread in recent years, boosted by a number of well-publicized food scares. As a result, food safety issues are high on the policy agenda in Europe and elsewhere, and a range of policies have been developed to deal with them. The paper illustrates and makes a case for the use of micro-simulation models as support for better economic assessment of food policies. When households deal with food safety issues they can show heterogeneous behaviors, and micro-simulation can provide a useful tool to take into account this heterogeneity. However, to the author's knowledge, no attempt to use micro-simulation in the analysis of food safety policies has taken place so far. After an illustration of the way economists evaluate food safety policies, micro-simulation methodologies are introduced, stressing their potentiality for the assessment of food safety policies.     

Happiness of Hong Kong youth from 2000 to 2014: empirical evidence on the differential impact of socioeconomic conditions on youth versus other age groups

This paper aims to explain why Hong Kong youth have become unhappier than older people in recent years. We test in this connection six hypotheses, using random effects ordinal logistic regressions with a combined macro-micro dataset from official statistics and a longitudinal opinion survey project from 2000 to 2014. The results show that the decline in the happiness level in recent years is more significant for Hong Kong youth than for older people. We also find evidence of age heterogeneity in the impacts of both macroeconomic conditions and the housing price on happiness. GDP per capita has a weaker positive effect on youth’s happiness than it does on older people. The positive correlation between housing price and happiness is valid for older people only. Rapid economic growth and the rise in the price of housing have made older people happier than youth since the early 2010s.     

Successful adoption and implementation of a comprehensive casework practice model in a public child welfare agency: Application of the Getting to Outcomes (GTO) model

In recent years, several states have been developing or adopting casework practice models in an effort to shape the thinking and behavior of front line child welfare workers with a commitment to improving the safety, permanency and well-being outcomes of vulnerable children in their care (Antle, Christensen, Barbee & Martin, 2008; Christensen, Todahl & Barrett, 1999; Courtney, 2009; Folaron, 2009). This article presents one framework for approaching the organizational changes that need to be made in order to support a practice model. The Getting to Outcomes” Framework (Wandersman, 2009) is a useful approach for ensuring that all areas to support practice change are addressed.     

Patterns of practice: a reflection on the development of quantitative/mixed methodologies capturing everyday life related to water consumption in the UK

There is a growing body of research arguing the relevance of practice approaches to understand resource consumption, and to highlight alternative pathways to sustainability. These practice approaches offer an alternative conceptualisation of demand and have been demonstrated largely by qualitative research, particularly in the work on water and energy consumption in the home. However, these historical narratives and qualitative research have not, to date, lead to the development of quantitative or mixed methodologies that could potentially reflect the diversity of performances of practice across populations in a more systematic way. This paper reflects, critically, on one such attempt to scale a practice-based perspective into a quantitative survey on water consumption and practice in homes in the south and South-East of England. The use of quantitative and mixed methodology has substantial potential – from translating practice-based research to policy; developing indictors to track patterns of practices as they change over time; and the exploration of methodologies that reflect the bundling and coordination of practices associated with water use inside and outside the home. The benefits and utility of such a methodological approach are highlighted as cautions and future research directions.