Evaluation of a cross-sector community initiative partnership: Delivering a local sport program

Corporate community initiatives (CCI) are often established via cross-sector partnerships with nonprofit agencies to address critical social problems. While there is a growing body of literature exploring the effectiveness and social impact of these partnerships, there is a limited evaluative research on the implementation and execution processes of CCIs. In this paper, we examined the implementation and operational processes in the delivery of a professional sport organization's CCI initiative using program theory evaluation. The findings showed discrepancies between the associate organization and the implementers regarding understanding and fulfilling responsibilities with performing certain aspects (maintaining accurate records and program marketing) of the service delivery protocol. Despite program stakeholders being satisfied overall with the program delivery, contradictions between program stakeholders’ satisfaction in the quality of program delivery was found in critical components (marketing and communications) of the service delivery. We conclude that ongoing evaluations are necessary to pinpoint the catalyst of the discrepancies along with all partners valuing process evaluation in addition to outcome evaluation.     

Planning and implementation of a participatory evaluation strategy: A viable approach in the evaluation of community-based participatory programs addressing cancer disparities

Community-based participatory research (CBPR) has been posited as a promising methodology to address health concerns at the community level, including cancer disparities. However, the major criticism to this approach is the lack of scientific grounded evaluation methods to assess development and implementation of this type of research. This paper describes the process of development and implementation of a participatory evaluation framework within a CBPR program to reduce breast, cervical, and colorectal cancer disparities between African Americans and whites in Alabama and Mississippi as well as lessons learned. The participatory process involved community partners and academicians in a fluid process to identify common ground activities and outcomes. The logic model, a lay friendly approach, was used as the template and clearly outlined the steps to be taken in the evaluation process without sacrificing the rigorousness of the evaluation process. We have learned three major lessons in this process: (1) the importance of constant and open dialogue among partners; (2) flexibility to make changes in the evaluation plan and implementation; and (3) importance of evaluators playing the role of facilitators between the community and academicians. Despite the challenges, we offer a viable approach to evaluation of CBPR programs focusing on cancer disparities.     

Coalition for a Healthier Community: Lessons learned and implications for future work

The Coalition for a Healthier Community (CHC) initiative was implemented to improve the health and well-being of women and girls. Underpinning CHC is a gender-based focus that uses a network of community partners working collaboratively to generate relevant behavior change and improved health outcomes. Ten programs are trying to determine whether gender-focused system approaches are cost-effective ways to address health disparities in women and girls. Programs implemented through coalitions made up of academic institutions, public health departments, community-based organizations, and local, regional, and national organizations, are addressing health issues such as domestic violence, cardiovascular disease prevention, physical activity, and healthy eating. Although these programs are ongoing, they have made significant progress. Key factors contributing to their early success include a comprehensive needs assessment, robust coalitions, the diversity of populations targeted, programs based on findings of the needs assessments, evaluations taking into consideration the effect of gender, and strong academic–community partnerships. A noteworthy impact of these programs has been their ability to shape and impact public, social, and health policies at the state and local levels. However, there have been challenges associated with the implementation of such a complex program. Lessons learned are discussed in this paper.     

Connecting the dots: implementing and evaluating a network intervention to foster scientific collaboration and productivity

This article presents the design and implementation of a network intervention to foster scientific collaboration at a research university, and describes an experimental framework for rigorous evaluation of the intervention’s impact. Based on social network analysis of publication and grant data, an innovative type of research funding program was developed as a form of alteration of the university’s collaboration network. The intervention consisted in identifying research communities in the network and creating a new collaborative relation between pairs of unconnected researchers in selected communities. The new collaboration was created to maximally increase the overall cohesion of the target research community. In order to evaluate the impact of the program, we designed a randomized experiment with treatment and control communities based on the Rubin Causal Model approach. The paper describes the intervention design, reports findings from the program implementation, and discusses the statistical framework for future evaluation of the intervention.     

30 years of ergonomics at 3M: a case study

The added value of the Ergonomics Program at 3M was found to be improved employee safety, compliance with regulations and reduction of work-related illness, increases in productivity, and quality and operating efficiency. This paper describes the thirty years of existence of this program. For the first twenty years, the program objectives were to: respond to requests for assistance related to work-related musculoskeletal disorder (WMSD) concerns, raise employee awareness of MSDs and ergonomics; educate engineers in ergonomics design; and develop ergonomics teams at manufacturing locations. Since the year 2000, 3M's Ergonomics Program has been in transition from a US-centric and corporate-based technical-expertled program to a global program applying participatory ergonomics strategies within a macroergonomics framework. During that transition, the existing program requirements were revised, new methods and program tools were created, and expectations for implementation at the manufacturing locations clarified. This paper focuses on the company's manufacturing ergonomics program activities during the past ten years and includes specifics of the program's objectives, risk assessment reduction process, and ergonomics technical expertise development. The main benefit achieved throughout the company is reducing employee injury while also increasing productivity and operating efficiency.     

Short‐Term‐Effectiveness of a Relationship Education Program for Distressed Military Couples,in the Context of Foreign Assignments for the German Armed Forces. Preliminary Findings From a Randomized Controlled Study

This study examines the short‐term effectiveness of a relationship education program designed for military couples. Distressed couples were randomly placed in either a wait‐list control group or an intervention group. We conducted training sessions before a 3‐month foreign assignment, and refresher courses approximately 6‐week post‐assignment. We analyzed the dyadic data of 32 couples, using hierarchical linear modeling in a two‐level model. Reduction in unresolved conflicts was found in the intervention group, with large pre–post effects for both partners. Relationship satisfaction scores were improved, with moderate‐to‐large effects only for soldiers, rather than their partners. Post‐follow‐up effect sizes suggested further improvement in the intervention group. Future research should examine the long‐term effectiveness of this treatment.     

Applying RE-AIM to evaluate two community-based programs designed to improve access to eye care for those at high-risk for glaucoma

IntroductionGlaucoma is a leading cause of vision loss and blindness in the U.S. Risk factors include African American race, older age, family history of glaucoma, and diabetes. This paper describes the evaluation of a mobile eye health and a telemedicine program designed to improve access to eye care among people at high-risk for glaucoma.MethodsThe RE-AIM (reach, efficacy, adoption, implementation, and maintenance) evaluation framework was used to harmonize indicators. Both programs provided community-based eye health education and eye services related to glaucoma detection and care. Each program reported data on participants and community partners. An external evaluator conducted site visit interviews with program staff and community partners. Quantitative and qualitative data were integrated and analyzed using the RE-AIM dimensions.DiscussionBy targeting high-risk populations and providing comprehensive eye exams, both programs detected a large proportion of new glaucoma-related cases (17–19%) – a much larger proportion than that found in the general population (<2%). The educational intervention increased glaucoma knowledge; evidence that it led people to seek eye care was inconclusive.ConclusionsEvaluation findings from the mobile eye health program and the telemedicine program may provide useful information for wider implementation in public health clinics and in optometrist clinics located in retail outlets.     

Chapter 3: Health and Productivity Management

SUMMARY

This article focuses on the emergence of Health and Productivity Management (HPM) as a context for understanding the integration of health and wellness into employee assistance and work-life programs. Major factors influencing the growth of HPM are raising medical costs as well as the rising incidence of obesity. The Wake-up Call to Corporate America underscores the need to manage these costs and improve health through the proactive alignment of health-related benefits and programs. HPM is described as an integrated approach to capture direct medical costs as well as the indirect costs associated with poor health and lost productivity. Critical design and implementation features that are likely to contribute to a successful integrated approach are described and highlighted. An HPM case example of a large pharmaceutical company examines a unique cross referral program including considerations for continuous improvement. Finally, the article concludes with key challenges related to marketplace competition, pricing pressures, concerns regarding the sub-optimization of programs and the influence of current health care system changes.     

The evolution of Rare Pride: Using evaluation to drive adaptive management in a biodiversity conservation organization

Rare Pride is a social marketing program that stimulates human behavior change in order to promote biodiversity conservation in critically threatened regions in developing countries. A series of formal evaluation studies, networking strategies, and evaluative inquiries have driven a 20-year process of adaptive management that has resulted in extensive programmatic changes within Pride. This paper describes the types of evaluation that Rare used to drive adaptive management and the changes it caused in Pride's theory-of-change and programmatic structure. We argue that (a) qualitative data gathered from partners and staff through structured interviews is most effective at identifying problems with current programs and procedures, (b) networking with other organizations is the most effective strategy for learning of new management strategies, and (c) quantitative data gathered through surveys is effective at measuring program impact and quality. Adaptive management has allowed Rare to increase its Pride program from implementing about two campaigns per year in 2001 to more than 40 per year in 2009 while improving program quality and maintaining program impact.     

The results framework--an innovative tool for program planning and evaluation.

This article constitutes a case study of the development and implementation of the "results framework," an innovative planning and evaluation tool that is rapidly becoming a standard requirement for United States Agency for International Development (USAID) projects. The framework is used in a USAID-funded regional initiative for HIV/AIDS prevention in Central America. This new program evaluation and monitoring tool provides many advantages over traditional evaluation approaches that use outside consultants to provide midterm and end-of-project evaluations. The results-framework process, which spans the life of the project, provides an opportunity for program staff, donors, partners, and evaluators to work as a team to collect and use rich, longitudinal data for project planning, implementation, and evaluation purposes.     

An innovative medical and dental hygiene clinic for street youth: Results of a process evaluation

Canada has a noteworthy reputation for high quality health care. Nonetheless, street youth are one of our most vulnerable yet underserved populations. Consequently, a medical and dental clinic was created in downtown Ottawa, Ontario to respond to their needs. The purpose of this study is to describe a process evaluation of the clinic during its first year of operation with a focus on program fidelity, dose, reach, and satisfaction. A mixed methods approach was used involving interviews with providers, focus groups with street youth, analysis of Electronic Medical Record (EMR) data, and supplemental information such as document reviews. The evaluation identified areas that were working well along with challenges to program implementation. Areas of concerns and possible solutions were presented to the management team that then helped to plan and make improvements to the clinic. Our evaluation design and working relationship with clinic management promoted the integration of real-time evidence into program improvements.     

Improving productivity in child welfare programs

The project met its objective of finding ways to improve the efficiency of a child welfare program. The agency will continue an intensive evaluation of the program to reassess the procedures' impact on productivity. The project found that, with adaptation, work-methods techniques developed in industry can be useful to child welfare programs. Critical elements in the success of a productivity improvement project include full, visible support by the agency's policy makers and top management; early intensive involvement of affected personnel; wide implementation of recommended improvements; and continued review of their effectiveness. Although no single work-methods project can solve all problems connected with an agency's way of carrying out its work, the process developed by the Texas Department of Human Resources can be applied usefully by both public and private child welfare agencies as they seek ways to improve the efficiency of their programs.     

Research Note—Program Evaluation: Enhancing Academic Quality and Program Integrity During Rapid Off-Campus Growth

By creating a program evaluation model that assesses multiple components from several perspectives, a social work division is able to determine the quality, effectiveness, and integrity of its offerings and support its mission to develop social work students for practice. This task is critical in supporting an off-campus program’s ability to navigate change and uncertainty in higher education. This research note discusses how a small, bachelor of social work program with a dual-delivery system (traditional and community-site) used the Council on Social Work Education reaccreditation process to adapt its program evaluation to ensure academic quality and program integrity during a period of rapid growth.     

Evaluating the design and implementation fidelity of an adapted Plan-Do-Study-Act approach to improve health system performance in a Nigerian state

BackgroundThe Plan-Do-Study-Act (PDSA) cycle is fundamental to many quality improvement (QI) models. For the approach to be effective in the real-world, variants must align with standard elements of the PDSA. This study evaluates the alignment between theory, design and implementation fidelity of a PDSA variant adapted for Nigeria’s health system performance improvement.MethodsAn iterative consensus building approach was used to develop a scorecard evaluating new conceptual indices of design and implementation fidelity of QI interventions (design and implementation index, defects and gaps) based on Taylor’s theoretical framework.ResultsDesign (adaptation) scores were optimal across all standard features indicating that design was well adapted to the typical PDSA. Conversely, implementation fidelity scores were only optimal with two standard features: prediction-based test of change and the use of data over time. The other features, use of multiple iterative cycles and documentation had implementation gaps of 17 % and 50 % respectively.ConclusionThis study demonstrates how both adaptation and implementation fidelity are important for success of QI interventions. It also presents an approach for evaluating other QI models using Taylor’s PDSA assessment framework as a guide, which might serve to strengthen the theory behind future QI models and provide guidance on their appropriate use.     

The Development of a Computerized Information System for Integrated Home Care in The Netherlands

ABSTRACT

In September 1992, The Netherlands Institute of Care and Welfare (NEW) started development of GIRST_S a computerized information system for the new integrated home care services in The Netherlands. These services integrated home nursing and family care services, with many institutions merging to facilitate the process. Integration of services required reorganization of all of the information processes and new technology provided the opportunity for improvement in their quality. These factors of reorganisation and of new technology opportunities stimulated NIZW to develop GIRST, which is designed as a tool to facilitate provision of care. It is not focused on managerial issues but on the information needs and concerns of those who actually provide care. The process of providing home care can be divided into four stages which correspond with the four modules of GIRST: care coordination-care allocation-care in practice-care evaluation.     

关于上海新一轮改革的若干思考

当前是上海加快推进“四个率先”、加快建设“四个中心”和社会主义现代化国际大都市的关键时期,国内外发展的新形势对改革的重点、方式、任务、动力以及实施路径提出了新的要求,上海必须根据当前改革发展的阶段特征,按照顶层设计的改革推进思路,着力围绕转型发展的目标要求,重点在民生改善、服务经济、内外开放、政府改革和改革机制完善等方面实现关键性突破,以带动上海改革的整体推进和持续深化。     

The 90's

Gender differences in AIDS-related heterosexual concerns and sexual behavior with new sexual partners, with an emphasis on condom use and subjective condom experiences, were assessed by a questionnaire survey of middleclass bar patrons, on location, in New York City. Patrons were asked about their number of sexual partners, the extent of their AIDS-related womes and behavioral responses to such concerns, as well as their use of condoms in different sexual situations and their subjective condom experiences. Most people report that AIDS-related concerns have affected their behavior in some way, primarily in the increased use of condoms. Condom use varies according to type of relationship and type of sexual act and results generally in more peace of mind, but less sexual enjoyment. Peace of mind strongly predicts condom use during casual sex for both genders. In terms of gender differences, women worry about AIDS with new sexual partners more than men do and they tend to limit their number of partners more. Women appreciate the use of condoms more than men do, most likely because erection and ejaculatory concerns are fundamental aspects of men's, but not women's, condom experiences. "Worrying about AIDS" predicts condom use during casual sex for men, but not women. More men than women had a condom with them when surveyed.     

Evaluation of a complex, multisite, multilevel grants initiative

The Safe Schools/Healthy Students (SS/HS) national evaluation seeks to assess both the implementation process and the results of the SS/HS initiative, exploring factors that have contributed to or detracted from grantee success. Each site is required to forge partnerships with representatives from education, mental health, juvenile justice, and law enforcement, coordinating and integrating their efforts and working together to contribute to comparable outcomes (e.g., reduced violence and alcohol and drug use, improved mental health services). The evaluation uses multiple data collection techniques (archival data, surveys, site visits, interviews, and focus groups) from a variety of sources (project directors, community partners, schools, and students) over several years. Certain characteristics of the SS/HS initiative represent unique challenges for the evaluation, including the absence of common metrics for baseline, outcome data, and lack of comparison group. A unifying program theory was required to address these challenges and synthesize the large amounts of qualitative and quantitative information collected. This article stresses the role of program theory in guiding the evaluation.     

ISO9000标准族在新建高职院校质量管理中的应用——以柳州铁道职业技术学院为例

十一·五以来,许多新建高职院校在规模扩张的同时,内涵建设却并没有得到有效改善,人才培养质量难以符合市场需要。适时导入ISO9000标准族,构建"全员参与、全面实施、全程控制"质量管理体系,实现依法治校,质量立校的要求,对新建高职院校提升内涵建设水平、提高教育教学质量起到了推动和促进作用。     

Australian Approaches to Resident Classification and Quality Assurance in Residential Care

Summary

This paper begins with an account of the structure of Australia's residential long-term care program, which was divided into two distinct levels of hostel and nursing home care until 1997. In response to changed policy objectives, a number of measures were then taken to create an integrated residential care system. The main measures were the development of a single scale for classification of resident care need and associated funding to replace two previous separate scales, and the implementation of a new quality assurance system, which included new standards for buildings as well as revised standards for care. I give accounts of these measures and the extent to which they have achieved their intended outcomes before proposing some further developments that could see closer links among pre-admission assessment, resident classification, and quality assurance.