The Father Friendly Initiative within Families: Using a logic model to develop program theory for a father support program

The transition to fatherhood, with its numerous challenges, has been well documented. Likewise, fathers’ relationships with health and social services have also begun to be explored. Yet despite the problems fathers experience in interactions with healthcare services, few programs have been developed for them. To explain this, some authors point to the difficulty practitioners encounter in developing and structuring the theory of programs they are trying to create to promote and support father involvement (Savaya, R., & Waysman, M. (2005). Administration in Social Work, 29(2), 85), even when such theory is key to a program's effectiveness (Chen, H.-T. (2005). Practical program evaluation. Thousand Oaks, CA: Sage Publications). The objective of the present paper is to present a tool, the logic model, to bridge this gap and to equip practitioners for structuring program theory. This paper addresses two questions: (1) What would be a useful instrument for structuring the development of program theory in interventions for fathers? (2) How would the concepts of a father involvement program best be organized? The case of the Father Friendly Initiative within Families (FFIF) program is used to present and illustrate six simple steps for developing a logic model that are based on program theory and demonstrate its relevance.     

Using logic model and visualization to conduct portfolio evaluation

Portfolio evaluation is the evaluation of multiple projects with a common purpose. While logic models have been used in many ways to support evaluation, and data visualization has been used widely to present and communicate evaluation findings, adopting logic models for portfolio evaluation and using data visualization to share findings simultaneously is surprisingly limited in the literature. With the data from a sample portfolio of 209 projects which aims to improve the system of early care and education (ECE), this study illustrated how to use logic model and data visualization techniques to conduct a portfolio evaluation by answering two evaluation questions: “To what extent are the elements of a logic model (strategies, sub-strategies, activities, outcomes, and impacts) reflected in the sample portfolio?” and “Which dominant paths through the logic model were illuminated by the data visualization technique?” For the first question, the visualization technique illuminated several dominant strategies, sub-strategies, activities, and outcomes. For the second question, our visualization techniques made it convenient to identify critical paths through the logic model. Implications for both program evaluation and program planning were discussed.     

Defining,illustrating and reflecting on logic analysis with an example from a professional development program

Program designers and evaluators should make a point of testing the validity of a program's intervention theory before investing either in implementation or in any type of evaluation. In this context, logic analysis can be a particularly useful option, since it can be used to test the plausibility of a program's intervention theory using scientific knowledge. Professional development in public health is one field among several that would truly benefit from logic analysis, as it appears to be generally lacking in theorization and evaluation. This article presents the application of this analysis method to an innovative public health professional development program, the Health Promotion Laboratory. More specifically, this paper aims to (1) define the logic analysis approach and differentiate it from similar evaluative methods; (2) illustrate the application of this method by a concrete example (logic analysis of a professional development program); and (3) reflect on the requirements of each phase of logic analysis, as well as on the advantages and disadvantages of such an evaluation method. Using logic analysis to evaluate the Health Promotion Laboratory showed that, generally speaking, the program's intervention theory appeared to have been well designed. By testing and critically discussing logic analysis, this article also contributes to further improving and clarifying the method.     

Devil is in the details: Using logic models to investigate program process

Theory-based logic models are commonly developed as part of requirements for grant funding. As a tool to communicate complex social programs, theory based logic models are an effective visual communication. However, after initial development, theory based logic models are often abandoned and remain in their initial form despite changes in the program process. This paper examines the potential benefits of committing time and resources to revising the initial theory driven logic model and developing detailed logic models that describe key activities to accurately reflect the program and assist in effective program management. The authors use a funded special education teacher preparation program to exemplify the utility of drill down logic models. The paper concludes with lessons learned from the iterative revision process and suggests how the process can lead to more flexible and calibrated program management.     

Policies supporting participation in leisure activities for children and youth with disabilities in Canada: from policy to play

Children with disabilities are at risk for lower participation in leisure. This study maps and analyzes disability-related leisure policies in Canada. Policy mapping and analysis were performed based on scope, social justice, disability models, and use of evidence. Few policies use scientific evidence, and their scope in supporting participation is often limited to organized sports, transportation, and financial incentives/exemptions. Few policies exist to support leisure promotion for children with disabilities. Action plans, benchmarking, and uptake measurement are rarely identified. Research-based evidence can be used to support socially-just policies to promote participation in leisure. Families should be able to identify mechanisms to guarantee access to their rights. Researchers, service providers, and families can use this framework of policy mapping and analysis to identify gaps where evidence can support policy development, to use these policies where services are needed, and to foster participation of children with disabilities in leisure.     

Using a participatory evaluation design to create an online data collection and monitoring system for New Mexico's Community Health Councils

We present the collaborative development of a web-based data collection and monitoring plan for thirty-two county councils within New Mexico's health council system. The monitoring plan, a key component in our multiyear participatory statewide evaluation process, was co-developed with the end users: representatives of the health councils. Guided by the Institute of Medicine's Community, Health Improvement Process framework, we first developed a logic model that delineated processes and intermediate systems-level outcomes in council development, planning, and community action. Through the online system, health councils reported data on intermediate outcomes, including policy changes and funds leveraged. The system captured data that were common across the health council system, yet was also flexible so that councils could report their unique accomplishments at the county level. A main benefit of the online system was that it provided the ability to assess intermediate, outcomes across the health council system. Developing the system was not without challenges, including creating processes to ensure participation across a large rural state; creating shared understanding of intermediate outcomes and indicators; and overcoming technological issues. Even through the challenges, however, the benefits of committing to using participatory processes far outweighed the challenges.     

Brief online surveys to monitor and evaluate facilitated peer support groups for caregivers of children with special needs

BackgroundThere have been few systematic evaluations of experiences of participating in peer support groups for parents and other caregivers of children with special needs. In Australia, facilitated groups are available to caregivers in community settings, through a nationally funded program, MyTime. Mechanisms for ongoing monitoring and evaluation have not yet been instituted.AimTo establish whether brief, online surveys can be used for monitoring and evaluating peer support groups for caregivers of children with special needs.MethodsTwo brief, online surveys, with both fixed-choice and open-ended questions, were developed. All caregivers who attended any MyTime group during a 1-month period were invited to participate.ResultsOf 89 caregivers who expressed interest in participating, 54 and 31 respondents completed respectively. Respondents represented a variety of backgrounds and circumstances. Responses revealed both positive and negative aspects of group participation. Linked data on expectations and experiences provided important feedback for the program.ConclusionBrief, online surveys are a suitable mechanism for ongoing monitoring and evaluation of peer support group programs for caregivers.     

Using video vignettes in research and program evaluation for people with intellectual and developmental disabilities: A case study of the Leadership for Empowerment and Abuse Prevention (LEAP) project

People with intellectual and developmental disabilities (IDD) are often excluded from meaningful participation in research and program evaluation for various reasons, including protocols and measures that are inaccessible for people with varying cognitive and communication abilities. Emancipatory research models emphasize the importance of inclusive research practices. Video vignettes are a promising tool for research and program evaluation with people with IDD because they are standardized, they use visual imagery rather than relying on written or verbal communication, and they allow for distance from sensitive topics. The Leadership for Empowerment and Abuse Prevention (LEAP) project used video vignettes to evaluate a healthy relationship program for people with IDD. The authors discuss the process of piloting various protocols and measures, which then ultimately led to the use of video vignettes in the evaluation.