Primary healthcare planning,bottleneck analysis and performance improvement: An evaluation of processes and outcomes in a Nigerian context

Operational planning of interventions defines roadmaps, timelines and resources necessary for translating policies into expected health outcomes along the evidence-policy-implementation continuum. However, bottlenecks often hinder the attainment of objectives and the timely delivery of intervention packages leading to sub-optimal performance of health systems. Bottleneck identification, analysis and removal approaches to planning, which requires key stakeholders’ participation, have been recommended to improve health system outcomes in LMICs. This study demonstrates how integration of participatory action research (PAR) within a quality improvement model can help navigate the complexities of health system bottleneck analyses, planning and performance improvement in a Nigerian sub-national context.The study is based on data collected between June 2016 and June 2017, from Chikun LGA in Kaduna State Nigeria. PAR was integrated into a quality improvement model called DIVA (Diagnose-Intervene-Verify-Adjust) applied across selected interventions (eMTCT, Antenatal care, skilled birth attendance, immunization and Integrated Management of Childhood Illnesses). PAR was used to identify and analyse health system bottlenecks, as well as develop, monitor implementation and follow-up on action plans to address them. Evaluations were conducted involving 2 cycles of DIVA. The outputs (bottleneck analysis charts, driver diagrams, operational plans, M/E reports, etc.) from each cycle of the DIVA process were collated and analysed. Bottlenecks identified include availability of human resources for health, availability of health commodities as well as geographical accessibility. These had implications on acceptability and quality of services. Mean Improvements recorded were 20.4%, 14.0% and 10.8% and 11.2%, 7.5%; 5.5% (across eMTCT, maternal health and child health interventions) in the 1 st and 2nd DIVA cycles respectively. This study highlights processes and outcomes of integrating PAR in quality improvement design and operations in health intervention programmes with a focus on health systems strengthening in a Nigerian context. Implementing the DIVA model using a PAR approach may be considered an effective strategy for planning and implementing health interventions in comparable settings.     

Interorganizational network findings from a nationwide cardiovascular disease prevention initiative

ObjectivesTo use network analysis in order to evaluate the effectiveness of interorganizational networks in implementing policy, systems, and environmental interventions for cardiovascular disease prevention throughout the United States.MethodsEvaluators conducted an interorganizational network (ION) survey to examine information sharing and joint planning within organizational relationships in 15 community-based cardiovascular disease prevention partnership networks. Density and betweenness centrality scores at the node- and network-level were calculated for each partnership network using UCINET© network analysis software. Common data patterns were then extracted using a multiple case study format.ResultsNetwork density scores ranged from 0.50 to 1.00 (M = 0.84, SD = 0.14) for information sharing and 0.43–1.00 (M = 0.77, SD = 0.15) for joint planning. Centralization indices ranged from 0.00 to 0.11 (M = 0.04, SD = 0.03), and 0.00-0.17 (M = 0.06, SD = 0.05), respectively. Overall, 73.33 % of communities were successful in meeting their partnership goals.ConclusionsWhen planning and implementing interorganizational networks, high betweenness centrality and more hierarchically structured networks were identified as the most salient partnership characteristics to programmatic success. The network findings were triangulated with previously published qualitative data to provide context. These findings provide valuable insight on how national networks can be designed and leveraged to implement systematic community health projects.     

Can a workbook work? Examining whether a practitioner evaluation toolkit can promote instrumental use

In large-scale, multi-site contexts, developing and disseminating practitioner-oriented evaluation toolkits are an increasingly common strategy for building evaluation capacity. Toolkits explain the evaluation process, present evaluation design choices, and offer step-by-step guidance to practitioners. To date, there has been limited research on whether such resources truly foster the successful design, implementation, and use of evaluation findings. In this paper, we describe a multi-site project in which we developed a practitioner evaluation toolkit and then studied the extent to which the toolkit and accompanying technical assistance was effective in promoting successful completion of local-level evaluations and fostering instrumental use of the findings (i.e., whether programs directly used their findings to improve practice, see Patton, 2008). Forensic nurse practitioners from six geographically dispersed service programs completed methodologically rigorous evaluations; furthermore, all six programs used the findings to create programmatic and community-level changes to improve local practice. Implications for evaluation capacity building are discussed.     

Testing the efficacy of the theory of planned behavior to explain strength training in older adults

The purpose of this study was to use the constructs of the theory of planned behavior (TPB) to gain a better understanding of the factors influencing older adults' participation in strength training. Two hundred men and women age 55 years and older were purposely sampled from seniors' centers in Ontario Canada. Participants completed a TPB questionnaire and reported their current physical activity participation. It was hypothesized that perceived behavioral control followed by attitude would be the strongest determinants of strength-training intentions and that intention would be the strongest determinant of strength-training behavior. Regression analyses revealed that subjective norm and perceived behavioral control explained 42% of the variance in intention and intention explained 40% of the variance in behavior. Gender and current strength-training participation did not significantly moderate the relationship between the TPB variables. The results suggest that interventions targeting subjective norm and perceived control might be helpful in promoting strength-training behavior among older adults.     

An application of the Impact Evaluation Process for designing a performance measurement and evaluation framework in K-12 environments

This article illustrates the application of the Impact Evaluation Process for the design of a performance measurement and evaluation framework for an urban high school. One of the key aims of this framework is to enhance decision-making by providing timely feedback about the effectiveness of various performance improvement interventions. The framework design process is guided by the Impact Evaluation Process, and included the participation of key stakeholders including administrative and teaching staff who all contributed to the performance measurement and evaluation framework design process. Key performance indicators at the strategic, tactical, and operational levels were derived from the school vision, and linked to specific interventions to facilitate the continuous evaluation and improvement process.     

An application of the Impact Evaluation Process for designing a performance measurement and evaluation framework in K-12 environments

This article illustrates the application of the Impact Evaluation Process for the design of a performance measurement and evaluation framework for an urban high school. One of the key aims of this framework is to enhance decision-making by providing timely feedback about the effectiveness of various performance improvement interventions. The framework design process is guided by the Impact Evaluation Process, and included the participation of key stakeholders including administrative and teaching staff who all contributed to the performance measurement and evaluation framework design process. Key performance indicators at the strategic, tactical, and operational levels were derived from the school vision, and linked to specific interventions to facilitate the continuous evaluation and improvement process.     

After family treatment drug court: maternal, infant, and permanency outcomes

This study reports on maternal functioning, infant developmental, and permanency outcomes for 52 families following maternal participation in a family treatment drug court (FTDC) for perinatal substance users. Although the majority of families experienced positive child welfare outcomes, over time, maternal functioning deteriorated and infant developmental concerns were identified. Even when promising interventions like FTDC are used, long-term needs of families affected by perinatal substance use need to be considered and addressed.     

Addressing substance abuse treatment needs of parents involved with the child welfare system

The goal of this paper is to synthesize available data to help guide policy and programmatic initiatives for families with substance abuse problems who are involved with the child welfare system, and identify gaps in the research base preventing further refinement of practices in this area. To date, Family Treatment Drug Court and newly developed home-based substance abuse treatment interventions appear the most effective at improving substance abuse treatment initiation and completion in child welfare populations. Research is needed to compare the efficacy of these two approaches, and examine cost and child well-being indicators in addition to substance abuse treatment and child welfare outcomes.     

The evaluation of healthy city projects in developing countries

In international public health and other disciplines there is a debate about the use of process versus impact (sometimes referred to as outcome) indicators in project evaluations. With growing international and national support for Healthy City Projects in developing countries it is timely to address the issue of how to evaluate such projects. This paper briefly describes process and impact indicators, considers the importance of process indicators given the role of institutional strengthening and capacity building in Healthy City Projects and discusses the balance between locally and internationally generated indicators. A case study of the first Healthy City Project in a developing country in Asia (Chittagong, Bangladesh) illustrates how the project is linked with urban management and can enhance co-operation, co-ordination, motivation, internal organisation, decentralisation and community participation. Finally, some institutional indicators for evaluation are suggested.     

Organizational merger and cultural change for better outcomes: the first five years of the New York State Office of Children and Family Services

Since its creation, New York State Office of Children and Family Services (OCFS) has used child and family development research to bridge the chasm between child welfare and juvenile justice policy and practice. OCFS's major challenges have been to help the staff and stakeholders of the merged child welfare and juvenile justice agencies develop a common culture and design new financial and programmatic infrastructures to promote better outcomes for children and families.     

Counselor confounds in evaluations of vocational rehabilitation methods in substance dependency treatment

Evaluation research on vocational counseling in substance dependency treatment should distinguish between the effects of counselors and counseling methods on clients' employment outcomes. Three experimental designs permit investigation of possible confounds between these types of effects: (a) nested designs (each counselor delivers one counseling method so counselors are nested under methods), (b) crossed designs (each counselor delivers all counseling methods so counselors are crossed with methods), and (c) no-treatment control group designs (experimental group's counselors all deliver the same method). Each design is optimal for one stage of evaluation research. No-treatment control group designs are best for exploratory evaluations of new types of vocational counseling. Nested designs are best for outcome evaluations of different types of interventions. Crossed designs are best for (causal) process evaluations of counseling methods of demonstrated efficacy. Despite the importance of methodological issues and problems, vocational rehabilitation in substance dependency treatment has a greater need for stronger interventions than better evaluation designs.     

Diversity training for the community aged care workers: A conceptual framework for evaluation

Older Australians are an increasingly diverse population, with variable characteristics such as culture, sexual orientation, socioeconomic status, and physical capabilities potentially influencing their participation in healthcare. In response, community aged care workers may need to increase skills and uptake of knowledge into practice regarding diversity through appropriate training interventions. Diversity training (DT) programs have traditionally existed in the realm of business, with little research attention devoted to scientifically evaluating the outcomes of training directed at community aged care workers. A DT workshop has been developed for community aged care workers, and this paper focuses on the construction of a formative evaluative framework for the workshop. Key evaluation concepts and measures relating to DT have been identified in the literature and integrated into the framework, focusing on five categories: Training needs analysis; Reactions; Learning outcomes, Behavioural outcomes and Results The use of a mixed methods approach in the framework provides an additional strength, by evaluating long-term behavioural change and improvements in service delivery. As little is known about the effectiveness of DT programs for community aged care workers, the proposed framework will provide an empirical and consistent method of evaluation, to assess their impact on enhancing older people’s experience of healthcare.     

Conceptual problems of the monitoring and evaluation of qualitative objectives of rural development

Monitoring and evaluation are crucial aspects of all rural development projects in terms of ensuring that objectives are being met and assessing the impact of various interventions. Thus, monitoring and evaluation should be built into a project's organizational and implementation structure and constitute a continuous, systematic project activity. In need of further refinement, however, are techniques for monitoring and evaluating the nonmaterial, nonobjective dimensions of development projects such as community participation. Needed is an approach that is not based exclusively on the measurement of material impact, but is also able to explain what happens in a rural development project that seeks to promote participation. In dealing with participation, there is a concern not only with quantitative results, but also qualitative processes. Toward this end, project beneficiaries should have a role in describing the processes involved, analyzing the results, and making a judgment as to the outcome of project activities. The unmet need in this area is to devise indicators of economic, political, cultural, and vulnerable-group participation in rural development projects and to identify ways such indicators could be used to collect relevant data.     

Zero effects in substance abuse programs. Avoiding false positives and false negatives in the evaluation of community-based programs

The evaluation of community-based programs poses special design and analysis problems. The present article focuses on two major types of errors that can occur in such evaluations: false positive--incorrectly declaring a program to be effective--and false negatives--incorrectly declaring a program to be ineffective. The evaluation of a national demonstration of community-based programs to reduce substance abuse, Fighting Back, is used to illustrate several approaches to reduce the probability of errors. Both those errors that are affected by the design and those by analytic approaches are considered. Ways to assess multiple outcomes and to match the complexity of the program with design and analytic strategies are proposed. Community trials are complex interventions, and, although they can provide very useful information, their outcomes have to be understood in terms of the constructs they test and the contexts within which they are carried out.     

Transforming gender roles in domestic and caregiving work: preliminary findings from engaging fathers in maternal,newborn, and child health in Rwanda

This article draws on Promundo and RWAMREC's programmatic experiences in Rwanda of implementing MenCare+, a gender transformative approach to engaging young and adult men (ages 15–35) in caregiving, maternal, newborn, and child health, and sexual and reproductive health and rights. We present initial results from fathers' groups with more than 600 men, including the impact of participation in these groups on men's participation in care work. The results confirm the importance for practitioners' planning strategies to engage fathers to think beyond men's token participation in care work, to use father participation as an entry point to truly transform gender dynamics within the home. The article provides practical lessons learnt to guide other organisations interested in working with men to transform norms around fatherhood and care work.     

Body image,self-esteem,and behavioral risk for chronic disease among college students: Additional evidence for integrated prevention

Abstract

Objective: To examine associations between two body image constructs (body appreciation and body satisfaction) and five health behaviors (diet, physical activity, weight management, tobacco exposure, and alcohol intake) associated with risk for chronic disease, controlling for self-esteem. Participants: Three hundred and forty-four college students enrolled at a large, public university in the southeastern US. Methods: Students completed an online survey composed of measures of body appreciation, body satisfaction, self-esteem, and frequency of engagement in preventive health behaviors. Hierarchical linear regression models were used to assess relationships between constructs. Results: Body appreciation, but not body satisfaction or self-esteem, significantly and positively predicted engagement in diet-, physical activity-, and weight-related health behaviors. No associations were found for substance use outcomes. Conclusions: Results illuminate health risks among college students and provide additional evidence to support the development of holistic preventive interventions that simultaneously address aspects of mental and physical health among college students.     

The use of evaluation models for decision making: Application to the community anti-crime program

Evaluations are typically designed to provide useful information, including measures of program effectiveness. Although this information is intended to be useful, the general literature on evaluations suggests that instances in which evaluations are explicitly used for decision making are rare. This article discusses the use of decision-theoretic evaluation models, and their applicability in practice as well as in principle. A case study is presented from the evaluation of LEAA's Community Anti-Crime Program, in which a decision analysis based on a decision-theoretic evaluation model influenced a programmatic decision which arose during the conduct of the evaluation.     

Assessing participants’ perceptions on group-based principles for action in community-based health enhancing physical activity programmes: The APEF tool

In community-based health enhancing physical activity (CBHEPA) programmes, group-based principles for action such as active participation, enjoyment, and fostering group processes are widely advocated. However, not much is known about participants’ perceptions of these principles as there are no assessment tools available. Therefore, this article describes the development of the APEF (Active Participation, Enjoyment, and Fostering group processes) tool and reports on its implementation in a Dutch CBHEPA programme. Indicators for the principles have been identified from literature research, interviews with professionals, and secondary analysis of three group interviews with 11 practitioners. To address the identified indicators, the APEF tool was developed, pretested, and used in 10 focus groups with 76 participants.The APEF tool consists of eight statements about group-based principles for action, on which CBHEPA participants vote, followed by in-depth discussion. The voting procedure engages participants. Spider diagrams visualise participants’ perceptions of group-based principles.The APEF tool addresses the challenge of relating group level outcomes to individual outcomes such as physical activity behaviour. The tool facilitates as well as evaluates group-based principles for action, it stimulates dialogue and is culturally sensitive, but it needs strong facilitating skills to manage group dynamics.     

Measuring conflict management,emotional self-efficacy,and problem solving confidence in an evaluation of outdoor programs for inner-city youth in Baltimore,Maryland

Substantial evidence supports the value of outdoor education programs for promoting healthy adolescent development, yet measurement of program outcomes often lacks rigor. Accurately assessing the impacts of programs that seek to promote positive youth development is critical for determining whether youth are benefitting as intended, identifying best practices and areas for improvement, and informing decisions about which programs to invest in.We generated brief, customized instruments for measuring three outcomes among youth participants in Baltimore City Outward Bound programs: conflict management, emotional self-efficacy, and problem solving confidence. Measures were validated through exploratory and confirmatory factor analyses of pilot-testing data from two groups of program participants. We describe our process of identifying outcomes for measurement, developing and adapting measurement instruments, and validating these instruments.The finalized measures support evaluations of outdoor education programs serving urban adolescent youth. Such evaluations enhance accountability by determining if youth are benefiting from programs as intended, and strengthen the case for investment in programs with demonstrated success.     

Exploring the unmet needs for creating an enabling environment for nurturing care to promote migrant child health in Bishkek,Kyrgyzstan: A theory-guided community-based participatory action research

Empirical evidence on community-driven child health promotion programs in disadvantaged migrant populations is limited despite various promotional strategies. Therefore, we implemented a developmental process to shape child health interventions using theory-guided community-based participatory action research (CBPAR) in a migrant community in Bishkek, Kyrgyzstan between 2015 and 2019.The collaborative and iterative CBPAR process was conducted through participatory workshops and focus group discussions with support from trusting and collegial partnerships between community members and the research team. The goal and scope of the intervention was guided by enabling environments for nurturing care, including the four domains of caregivers’ capabilities, empowered communities, supportive services, and enabling policies. Diverse interests and needs identified by community members were aggregated in the theoretical model and reflected in the intervention.Community-driven intervention is perceived as a culturally acceptable, sustainable, sensitive and relevant approach to solve problems. There are several challenges in conducting the CBPAR, such as the effort and time spent on building partnerships, co-learning and mutual understanding, and the power equilibrium involved. Despite this, the success of the CBPAR process provided opportunities for community mobilization, empowerment and sustainability of the intervention. Evaluation of the process and outcomes of the intervention provided community health researchers and practitioners with evidence of the theory-guided community participatory approach.