Custody Evaluations: Ethics in Training Evaluators

Little has been written addressing ethical considerations in the training of custody evaluators and mediators. This article summarizes the ethical principles involved in training new evaluators and in providing ongoing training to experienced evaluators. Model programs for both types of training are presented.     

Analyzing mental health evaluation: moral and ethical dimensions

Now that evaluators have been sensitized to the importance of moral and ethical issues in their work, it is time to move beyond generalities and examine the moral and ethical implications of specific evaluation models in specific settings. This paper proposes a framework that can be used to examine moral and ethical dimensions of evaluation and illustrates it by analyzing a selected model of mental health evaluation. Such a systematic identification of moral issues can improve evaluation practice both proactively, by shaping the training of evaluators, and retrospectively, by contributing to meta-evaluation.     

The value of unattractive characteristics in social entitlements.

Social programs legislated as entitlements are attractive to promoters of health and welfare services because of the programmatic security and stability such a status offers. Yet existing entitlements and other publicly funded services routinely fail to exhibit the high performance standards that an entitlement status requires. Such "flaws" as confusion over purpose, obstructive procedures, inefficient administration, and cost overruns can also serve important functional and political purposes. Therefore, administrators and evaluators who are in a position to witness these discrepancies will sometimes find their roles as manager and critic compromised. Instead of pursuing policies that would satisfy entitlement principles, they may only be in a position to implement and defend the "flaws" as reasonable compromises. There is the risk, however, that the public will remain skeptical about overall programmatic results despite optimistic evaluation reports. More careful legislation, more disciplined administration, and more neutral evaluation will be required if human services are to retain the public's confidence.     

Lessons learned about collaborative evaluation using the Capacity for Applying Project Evaluation (CAPE) framework with school and district leaders

Schools, districts, and state-level educational organizations are experiencing a great shift in the way they do the business of education. This shift focuses on accountability, specifically through the expectation of the effective utilization of evaluative-focused efforts to guide and support decisions about educational program implementation. In as much, education leaders need specific guidance and training on how to plan, implement, and use evaluation to critically examine district and school-level initiatives. One specific effort intended to address this need is through the Capacity for Applying Project Evaluation (CAPE) framework. The CAPE framework is composed of three crucial components: a collection of evaluation resources; a professional development model; and a conceptual framework that guides the work to support evaluation planning and implementation in schools and districts. School and district teams serve as active participants in the professional development and ultimately as formative evaluators of their own school or district-level programs by working collaboratively with evaluation experts.The CAPE framework involves the school and district staff in planning and implementing their evaluation. They are the ones deciding what evaluation questions to ask, which instruments to use, what data to collect, and how and to whom results should be reported. Initially this work is done through careful scaffolding by evaluation experts, where supports are slowly pulled away as the educators gain experience and confidence in their knowledge and skills as evaluators. Since CAPE engages all stakeholders in all stages of the evaluation, the philosophical intentions of these efforts to build capacity for formative evaluation strictly aligns with the collaborative evaluation approach.     

Lessons learned about collaborative evaluation using the Capacity for Applying Project Evaluation (CAPE) framework with school and district leaders

Schools, districts, and state-level educational organizations are experiencing a great shift in the way they do the business of education. This shift focuses on accountability, specifically through the expectation of the effective utilization of evaluative-focused efforts to guide and support decisions about educational program implementation. In as much, education leaders need specific guidance and training on how to plan, implement, and use evaluation to critically examine district and school-level initiatives. One specific effort intended to address this need is through the Capacity for Applying Project Evaluation (CAPE) framework. The CAPE framework is composed of three crucial components: a collection of evaluation resources; a professional development model; and a conceptual framework that guides the work to support evaluation planning and implementation in schools and districts. School and district teams serve as active participants in the professional development and ultimately as formative evaluators of their own school or district-level programs by working collaboratively with evaluation experts. The CAPE framework involves the school and district staff in planning and implementing their evaluation. They are the ones deciding what evaluation questions to ask, which instruments to use, what data to collect, and how and to whom results should be reported. Initially this work is done through careful scaffolding by evaluation experts, where supports are slowly pulled away as the educators gain experience and confidence in their knowledge and skills as evaluators. Since CAPE engages all stakeholders in all stages of the evaluation, the philosophical intentions of these efforts to build capacity for formative evaluation strictly aligns with the collaborative evaluation approach.     

A model of impacts of research partnerships in health and social services

Alliances have long been an important strategy for health promotion and community development. They are now becoming an important approach to generating research that is thought to have real-world relevance and easy applicability because of the involvement of community members. A comprehensive, dynamic model of community impacts of research partnerships in social or health services is presented. This process-oriented, systems model outlines three major domains of mid-term impact (Enhanced Knowledge, Enhanced Research Skills, and Use of Information) that correspond to the core functions of collaborative research partnerships discussed in the literature (i.e. knowledge generation, research education and training, and knowledge sharing). The model will assist research partners, intended recipients, and funders to understand and evaluate the real-world impacts of community–university research partnerships. The model provides research partnerships with a tool to demonstrate their accountability and to improve their operations and impacts, and evaluators with a tool to guide planning and evaluation efforts.     

The Impact of Experiential Exercises on Communication and Relational Skills in a Suicide Prevention Gatekeeper-Training Program for College Resident Advisors

Abstract

Objective: In an effort to identify students at risk for suicide, many colleges are implementing suicide prevention training for campus gatekeepers. This study evaluated the efficacy of a 3-hour, experiential-based gatekeeper training that included an emphasis on enhancing communication skills and relational connection in addition to the didactic foci of standard gatekeeper training. Participants: Sixty-five college student resident advisors (RAs) were trained with Campus Connect. Methods: The training was dismantled to examine the specific contribution of experiential exercises on training outcomes. Results: Compared to didactic training alone, following participation in experiential exercises RAs’ training outcome scores exhibited additional improvement on the Suicide Intervention Response Inventory–2 and a 14-item self-report measure of self-efficacy for specific suicide- and crisis-related knowledge and skills. Conclusions: In gatekeeper training, experiential exercises emphasizing awareness and empathic responding and practice of these skills contribute to an improvement in crisis response skills above and beyond that of didactic training alone.     

Bridging the gap: The impact of home visiting programs for orphans and vulnerable children on social grant uptake in South Africa

Cash transfer programs hold significant potential to mitigate the economic burdens resulting from the HIV epidemic and enhance the wellbeing of affected children. South Africa offers two cash transfers designed specifically to benefit children: the Child Support Grant, for low income families with children, and the Foster Child Grant, for children living outside of parental care. Given the high proportion of HIV-affected children who qualify for these grants, increasing grant access among eligible families is a natural objective for many programs targeting orphans and vulnerable children. We present results from a quasi-experimental study examining differences in grant uptake over a two year period among 1487 children enrolled in one of two types of supportive home visiting programming: volunteer-based or paraprofessional. The study also examined related outcomes including household food security and children's access to basic educational and material needs. Results show that programs staffed with trained paraprofessionals who received training, compensation and other support were significantly more effective at linking families to social grants for children. Controlling for important covariates, at follow-up participants in the paraprofessional model programs were nearly three times as likely as volunteer-based service recipients to have access to the highest grant they were eligible to receive. Grant receipt was also positively associated with household food security and children's obtainment of basic educational and material resources. Effective strategies for promoting social grant access among HIV-affected households therefore have the potential to yield significant improvements in wellbeing for orphans and vulnerable children.     

A comparison of linear and systems thinking approaches for program evaluation illustrated using the Indiana Interdisciplinary GK-12

Logic models are based on linear relationships between program resources, activities, and outcomes, and have been used widely to support both program development and evaluation. While useful in describing some programs, the linear nature of the logic model makes it difficult to capture the complex relationships within larger, multifaceted programs. Causal loop diagrams based on a systems thinking approach can better capture a multidimensional, layered program model while providing a more complete understanding of the relationship between program elements, which enables evaluators to examine influences and dependencies between and within program components. Few studies describe how to conceptualize and apply systems models for educational program evaluation. The goal of this paper is to use our NSF-funded, Interdisciplinary GK-12 project: Bringing Authentic Problem Solving in STEM to Rural Middle Schools to illustrate a systems thinking approach to model a complex educational program to aid in evaluation. GK-12 pairs eight teachers with eight STEM doctoral fellows per program year to implement curricula in middle schools. We demonstrate how systems thinking provides added value by modeling the participant groups, instruments, outcomes, and other factors in ways that enhance the interpretation of quantitative and qualitative data. Limitations of the model include added complexity. Implications include better understanding of interactions and outcomes and analyses reflecting interacting or conflicting variables.     

Practices and challenges in educational program evaluation in the Asia-Pacific region: results of a Delphi study

While educational program evaluation has become more important in recent years because of increasing governmental demands for accountability, little is known about the development of and issues in regard to this topic in the Asia-Pacific region. The findings from a Delphi study conducted in a number of relevant countries are provided in this article. Thirty panelists participated in three rounds of iterations to identify current status, challenges, and concerns in the near future. The results should be of value to educational evaluators, researchers, and decision makers.     

Ethical Violations: What Can and Does Go Wrong in Child Custody Evaluations?

Child custody evaluations can have a significant impact on the adjustment and well-being of children and family members. Custody evaluators must be qualified and held to the highest professional standards. The authors of this article review custody evaluations and often witness inadequate and unprofessional evaluations. In this article, ethical mistakes seen regularly are highlighted. Many mistakes occur when cases are complex and include allegations of child maltreatment or intimate partner violence. Therefore, it is important that custody evaluators be trained in family and child psychology, attachment dynamics, psychological assessment, child abuse, and intimate partner violence. Child custody work needs to be monitored closely to ensure that mistakes can be eliminated in order to avoid unnecessary harm to children and families.     

Assessing Competency in A Specialized Program

This paper discusses the results of fusing different social work education trends in planning a relevant curriculum. These trends include a focus on practice-specific competencies, specialization of graduate programs, and the emerging role of students as consumers and evaluators of their educational experiences. Recommendations for designing processes of evaluation are offered.     

Recognizing experiential knowledge in the process of qualification of peer helpers: an analysis of an educational program for service users in France

ABSTRACT

This article analyzes an innovative training program enabling the qualification of peer carers, working within the sector of ‘social inclusion’. Since the French national debate on social work conducted within the ‘Estates General’ of 2015, peer workers have become key players in training programs in virtue of their experiential knowledge and their understanding of issues related to the process of social exclusion. This article addresses the role of peer helpers’ experiential knowledge in the training process. Because of their ‘experiential’ and ‘empirical’ knowledge about questions linked to the process of exclusion, peer helpers have become key players in social work teams and within social institutions, thus contributing to new methods of socio-educational intervention. What impact will this recognition of peer helpers’ and service users’ experiential knowledge have on education in social work? This article gives an account of an 8-month training program for peer helpers examined on the methodological level through a process of Participant observation, and based on data from comprehensive interviews carried out with a panel of peer helpers.     

Aligning collaborative and culturally responsive evaluation approaches

The authors, three African-American women trained as collaborative evaluators, offer a comparative analysis of collaborative evaluation (O'Sullivan, 2004) and culturally responsive evaluation approaches (Frierson, Hood, & Hughes, 2002; Kirkhart & Hopson, 2010). Collaborative evaluation techniques immerse evaluators in the cultural milieu of the program, systematically engage stakeholders and integrate their program expertise throughout the evaluation, build evaluation capacity, and facilitate the co-creation of a more complex understanding of programs. However, the authors note that without explicit attention to considerations raised in culturally responsive evaluation approaches (for example, issues of race, power, and privilege), the voices and concerns of marginalized and underserved populations may be acknowledged, but not explicitly or adequately addressed. The intentional application of collaborative evaluation techniques coupled with a culturally responsive stance enhances the responsiveness, validity and utility of evaluations, as well as the cultural competence of evaluators.     

Aligning collaborative and culturally responsive evaluation approaches

The authors, three African-American women trained as collaborative evaluators, offer a comparative analysis of collaborative evaluation (O'Sullivan, 2004) and culturally responsive evaluation approaches (Frierson et al., 2002, Kirkhart and Hopson, 2010). Collaborative evaluation techniques immerse evaluators in the cultural milieu of the program, systematically engage stakeholders and integrate their program expertise throughout the evaluation, build evaluation capacity, and facilitate the co-creation of a more complex understanding of programs. However, the authors note that without explicit attention to considerations raised in culturally responsive evaluation approaches (for example, issues of race, power, and privilege), the voices and concerns of marginalized and underserved populations may be acknowledged, but not explicitly or adequately addressed. The intentional application of collaborative evaluation techniques coupled with a culturally responsive stance enhances the responsiveness, validity and utility of evaluations, as well as the cultural competence of evaluators.     

Career Pathways and Integrated Instruction: A National Program Review of I-BEST Implementations

Contextualized educational programs nested within career pathways provide instruction aimed at improving academic and vocational skills in the context of learning about a specific career or occupation. When the emphasis is on vocational training, contextualized education is commonly referred to as integrated instruction. This review provides an overview of the conceptual motivation and major elements of the career pathways and contextualized education approaches, with an emphasis on integrated instruction. This conceptual framework is used as the basis for conducting a detailed review of the important elements, operation, and empirical evidence of various local implementations of the Integrated Basic Education and Skills Training (I-BEST) model. We focus on variants of the I-BEST model, initially conceived and implemented in Washington, because it is the most widely known application of these approaches. Although several evaluations of Washington’s program found it to be more effective than traditional alternatives, we find little empirical evidence in support of the short-term local variants implemented elsewhere. This review provides important considerations for program administrators and policymakers as well as observations regarding gaps in knowledge regarding program operation and outcomes that deserve additional research.     

The team approach: Developing an experiential knowledge base for the treatment of the pathological gambler

It is not surprising that developing professional competence in treating the pathological gambler is a major problem today, given the scarcity of treatment programs for pathological gambling. This paper explores a learning model which weaves together the experiential knowledge of the recovering pathological gambler with the skills of professional mental health clinicians. An example of this model is the team approach originated at the Johns Hopkins Compulsive Gambling Counseling Center in 1979 and which was evolved into an interdisciplinary treatment model at Taylor Manor Hospital. An examination of this evolving treatment approach is analyzed, reviewed and updated with a focus first on the use of the team approach as a paradigm for developing an experiential knowledge base in the treatment of selected disorders, and second, its use as a model for newly developing treatment programs.     

Are cost-inclusive evaluations worth the effort?

Relative costs are as important as relative effectiveness when choosing between program alternatives or among a set of programs competing for scarce funds. Nevertheless, the number of cost-inclusive evaluations remains comparatively small. This article presents the results of three first-time cost-inclusive evaluations each performed by an experienced evaluator. Each evaluator performed a different type of cost-inclusive evaluation using different tobacco control programs as examples: "standard" cost-effectiveness analysis, threshold or break-even analysis, and a simulation model. Results are presented in terms of the challenges faced, and informational and insight benefits gained, as well as in terms of program cost-effectiveness. All three evaluators agreed that the benefits from performing cost-inclusive evaluations are well worth time and effort involved. They also found that this type of evaluation provides abundant information that can be used to improve program effectiveness and cost-effectiveness.     

Techniques for the analysis of geographic data in evaluation

Although geographic relationships are often an important aspect of a program or system under evaluation, these relationships have frequently been ignored by evaluators. Standard techniques do exist, however, for the analysis of geographic data. This paper presents three such techniques, Geocode Analysis, Trend Surface Analysis, and Social Area Analysis, and discusses their use in evaluation. With these techniques, evaluators can more adequately deal with the questions of geographic relationships which arise in many studies.