Custody Evaluations: Ethics in Training Evaluators

Little has been written addressing ethical considerations in the training of custody evaluators and mediators. This article summarizes the ethical principles involved in training new evaluators and in providing ongoing training to experienced evaluators. Model programs for both types of training are presented.     

Analyzing mental health evaluation: moral and ethical dimensions

Now that evaluators have been sensitized to the importance of moral and ethical issues in their work, it is time to move beyond generalities and examine the moral and ethical implications of specific evaluation models in specific settings. This paper proposes a framework that can be used to examine moral and ethical dimensions of evaluation and illustrates it by analyzing a selected model of mental health evaluation. Such a systematic identification of moral issues can improve evaluation practice both proactively, by shaping the training of evaluators, and retrospectively, by contributing to meta-evaluation.     

Private thoughts and public actions:: A commentary on articles dealing with ethical issues in evaluation

This commentary represents a personal reaction to seven articles on ethical issues in evaluation. The author of the commentary begins with two assumptions: (1) Professional evaluators might not have the vocabulary and constructs to analyze their thoughts and actions in ethical terms, and (2) professional evaluators have often confronted troubling situations that probably contain ethical issues embedded within them. Consequently, she queries the seven articles individually and collectively in terms of the following questions: 1. What is the range of ethical problems I might encounter as an evaluator; and how do I recognize the ethical characteristics of professional dilemmas? 2. How do I describe troubling evaluation problems I have encountered that seem to have an ethical component? 3. What are some approaches I have used or might use in thinking about a proper course of action? 4. How do I know if, in a given situation, I have behaved ethically? 5. Where do we go from here?     

Demands of the joint committee's standards for educational evaluation

The study examines demands placed on evaluators by the the Joint Committee on Educational Evaluation's standards. Twenty-two experts in evaluation rated the compatibility of implementing pairs of standards or rated individual standards on a series of scales. Multidimensional scaling of the compatibility judgments revealed five demands of ethical practice: evaluators must attend to the contextual and political circumstances of the evaluation; they must exhibit professionalism in their work; they must conduct the evaluation openly and consult widely; evaluators must meet canons of social science practice; and finally, evaluators' research activities must attend to the specific evaluation setting. The demand to attend to the contextual and political circumstances of the evaluation presents the greatest potential for conflict with other demands and is in greatest need of development.     

Ethical practice and evaluation of interventions in crime and justice. The moral imperative for randomized trials

In considering the ethical dilemmas associated with randomized experiments, scholars ordinarily focus on the ways in which randomization of treatments or interventions violates accepted norms of conduct of social science research more generally or evaluation of crime and justice questions more specifically. The weight of ethical judgment is thus put on experimental research to justify meeting ethical standards. In this article, it is argued that just the opposite should be true, and that in fact there is a moral imperative for the conduct of randomized experiments in crime and justice. That imperative develops from our professional obligation to provide valid answers to questions about the effectiveness of treatments, practices, and programs. It is supported by a statistical argument that makes randomized experiments the preferred method for ruling out alternative causes of the outcomes observed. Common objections to experimentation are reviewed and found overall to relate more to the failure to institutionalize experimentation than to any inherent limitations in the experimental method and its application in crime and justice settings. It is argued that the failure of crime and justice practitioners, funders, and evaluators to develop a comprehensive infrastructure for experimental evaluation represents a serious violation of professional standards.     

Psychological Testing in Child Custody Evaluations With Ethnically Diverse Families: Ethical Concerns and Practice Recommendations

Given increasing ethnic diversity of divorcing families, it is more important than ever for family court evaluators to understand the ethical concerns and principles underlying the use of psychological testing with ethnic family members. Major professional organizations have promulgated the guidelines of child custody evaluation with an imperative to consider cultural factors in the evaluation. However, there is a paucity of literature that addresses ethnicity and psychological testing in child custody evaluations. This article addresses three major ethical domains: (a) competence, (b) test fairness, and (c) test interpretation when psychological tests are used. Practice recommendations provide guidelines to ensure ethical standards and competence in using psychological testing with ethnically diverse families in child custody and other family court evaluations.     

Ethical problems in program evaluation: Advice for trapped evaluators

This paper argues that ethical problems in program evaluation are increased when conflicting or incompatible evaluation models are applied concurrently. Three models are illustrated: an Amelioration Model, generating better information for a program's own decision makers; an Accountability Model, focusing on public data disclosure and citizen participation in evaluation; and an Advocacy Model, in which the evaluation is designed to advance the program's interests in the competition for resources. Ethical problems inherent in each model, and arising from combining models are presented, along with case examples and discussion. The authors suggest several directions that program evaluators can consider to reduce their vulnerability to ethical problems.     

Ethical Violations: What Can and Does Go Wrong in Child Custody Evaluations?

Child custody evaluations can have a significant impact on the adjustment and well-being of children and family members. Custody evaluators must be qualified and held to the highest professional standards. The authors of this article review custody evaluations and often witness inadequate and unprofessional evaluations. In this article, ethical mistakes seen regularly are highlighted. Many mistakes occur when cases are complex and include allegations of child maltreatment or intimate partner violence. Therefore, it is important that custody evaluators be trained in family and child psychology, attachment dynamics, psychological assessment, child abuse, and intimate partner violence. Child custody work needs to be monitored closely to ensure that mistakes can be eliminated in order to avoid unnecessary harm to children and families.     

Avoiding evaluation cooptation. Lessons from a renal dialysis evaluation.

An evaluation of the impact of terminating contracts to transport low-income patients for renal dialysis demonstrates how evaluators may become apologists for management and conduct studies too narrow for effective decision making. The evaluation found that patients continued to receive treatment; their death rates had not increased. Changes in their economic and emotional status were not studied; consequently, the evaluators erroneously concluded that the clients had not been seriously harmed. The article reviews the process the evaluation used. The authors conclude that to avoid cooptation, evaluators need independent information available through conducting and analyzing pilot studies and meeting with program constituents. To avoid overly narrow studies, the next level of management should participate in their design. To use the information effectively, evaluators need to see themselves as advisors rather than technicians.     

Looking Forward Not Looking Back: The Evaluation of Community Living

This article outlines the development of evaluative methods within the field of mental handicap services. It suggests that evaluators have too often taken for granted the tacit assumptions underlying service models, using these to determine the criteria for evaluating success. It argues that evaluators should be more explicit about the values they adopt, and what patterns of life services should be enabling people to achieve. By more closely linking evaluation criteria to what people with a mental handicap should achieve, evaluation can become more effective in promoting positive change in service models. The article describes an approach developed by the authors for the evaluation of the 'All-Wales Strategy for the Development of Services for Mentally Handicapped People'.     

Relational responsibilities in responsive evaluation

This article explores how we can enhance our understanding of the moral responsibilities in daily, plural practices of responsive evaluation. It introduces an interpretive framework for understanding the moral aspects of evaluation practice. The framework supports responsive evaluators to better understand and handle their moral responsibilities. A case is introduced to illustrate our argument.Responsive evaluation contributes to the design and implementation of policy by working with stakeholders and coordinating the evaluation process as a relationally responsible practice. Responsive evaluation entails a democratic process in which the evaluator fosters and enters a partnership with stakeholders. The responsibilities of an evaluator generally involve issues such as ‘confidentiality’, ‘accountability’ and ‘privacy’. The responsive evaluator has specific responsibilities, for example to include stakeholders and vulnerable groups and to foster an ongoing dialogue. In addition, responsive evaluation involves a relational responsibility, which becomes present in daily situations in which stakeholders express expectations and voice demands. In our everyday work as evaluators, it is difficult to respond to all these demands at the same time. In addition, this article demonstrates that novice evaluators experience challenges concerning over- and underidenfitication with stakeholders. Guidelines and quality criteria on how to act are helpful, but need interpretation and application to the unique situation at hand.     

Involving communities in the evaluation of programmes with ‘at risk’ children and young people

This article explores the relationships between evaluators and those who take part in community‐based programmes. It draws specifically on experience gained from the continuing evaluation of the Communities that Care (CTC) programme in Britain. Different community perspectives towards evaluation are discussed, highlighting the challenges these create for evaluators. In the final sections examples are used to identify different strategies used on the CTC evaluation, showing how ‘trust’, ‘boundary setting’, ‘independence’ and ‘identification of benefits’ are central to including community participants in evaluation. Copyright © 2001 John Wiley & Sons, Ltd.     

Pathways to becoming an internal evaluator: Perspectives from the Australian non-government sector

There is a lack of clarity around intra-organisational evaluation roles and pathways into these roles in non-government organisations (NGOs). This article presents three auto-narratives from the authors who are working as internal evaluators in the NGO sector. We examine this phenomenon of role ambiguity by exploring our evaluation journeys and struggles to find identities in the formal evaluation community. Findings from the auto-narratives identify implications for the evaluation field regarding professionalisation. This article explores how aspects of professionalisation, such as clarification of roles and tasks of internal evaluators, could facilitate their recruitment, assess credibility and guide career trajectory. Elucidating internal evaluation career pathways contributes to the evaluation discipline by providing information relevant for evaluation capacity building, evaluator training, and the professionalisation movement.     

Risks and wrongs in social science research. An evaluator's guide to the IRB

Having an Institutional Review Board (IRB) review and monitor the use of human subjects is now fundamental to ethical research. Yet social scientists appear increasingly frustrated with the process. This article aims to assist evaluators struggling to understand and work with IRBs. The author theorizes why IRBs frustrate and insists there is only one remedy: We must accept the legitimacy of IRB review and (a) learn more about IRB regulations, imperatives, and the new pressures on them; and (b) educate IRBs about social scientific methodologies and empirically demonstrable risks. A research agenda and tips are offered.     

Community-Based Program Research

ABSTRACT

At first glance, the worlds of program professionals and evaluators may seem quite separate. However, there are common issues whose resolution will enhance both program development and research on programs. Program professionals and evaluators have a great deal to learn from each other. As that learning occurs, both program delivery and evaluation research will benefit. Both are concerned about matters of validity, whether it pertains to the nature of the program intervention itself or to the nature of the assessment of the program intervention. To reinforce the development of program evaluation in partnership, this paper discusses key points about evaluation research relevant to both program professionals and researchers. These include the contextual influences on a program, the “readiness” of a program for evaluation, and whether the evaluation research “works” for the program. Underlying our discussion is a call for the development of partnerships around research on programs. In that light, this article elaborates the process steps that should be taken to build program evaluation partnerships, including a discussion of what evaluators and program professionals need to know about each other, and their respective values, interests, and professional perspectives.     

Informed Consent Agreements: Standards of Care for Digital Social Work Practices

The inclusion of remote digital practices (e-mail, texting, blogging, social media) to deliver social work services raises unique ethical and legal challenges for client and practitioner. Competent care necessitates providing clients with information that would materially affect clients’ decisions whether to assume the risks, benefits, and conditions associated with e-practices. As standards of care for distance service provision have not been firmly established, social workers are duty bound to be well-informed about current laws, duties, and obligations including specific rules governing the practice of social work in the states where they deliver remote services. This article identifies distinct threats to privacy and confidentiality associated with e-practices and offers suggested content for a comprehensive informed consent policy.     

Mixed-methods library evaluation integrating the patron,library, and external perspectives: The case of Namibia regional libraries

The article presents a mixed-methods evaluation of regional libraries in Namibia, which incorporates three perspectives: the patron perspective (library users), the library perspective (library staff, management, and related officials), and the external perspective (including evaluators and monitoring data). Seven data collection methods were used: patron surveys, patron panel studies, focus group discussions, key informant and staff interviews, secondary data analysis, media analysis, and observations. The goal of the evaluation was to assess library performance for both formative and summative purposes by addressing evaluation questions on areas such as library services, use, and operations. Building upon the literature review of how mixed-methods approaches can contribute to library evaluation, the aim of this article is to show how a mixed-methods evaluation can be designed to examine multi-faceted library performance and to illustrate how the evaluation design allows information complementarity and can be utilized to present diverse viewpoints of the above three perspectives. The evaluation design, analysis process, and lessons learned from this study may be useful to evaluators engaged in evaluation of public services or programs (including public libraries) that examine multiple aspects of service performance and involve a variety of stakeholders.     

Being practical, being safe: doing evaluations in contested spaces

This final article summarizes and synthesizes the full special edition. This volume questioned evaluation as philosophy, enterprise, and practice from the point of view of doing evaluation under conditions of social violence, disruption, and division. In this final article, we clarify the issues and problems which challenge the professional practice of evaluation and propose guiding questions for evaluators designing and doing evaluation in worlds like these. We attend to the consequences for evaluation and evaluator of choosing to inhabit these violent worlds as an evaluator.