Advances in collaborative evaluation

Collaborative evaluation is an approach that offers, among others, many advantages in terms of access to information, quality of information gathered, opportunities for creative problem-solving, and receptivity to findings. In the last decade, collaborative evaluation has grown in popularity along with similar participatory, empowerment, and utilization-focused evaluation approaches. Increasingly, it has brought together evaluators and stakeholders from different sectors, disciplines, and cultures to exchange knowledge on how collaboration can be used as a strategic tool for fostering and strengthening evaluation practice. O'Sullivan (2004) and Rodríguez-Campos, 2005, Rodríguez-Campos, 2008 brought forward collaborative evaluation conceptual frameworks that distinguished it from other approaches. This article updates those conceptual frameworks and discusses key collaborative evaluation advances in order to further clarify and facilitate engagement in sound practice.     

A process-oriented approach. Selecting a nursing model for psychiatric nursing.

The nursing literature is replete with articles and books that describe nursing conceptual frameworks and models and encourage their use in clinical, education, and research activities. Although much information exists on the content of nursing models, less has been written about how a model is to be chosen and the process that may facilitate the choice of a model. This article reviews potential benefits and limitations of nursing models and conceptual frameworks and describes a three-phase process for selecting a model for psychiatric nursing practice.     

Family-Centered Practice in the 1990s:

This paper presents a multicultural perspective for family-centered social work practice. Such a perspective uses ethclass and value orientation theory as sensitizing conceptual frameworks for understanding a family's cultural reality. This understanding, used by a multicultural practitioner, leads to the use of approaches most congruent with the multicultural world views of the families served.     

A Qualitative Sociological Approach to Address Issues of Diversity for Social Work

ABSTRACT

This article identifies prevailing conceptual approaches employed to formulate issues of diversity for social work research and practice. It argues that the notions of “welfare” and “diversity” are customarily defined within unduly narrow realms. “Welfare” is normally defined from the ideological perspective of modern mainstream society, even though competing minority ideologies of “welfare” are both possible and evident. “Diversity” is normally organized around the central concept of “ethnicity,” even though the universality of this notion is questionable. In response a research approach to address these limitations is proposed which employs a sociological perspective and qualitative methodology to discern appropriate conceptual frameworks.     

Implementation frameworks in child,youth and family services – Results from a scoping review

Background & objectivesImplementation frameworks are designed to articulate the actions and behaviors considered necessary for successful implementation of interventions, programs or services. Such frameworks have been increasingly used in social services for children, youth and families (which include family and parenting support, out-of-home care (foster care) placements, child protection, family violence, juvenile justice and community services). The purposes of this review were (a) to identify studies employing an implementation framework in this field; (b) map the literature to better understand these frameworks and the ways in which they are being applied; (c) to ascertain the ways in which implementation frameworks are being tested; and (d) to describe the current state of evidence surrounding their use in the field.MethodFor this scoping review, searches of the literature were conducted within PsycINFO, MEDLINE, CINAHL, ASSIA, Embase, Embase Classic, Social Work Abstracts, ERIC and Sociological Abstracts. Databases were searched for published, peer reviewed English language evaluation studies that applied - and reported on this application - implementation frameworks in the child, youth and family service sector. No limits were placed on years. Any type of study design was eligible from single case studies to randomized controlled trials.ResultsOut of a total of 8541 publications located, thirty-three met the inclusion criteria. They included eight frameworks that have been applied in the sector. Few of the identified frameworks were based on rigorous research designs. Common strategies used within the frameworks included staging implementation, key influences (e.g., competencies, organizational factors, leadership), stakeholder identification and engagement, and capacity measurement and building. Rarely were these approaches theoretically grounded or fully developed, and limited information was provided about their characteristics, development or interconnectedness. In short, research underpinning frameworks and their use has been meager, especially considering their proliferation in the field.ConclusionsThis review identifies a need to strengthen the conceptualization of core strategies that are integrated into implementation frameworks, including an articulation of their underlying logic. In the future, implementation science and practice may gain from moving away from comprehensive and complex implementation frameworks towards a more flexible, modular approach to implementation based on the application and combination of effective ‘implementation core strategies’. Future research may also draw a more complete picture of the state of implementation frameworks by expanding search terms to also include other sectors and domains into systematic reviews.     

Three Psychotherapies for Suicidal Adolescents: Overview of Conceptual Frameworks and Intervention Techniques

Suicide is the second leading cause of death among youth, and as many as one in five youth report having had at least one serious thought of suicide in the past year. Despite the enormous emotional pain and suffering associated with suicidal thoughts and behaviors, up to 40 % of suicidal youth never receive treatment. Given that social workers are employed in multiple settings where suicidal children and adolescents are encountered (e.g. schools, homeless shelters, emergency departments, outpatient mental health agencies, private practice), they play a critical role in the identification and treatment of suicidal youth. In the past decade, evidence has emerged that attachment-based family therapy, integrated cognitive behavioral therapy, and dialectical behavior therapy can reduce suicidal ideation and/or suicide attempt in youth. The purpose of this article is to review the theoretical assumptions, conceptual frameworks and key intervention techniques for these three interventions so that clinicians can integrate these approaches into their practice with suicidal youth and families. Implications for practice are integrated throughout the review.     

Implementation of a Collaborative Care Initiative for PTSD and Depression in the Army Primary Care System

Collaborative care models hold promise as a means to improve the Military Health System (MHS) to better meet the mental health needs of U.S. military personnel. This article describes an Army-wide collaborative care initiative, RESPECT-Mil (“Re-engineering Systems of Primary Care Treatment for Depression and PTSD in the Military”), aimed at improving care for soldiers with depression and post traumatic stress disorder. Essential to the implementation of RESPECT-Mil has been the use of a centralized coordinating team to facilitate collaborative care practice and policy activities across the multiple levels of the MHS. The article explores the relevance of collaborative care frameworks to the social work profession and concludes with a discussion of future directions for the social work field to promote greater involvement in the integrated care movement.     

An approach to design long-term monitoring and evaluation frameworks in multi-actor systems—A case in water management

Learning-by-doing and adaptive management require careful monitoring and evaluation of the outcomes of environmental policies and programs under implementation. Selecting relevant indicators is difficult, especially when monitoring over a longer period of time. Further challenges arise when policies are developed as a collaborative effort among multiple actors.This paper discusses an approach to design frameworks for long-term monitoring and evaluation in multi-actor systems. It uses Dynamic Actor Network Analysis (DANA) as an actor-sensitive method to reconstruct program theories. This is combined with elements of assumption-based planning to identify critical assumptions and associated indicators to incorporate the dynamic aspects related to long-term monitoring.An application of this approach is described for a case of water management in the Netherlands. Here, mapping multiple perspectives and identifying critical assumptions helped to broaden the scope of monitoring in important ways. Identifying associated indicators and expectations on their development in response to policy implementation proved more difficult.From this case, it can be concluded that the approach is feasible, useful, but also demanding. However, with continuing trends of networked governance and adaptive management, additional efforts to reflect these trends in monitoring and evaluation, through this and similar approaches, are needed.     

An approach to design long-term monitoring and evaluation frameworks in multi-actor systems--a case in water management

Learning-by-doing and adaptive management require careful monitoring and evaluation of the outcomes of environmental policies and programs under implementation. Selecting relevant indicators is difficult, especially when monitoring over a longer period of time. Further challenges arise when policies are developed as a collaborative effort among multiple actors. This paper discusses an approach to design frameworks for long-term monitoring and evaluation in multi-actor systems. It uses Dynamic Actor Network Analysis (DANA) as an actor-sensitive method to reconstruct program theories. This is combined with elements of assumption-based planning to identify critical assumptions and associated indicators to incorporate the dynamic aspects related to long-term monitoring. An application of this approach is described for a case of water management in the Netherlands. Here, mapping multiple perspectives and identifying critical assumptions helped to broaden the scope of monitoring in important ways. Identifying associated indicators and expectations on their development in response to policy implementation proved more difficult. From this case, it can be concluded that the approach is feasible, useful, but also demanding. However, with continuing trends of networked governance and adaptive management, additional efforts to reflect these trends in monitoring and evaluation, through this and similar approaches, are needed.     

Interdiscursivity and Child Welfare: The Ascent and Durability of Psycho-legalism

This paper examines the knowledges affecting contemporary child welfare policy and practice. Using a number of conceptual frameworks, it seeks to challenge the view of some commentators that a new 'legalism' and a putative concern with 'surface form' has accorded formal psychological knowledge, and hence the 'psy' complex, a diminished and waning significance. The paper argues that, although there have been significant changes in child welfare practice, rumours of the waning of the 'psy' complex have been exaggerated. A detailed analysis of the way the law thinks, and of policy documents and practice guidance reveals both the complex interdiscursivity of the new 'legalism' and the durability of psychological and developmentalist forms of thought.     

Preparing Social Work Graduates for Digital Practice: Ethical Pedagogies for Effective Learning

ABSTRACT

Debate has arisen in social work about the value of online education, with proponents providing examples of innovative approaches to online teaching and those against it citing the need for face-to-face interactions to suitably teach and assess social work skills. At the same time, social workers are increasingly making use of social media, websites, online databases, email, and text messaging services for work with clients. Consequently, social workers must be adequately prepared to make use of online technologies in their practice. Drawing on examples of online interactive activities used at two regional universities and informed by Chi’s conceptual framework of cognitive processes of learning, this article argues that learning via information and communications technology (ICT) helps prepare graduates for emerging digital practice and should be an integral part of social work education. Further, ICT enables educators to monitor the levels and quality of student rehearsal of practice skills and knowledge in interactive learning experiences helping promote competent practice, grounded in social work values.     

Aligning collaborative and culturally responsive evaluation approaches

The authors, three African-American women trained as collaborative evaluators, offer a comparative analysis of collaborative evaluation (O'Sullivan, 2004) and culturally responsive evaluation approaches (Frierson, Hood, & Hughes, 2002; Kirkhart & Hopson, 2010). Collaborative evaluation techniques immerse evaluators in the cultural milieu of the program, systematically engage stakeholders and integrate their program expertise throughout the evaluation, build evaluation capacity, and facilitate the co-creation of a more complex understanding of programs. However, the authors note that without explicit attention to considerations raised in culturally responsive evaluation approaches (for example, issues of race, power, and privilege), the voices and concerns of marginalized and underserved populations may be acknowledged, but not explicitly or adequately addressed. The intentional application of collaborative evaluation techniques coupled with a culturally responsive stance enhances the responsiveness, validity and utility of evaluations, as well as the cultural competence of evaluators.     

Aligning collaborative and culturally responsive evaluation approaches

The authors, three African-American women trained as collaborative evaluators, offer a comparative analysis of collaborative evaluation (O'Sullivan, 2004) and culturally responsive evaluation approaches (Frierson et al., 2002, Kirkhart and Hopson, 2010). Collaborative evaluation techniques immerse evaluators in the cultural milieu of the program, systematically engage stakeholders and integrate their program expertise throughout the evaluation, build evaluation capacity, and facilitate the co-creation of a more complex understanding of programs. However, the authors note that without explicit attention to considerations raised in culturally responsive evaluation approaches (for example, issues of race, power, and privilege), the voices and concerns of marginalized and underserved populations may be acknowledged, but not explicitly or adequately addressed. The intentional application of collaborative evaluation techniques coupled with a culturally responsive stance enhances the responsiveness, validity and utility of evaluations, as well as the cultural competence of evaluators.     

Frameworks for Practice in the Systemic Field: Part 1 — Continuities and Transitions in Family Therapy Knowledge

This is the first of two articles to map the landscape of practice theory in systemic family therapy. In this first article, the representation of knowledge for practice is explored, and an argument is made that while frameworks remain important, the relationship to them is now more conditional and pragmatic. A particular chronology is offered of the development of family therapy practice theory frameworks, beginning with the frameworks that emerged in the 1960s to the 1970s. An analysis is given of the important transitions in the 1980s and three sets of influences in this decade — ecosystemic epistemology, the feminist challenge and postmodernism — are identified. This reading emphasises hidden continuities in the transition, despite the seemingly discontinuous shifts in practice theory from the beginning of the 1980s to the beginning of the 1990s. Context and relationship are identified as the enduring parameters of systemic family therapy knowledge, though understandings of context and relationship have been recast in the contemporary (post‐1990s) practice theory. The second article will explore the four contemporary influential approaches in Australian family therapy — the Milan‐systemic, narrative and solution‐focused frameworks, and the dialogical perspective — and point to intersections in practice ideas and integrative movements.     

Toward Public Relations Theory-Based Study of Public Diplomacy: Testing the Applicability of the Excellence Study

This study tested the applicability of the Excellence Study (L. A. Grunig, J. E. Grunig, & Dozier, 2002) to developing the study of public diplomacy to respond to Signitzer and Coombs's (1992) call for empirical research to "delineate and test" (p. 145) transferable public relations theories and to rigorously examine conceptual convergence between both spheres. This study delineated underdeveloped conceptions of public diplomacy practices and excellence in public diplomacy as a gap preventing theory building for both practices and excellence. It proposed the conceptual frameworks for public relations behavior and excellence in public relations in the Excellence Study to be applicable to developing the underdeveloped conceptions. In testing the applicability, this study examined the fits of 2 measurement models of public relations behavior and excellence in public relations developed from the conceptual frameworks of the Excellence Study. It tested the models with survey data on public diplomacy practices and management collected from 113 embassies in Washington, D.C. The findings showed that the conceptual and measurement frameworks of the Excellence Study are applicable: The 2 measurement models fit the public diplomacy data.     

Enhancing Information Literacy in an Interdisciplinary Collaboration

ABSTRACT

The study examined the impact of an interdisciplinary collaborative on the information literacy of its participants by using legitimate peripheral participation (Lave & Wenger, 1991) as a conceptual framework. The objectives of this exploratory study were to document participants' qualitative gains in information technology knowledge and uses, and to analyze expert involvement in supporting learning by novices. Data were gathered by using a participation observation approach, interviews, and survey. Participants cited more frequently the non-technical aspects of information literacy as being more beneficial than gains in technical know-how. Varying degrees of expert involvement ranging from enabling to instructing were identified. The implications in terms of convening and managing communities of practice and the development of legitimate peripheral participation theory are discussed.     

Identifying Practice Components of Youth Councils: Contributions of Theory

Social workers are involved in numerous efforts to engage youth in programs, communities, and civic life. One potential strategy has focused on engagement and empowerment of youth through the form of youth councils. Multiple theoretical frames have characterized the scholarly literature. This has limited the conceptual coherence of the field. In this paper, we report empirical data on the operation of several youth councils. We analyze the data to identify the implicit frameworks in use and apply the data from our study to sort practice components within frameworks. This effort is designed to improve conceptualization of youth councils, to inform the development of councils, and eventually to improve outcomes of councils.     

Rethinking Girls “At-Risk”

Abstract

This conceptual article critically analyzes influential theoretical frameworks concerning adolescent female development, including psychosocial identity stage theory, person-in-environment theory, and feminist psychology. Based on the strengths and limitations of these approaches, the author introduces social workers to an emerging framework that addresses the gender, race, and class intersections that contextualize adolescent girls' developmental experiences. This perspective also discusses young women's capacity to resist norms and stereotypes in their search for an authentic identity. This innovative theory complements a strengths-based approach to social work and can be used to guide research andpractice with this population.     

Collaborative evaluation of a high school prevention curriculum: How methods of collaborative evaluation enhanced a randomized control trial to inform program improvement

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome variables that can be used for program improvement. Throughout this project there were many examples of client learning from evaluator and evaluator learning from client. This article presents convincing evidence that collaborative evaluation can improve the design, implementation, and findings of the randomized control trial. Throughout this paper, we discuss many examples of good science, good evaluation, and other practical benefits of practicing collaborative evaluation. Ultimately, the authors created the term pre-formative evaluation to describe the period prior to data collection and before program implementation, when collaborative evaluation can inform program improvement.     

Agent-based modeling as a tool for program design and evaluation

Recently, systems thinking and systems science approaches have gained popularity in the field of evaluation; however, there has been relatively little exploration of how evaluators could use quantitative tools to assist in the implementation of systems approaches therein. The purpose of this paper is to explore potential uses of one such quantitative tool, agent-based modeling, in evaluation practice. To this end, we define agent-based modeling and offer potential uses for it in typical evaluation activities, including: engaging stakeholders, selecting an intervention, modeling program theory, setting performance targets, and interpreting evaluation results. We provide demonstrative examples from published agent-based modeling efforts both inside and outside the field of evaluation for each of the evaluative activities discussed. We further describe potential pitfalls of this tool and offer cautions for evaluators who may chose to implement it in their practice. Finally, the article concludes with a discussion of the future of agent-based modeling in evaluation practice and a call for more formal exploration of this tool as well as other approaches to simulation modeling in the field.